I am a second-year Ph.D. student in Computer Science at UIUC, co-advised by Prof. Shenlong Wang and Prof. Alexander Schwing. I am grateful to be selected as Amazon AI PhD Fellow and Siebel Scholar during my graduate study.

My research lies in 3D vision, generative modeling, and multimodal learning. Previously, I worked on reconstructing, interacting with, and simulating the physical world. Currently, I am particularly interested in:

  • Multimodal Perception: perceive the world through signals beyond pixels
  • Unified Representation Learning: end-to-end learning objectives that generalize across modalities
Sep 2026

Awarded the Amazon AI PhD Fellowship 2026–2027. 🎉

Oct 2025

Awarded the Amazon AI PhD Fellowship 2025–2026. 🎉

Aug 2025

Started my Ph.D. program in Computer Science at UIUC! 🌽

Jun 2025

PhysTwin was accepted to ICCV 2025.

Sep 2024

Awarded Siebel Scholar, Class of 2025 (USD $35,000). 🎉

* indicates equal contribution.

NeurIPS 2025

HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video

Hongchi Xia, Chih-Hao Lin, Hao-Yu Hsu, Quentin Leboutet, Katelyn Gao, Michael Paulitsch, Benjamin Ummenhofer, Shenlong Wang

We reconstruct simulation-ready and interactable digital twin assets from a single video.

ICCV 2025

PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos

Hanxiao Jiang, Hao-Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, Yunzhu Li

We optimize a spring-mass physics model of deformable objects and integrate it with 3D Gaussian Splatting for real-time re-simulation with rendering.

3DV 2025

AutoVFX: Physically Realistic Video Editing from Natural Language Instructions

Hao-Yu Hsu, Chih-Hao Lin, Albert J. Zhai, Hongchi Xia, Shenlong Wang

A system that generates dynamic, physically realistic visual effects (VFX) on a single video solely from text-based editing instructions.

NeurIPS 2022

SPoVT: Semantic Prototype Variational Transformer for Dense Point Cloud Semantic Completion

Sheng-Yu Huang*, Hao-Yu Hsu*, Yu-Chiang Frank Wang

A point cloud semantic completion framework that completes partial point clouds of 3D objects with a variational Transformer.

CVPR 2022

NeurMiPs: Neural Mixture of Planar Experts for View Synthesis

Zhi-Hao Lin, Wei-Chiu Ma*, Hao-Yu Hsu*, Yu-Chiang Frank Wang, Shenlong Wang

Uses an efficient 3D planar representation to model the geometry and appearance of a scene for novel view synthesis.

Research Assistant

Vision & Learning Lab, National Taiwan University

Sep. 2021 – Feb. 2023

Working with Prof. Yu-Chiang Frank Wang and Prof. Shao-Hua Sun on 3D vision and robot learning.

  • I love various forms of sport. I often play basketball 🏀 in my spare time, and also enjoy baseball ⚾, swimming 🏊, weightlifting 🏋️, and cycling 🚴.