Research Scientist · NVIDIA
I am a Research Scientist at NVIDIA in Song Han’s group. I received my PhD from Oxford VGG, advised by Andrew Zisserman. Before my PhD, I graduated from the Turing Class at Peking University. I see understanding, reasoning, generation, and interaction as complementary facets of intelligence in the 3D physical world, each informing and strengthening the others.
World Models · Physical AI
My research spans efficient world foundation and generative models for physical understanding and embodied interaction. I work on vision-language reasoning, efficient and agentic foundation models, and reinforcement learning, diffusion and video representations, and data curation. My experience also includes dataset and benchmark design, robotic manipulation, and navigation.
Efficient spatial reasoning with small vision-language models. Curated reasoning data, supervised fine-tuning, and reinforcement learning improve visual and amodal grounding.
@inproceedings{egm2026, title = {EGM: Efficient Visual Grounding Language Models}, author = {Zhan, Guanqi and Li, Changye and Liu, Zhijian and Lu, Yao and Wu, Yi and Han, Song and Zhu, Ligeng}, booktitle = {ECCV}, year = {2026}, url = {https://arxiv.org/abs/2601.13633} }Do video foundation models learn the physics behind motion? Probing generative and self-supervised representations for elasticity, viscosity, and friction, with PhysVid evaluating transfer from simulated to real-world videos.
@inproceedings{idpp2026, title = {Inferring Dynamic Physical Properties from Video Foundation Models}, author = {Zhan, Guanqi and Ma, Xianzheng and Xie, Weidi and Zisserman, Andrew}, booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (VGBE)}, year = {2026}, url = {https://arxiv.org/abs/2510.02311} }Connecting visual evidence to metric 3D space. Grounded visual chain-of-thought and monocular 3D grounding help vision-language models reason about object locations, distances, and spatial relationships.
@inproceedings{gr3d2026, title = {Grounded 3D-Aware Spatial Vision-Language Modeling}, author = {Cheng, An-Chieh and Fu, Yang and Ji, Yatai and Zhu, Ligeng and Zhan, Guanqi and Zhang, Zhuoyang and Yang, Zhaojing and Han, Song and Lu, Yao and Molchanov, Pavlo and Murali, Vidya Nariyambut and Kautz, Jan and Wang, Xiaolong and Yin, Hongxu and Liu, Sifei}, booktitle = {CVPR}, year = {2026}, url = {https://arxiv.org/abs/2605.30307} }Grasping a chosen object in a partially observed world. An occlusion-aware benchmark and a target–scene model combine shape completion with clutter context for robust target-driven robotic grasping.
@article{targo2026, title = {TARGO and TARGO-Net: Benchmarking Target-driven Object Grasping under Occlusions}, author = {Xia, Yan and Ding, Ran and Qin, Ziyuan and Zhan, Guanqi and Zhou, Kaichen and Yang, Long and Dong, Hao and Cremers, Daniel}, journal = {International Journal of Computer Vision}, year = {2026}, url = {https://arxiv.org/abs/2407.06168} }Adapting frozen vision-language foundation models efficiently. Text-conditioned visual prompts, hard-sample mining, and curated training data align visual representations with language queries and improve retrieval generalization.
@inproceedings{elip2025, title = {ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval}, author = {Zhan, Guanqi and Liu, Yuanpei and Han, Kai and Xie, Weidi and Zisserman, Andrew}, booktitle = {International Conference on Content-Based Multimedia Indexing}, year = {2025}, url = {https://arxiv.org/abs/2502.15682} }What physical knowledge emerges from large-scale pretraining? A common protocol probes generative and discriminative models for geometry, materials, support, lighting, occlusion, and depth, connecting learned representations to 3D physical understanding.
@inproceedings{physical2024, title = {A General Protocol to Probe Large Vision Models for 3D Physical Understanding}, author = {Zhan, Guanqi and Zheng, Chuanxia and Xie, Weidi and Zisserman, Andrew}, booktitle = {NeurIPS}, year = {2024}, url = {https://arxiv.org/abs/2310.06836} }Inferring complete objects beyond what is visible. Pretrained Stable Diffusion features support amodal segmentation, while a 3D-driven data pipeline creates real-image ground truth for evaluating completion with MP3D-Amodal.
@inproceedings{amodal2024, title = {Amodal Ground Truth and Completion in the Wild}, author = {Zhan, Guanqi and Zheng, Chuanxia and Xie, Weidi and Zisserman, Andrew}, booktitle = {CVPR}, year = {2024}, url = {https://arxiv.org/abs/2312.17247} }From language and visual observations to embodied decisions. Dynamic Chain-of-Navigation and multi-sourced value maps guide robots through unexplored environments under diverse instructions, without navigation-specific training.
@inproceedings{pmlr-v270-long25b, title = {InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment}, author = {Long, Yuxing and Cai, Wenzhe and Wang, Hongcheng and Zhan, Guanqi and Dong, Hao}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {2049--2060}, year = {2025}, volume = {270}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, url = {https://proceedings.mlr.press/v270/long25b.html} }Reasoning about physical dependencies before acting. Local dynamics propagate through a support graph to infer which objects support others and plan step-by-step object retrieval in cluttered real-world scenes.
@inproceedings{clutter2024, title = {Broadcasting Support Relations Recursively from Local Dynamics for Object Retrieval in Clutters}, author = {Li, Yitong and Wu, Ruihai and Lu, Haoran and Ning, Chuanruo and Shen, Yan and Zhan, Guanqi and Dong, Hao}, booktitle = {Robotics: Science and Systems}, year = {2024}, url = {https://arxiv.org/abs/2406.02283} }Learning where a robot can act under occlusion. Environment-aware affordances incorporate surrounding objects and robot constraints, using contrastive learning to connect 3D observations with feasible articulated-object manipulation.
@inproceedings{env2023, title = {Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions}, author = {Wu, Ruihai and Cheng, Kai and Shen, Yan and Ning, Chuanruo and Zhan, Guanqi and Dong, Hao}, booktitle = {NeurIPS}, year = {2023}, url = {https://arxiv.org/abs/2309.07510} }Generating coherent 3D structures from unassembled parts. Score-based modeling captures diverse plausible assemblies, with a fast predictor–corrector sampler for efficient generation of part configurations.
@inproceedings{score2023, title = {Score-PA: Score-based 3D Part Assembly}, author = {Cheng, Junfeng and Wu, Mingdong and Zhang, Ruiyuan and Zhan, Guanqi and Wu, Chao and Dong, Hao}, booktitle = {BMVC}, year = {2023}, url = {https://arxiv.org/abs/2309.04220} }Understanding the scene structure behind occlusion. Joint reasoning over an object, its occluder, and its occludee improves detection and segmentation of partially hidden objects.
@inproceedings{tri2022, title = {A Tri-Layer Plugin to Improve Occluded Detection}, author = {Zhan, Guanqi and Xie, Weidi and Zisserman, Andrew}, booktitle = {BMVC}, year = {2022}, url = {https://arxiv.org/abs/2210.10046} }From a set of parts to a coherent 3D object. Dynamic graph reasoning learns relationships between parts and iteratively refines their poses, generating plausible assemblies without semantic part labels.
@inproceedings{dgl2020, title = {Generative 3D Part Assembly via Dynamic Graph Learning}, author = {Huang, Jialei and Zhan, Guanqi and Fan, Qingnan and Mo, Kaichun and Shao, Lin and Chen, Baoquan and Guibas, Leonidas and Dong, Hao}, booktitle = {NeurIPS}, year = {2020}, url = {https://arxiv.org/abs/2006.07793} }Controllable generation through disentangled representations. Separating label-specific and label-independent features enables discrete attributes and reference images to guide fine-grained manipulation and cross-domain interpolation.
@article{dlgan2019, title = {DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation}, author = {Zhan, Guanqi and Zhao, Yihao and Zhao, Bingchan and Yuan, Haoqi and Chen, Baoquan and Dong, Hao}, journal = {arXiv preprint arXiv:1911.09943}, year = {2019}, url = {https://arxiv.org/abs/1911.09943} }VGG, University of Oxford
Yale University
UC Berkeley
Peking University
VGG, University of Oxford & Continental AG
Song Han’s group
Efficient and agentic foundation models, world models, and Physical AI.
Visual Geometry Group (VGG)
Supervised by Andrew Zisserman.
Turing Class.
Reviewer for CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, AAAI, and ACM MM.
NeurIPS 2025 Top Reviewer.
Beyond research
Inspired by Alyosha Efros, I love hiking along the Thames Path. I’m looking forward to exploring more trails in the United States.
Away from research, I also enjoy cooking, swimming, singing, playing the piano, and camping.
