Human-centered Physical Intelligence (HPI) Lab

The HPI Lab is part of the Department of Artificial Intelligence at Hanyang University ERICA.
Our central research question is how physical AI agents can reliably perceive, act, and interact in the real world, autonomously adapt to new environments and tasks, and continually improve itself through experience, so that it can safely work alongside people.
Our research addresses this question by studying the core components of physical intelligence: (1) 3D scene understanding from multi-modal sensors, (2) representations and world models that generalize beyond the training distribution, and (3) action and decision making that remain reliable in changing and unseen environments. We apply this physical intelligence to robot manipulation, navigation, and human-robot interaction in domains such as manufacturing, logistics, autonomous fabs, and household assistance.

Our research topics are (not limited to):

Multi-modal Robot Action Learning: adaptable and generalizable action models that map multi-modal sensory input to physical actions for manipulation and navigation tasks.
World Models & Model-based Planning: predictive models of how environments evolve under an agent's actions, used for planning and decision making.
Causal & Cross-modal Representation Learning: representations that capture underlying structure rather than spurious correlations, so that models transfer reliably across modalities and domains.
3D Visual Perception and Understanding: how physical AI agents perceive, map, and reason about complex 3D spaces.
Human-AI-Robot Interaction: seamless collaboration frameworks between humans and autonomous systems.

Open Positions

We are looking for highly motivated Undergraduate Interns and Graduate Students (M.S., Ph.D., Integrated) to pioneer the next generation of Physical AI. Apply now!

News (Full List)

Sep 2026	Prof. Jang gave a talk on Multi-modal Robot Learning at KAIST CTGS.
Sep 2026	Prof. Jang will serve as an Area Chair for the International Conference on Learning Representations (ICLR) 2027.
Sep 2026	Welcome Heeyeon Cho to the HPI Lab as an undergraduate intern.
Sep 2026	HPI Lab has been awarded an IITP AI Star Fellowship grant (AI최고급신진연구자지원사업, MSIT/IITP, 2026.07~2031.12). 🎉
Aug 2026	HPI Lab has been awarded an NRF basic research grant (개인기초연구-핵심연구(기본연구B), MSIT/NRF, 2026.09~2027.08). 🎉
Aug 2026	Prof. Jang gave a talk on Multi-modal Robot Learning at S.LSI, Samsung Electronics.
Aug 2026	HPI Lab has been selected to participate in the Artificial Intelligence Innovation Graduate School Program (인공지능혁신인재양성사업, MSIT/IITP, 2026.08~2031.12). 🎉
Jun 2026	A paper about “A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation” is sucessfully presented at ICRA 2026.
May 2026	Welcome Minchan Kim, Gisu Park, Duyeon Won, Su Lee, and Sangho Jung to the HPI Lab as undergraduate interns.
Apr 2026	Prof. Jang gave a talk on Human-centered Physical AI at Kakao Mobility.

Selected Publications (Full List)

ICRA

DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation

Xiongfeng Peng, Jiaqian Yu, Dingzhe Li, Yixiang Jin, Xu Lu, Yamin Mao, Chao Zhang, Weiming Li, Sujin Jang, Dongwook Lee, and Daehyun Ji

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2026

Bib PDF

@inproceedings{peng2026damvla,
  title = {DAM-VLA: A Dynamic Action Model-Based Vision-Language-Action Framework for Robot Manipulation},
  author = {Peng, Xiongfeng and Yu, Jiaqian and Li, Dingzhe and Jin, Yixiang and Lu, Xu and Mao, Yamin and Zhang, Chao and Li, Weiming and Jang, Sujin and Lee, Dongwook and Ji, Daehyun},
  booktitle = {Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)},
  year = {2026},
}

NeurIPS

Active Test-time Vision-Language Navigation

Heeju Ko, Sungjune Kim, Gyeongrok Oh, Jeongyoon Yoon, Honglak Lee, Sujin Jang, Seungryong Kim, and Sangpil Kim

In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), 2025

Bib PDF

@inproceedings{ko2025active,
  title = {Active Test-time Vision-Language Navigation},
  author = {Ko, Heeju and Kim, Sungjune and Oh, Gyeongrok and Yoon, Jeongyoon and Lee, Honglak and Jang, Sujin and Kim, Seungryong and Kim, Sangpil},
  booktitle = {Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS)},
  year = {2025},
}

ICML

Test-time Adaptation for Online Vision-Language Navigation with Feedback-based Reinforcement Learning

Sungjune Kim^*, Gyeongrok Oh^*, Heeju Ko, Daehyun Ji, Dongwook Lee, Byung-Jun Lee, Sujin Jang^†, and Sangpil Kim^†

In Proceedings of the International Conference on Machine Learning (ICML), 2025

Bib PDF

@inproceedings{kim2025test,
  title = {Test-time Adaptation for Online Vision-Language Navigation with Feedback-based Reinforcement Learning},
  author = {Kim, Sungjune and Oh, Gyeongrok and Ko, Heeju and Ji, Daehyun and Lee, Dongwook and Lee, Byung-Jun and Jang, Sujin and Kim, Sangpil},
  booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
  pages = {30654--30671},
  year = {2025},
}

CVPR

3D Occupancy Prediction with Low-Resolution Queries via Prototype-aware View Transformation

Gyeongrok Oh^*, Sungjune Kim^*, Heeju Ko, Hyung-gun Chi, Jinkyu Kim, Dongwook Lee, Daehyun Ji, Sungjoon Choi, Sujin Jang^†, and Sangpil Kim^†

In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 2025

Bib PDF

@inproceedings{oh20253d,
  title = {3D Occupancy Prediction with Low-Resolution Queries via Prototype-aware View Transformation},
  author = {Oh, Gyeongrok and Kim, Sungjune and Ko, Heeju and Chi, Hyung-gun and Kim, Jinkyu and Lee, Dongwook and Ji, Daehyun and Choi, Sungjoon and Jang, Sujin and Kim, Sangpil},
  booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
  pages = {17134--17144},
  year = {2025},
}

NeurIPS

Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation

Nayeon Kim^*, Hongje Seong^*, Daehyun Ji, and Sujin Jang^†

In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), 2024

Bib PDF

@inproceedings{kim2024unveiling,
  title = {Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation},
  author = {Kim, Nayeon and Seong, Hongje and Ji, Daehyun and Jang, Sujin},
  booktitle = {Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS)},
  pages = {111358--111381},
  year = {2024},
}

NeurIPS

STXD: Structural and Temporal Cross-Modal Distillation for Multi-View 3D Object Detection

Sujin Jang^*, Dae Ung Jo^*, Sung Ju Hwang, Dongwook Lee, and Daehyun Ji

In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), 2023

Bib PDF

@inproceedings{jang2023stxd,
  title = {STXD: Structural and Temporal Cross-Modal Distillation for Multi-View 3D Object Detection},
  author = {Jang, Sujin and Jo, Dae Ung and Hwang, Sung Ju and Lee, Dongwook and Ji, Daehyun},
  booktitle = {Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS)},
  year = {2023},
}

NeurIPS

DaDA: Distortion-aware Domain Adaptation for Unsupervised Semantic Segmentation

Sujin Jang, Joohan Na, and Dokwan Oh

In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS-Oral), 2022

Oral Bib PDF

Selected as one of the oral presentations

@inproceedings{jang2022dada,
  title = {DaDA: Distortion-aware Domain Adaptation for Unsupervised Semantic Segmentation},
  author = {Jang, Sujin and Na, Joohan and Oh, Dokwan},
  booktitle = {Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS-Oral)},
  volume = {35},
  pages = {18681--18693},
  year = {2022},
}