Yilun Chen

Research Scientist, Embodied AI Center at Shanghai AI Laboratory

I work on robotic foundation models, 3D vision, and embodied AI. My recent research aims to advance the frontiers of robotic foundation models.

Before joining Shanghai AI Laboratory, I completed my Ph.D. in the Department of Computer Science and Engineering at The Chinese University of Hong Kong, advised by Prof. Jiaya Jia.

Robotic Foundation Models
3D Vision
Autonomous Driving

Open Positions

We are actively seeking motivated research fellows and interns with experience or strong interest in robotics and 3D vision. If you are interested in robotic foundation models, especially topics such as world models, 3D perception, embodied manipulation, and generalist robot learning, send me an email with your background, interests, and representative work.

Recent Highlights

Mar 2026
StarVLA was named a top-10 open-source project in ModelScope's EAI-100 list.
Feb 2026
Our team RoboCola placed 2nd out of 62 in the RoCo Challenge at AAAI 2026. See the challenge leaderboard.
Feb 2026
Four papers were accepted to ICRA 2026.
Jan 2026
Four VLA papers were accepted to ICLR 2026, including ST4VLA, a spatial-training follow-up of InternVLA-M1.
Nov 2025
CronusVLA was accepted by AAAI 2026 as an oral presentation.
Oct 2025
Co-organized the Workshop and Challenge on Multimodal Robot Learning in Physical Worlds.
Sep 2025
Released InternVLA-M1, a spatially guided VLA framework for generalist robots.
Oct 2024
PointLLM was selected as an ECCV 2024 Best Paper Candidate.

Mar 2026
StarVLA was named a top-10 open-source project in ModelScope's EAI-100 list.
Feb 2026
Our team RoboCola placed 2nd out of 62 in the RoCo Challenge at AAAI 2026. See the challenge leaderboard.
Feb 2026
Four papers were accepted to ICRA 2026.
Jan 2026
Four VLA papers were accepted to ICLR 2026, including ST4VLA, a spatial-training follow-up of InternVLA-M1.
Nov 2025
CronusVLA was accepted by AAAI 2026 as an oral presentation.
Oct 2025
Co-organized the Workshop and Challenge on Multimodal Robot Learning in Physical Worlds.
Sep 2025
Released InternVLA-M1, a spatially guided VLA framework for generalist robots.
Mar 2025
GenManip and RoboGround were accepted by CVPR 2025.
Oct 2024
PointLLM was selected as an ECCV 2024 Best Paper Candidate.
Sep 2024
Three papers were accepted by NeurIPS 2024 and one paper was accepted by CoRL 2024.
Jul 2024
One paper was accepted by ECCV 2024.
Aug 2023
Code for FocalFormer3D was released.
Jul 2023
FocalFormer3D was accepted by ICCV 2023.
Mar 2023
FocalFormer3D ranked 1st on the nuScenes LiDAR 3D detection and 3D tracking leaderboards.
Sep 2022
One paper was accepted by NeurIPS 2022.
Aug 2022
DSGN++ was accepted by T-PAMI 2022 and code was released.
Mar 2022
Two papers were accepted by CVPR 2022.
Apr 2020
Code for DSGN was released.
Mar 2020
DSGN was accepted by CVPR 2020.
Jun 2019
Fast Point R-CNN was accepted by ICCV 2019.
Feb 2018
CPN was accepted by CVPR 2018.
Oct 2017
Won 1st Place in the COCO 2017 Keypoint Challenge.

Selected Publications

Technical Report, 2025

InternVLA-M1: A Spatially Grounded Foundation Framework for Generalist Robot Policy

InternVLA-M1 Team, Yilun Chen†

Dominated Hugging Face Robotics Trending with 6 of the top 8 models in September 2025.

Project Paper Code

ICRA 2026

Re³Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation

Xiaoshen Han, Minghuan Liu, Yilun Chen†, Junqiu Yu, Xiaoyang Lyu, Yang Tian, Bolun Wang, Weinan Zhang, Jiangmiao Pang

Project Paper Code

ICLR 2026

RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation

Hao Li*, Ziqin Wang*, Zi-han Ding, Shuai Yang, Yilun Chen†, Yang Tian, Xiaolin Hu, Tai Wang, Dahua Lin, Feng Zhao, Si Liu, Jiangmiao Pang

Project Paper Code

ICLR 2026

ST4VLA: Spatially Guided Training for Vision-Language-Action Models

Jinhui Ye*, Fangjing Wang*, Ning Gao*, Junqiu Yu*, Yangkun Zhu, Bin Wang, Jinyu Zhang, Weiyang Jin, Yanwei Fu, Feng Zheng, Yilun Chen†, Jiangmiao Pang

Project Paper

ICLR 2026

InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation

Shuai Yang*, Hao Li*, Bin Wang, Yilun Chen†, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, Yiyi Liao, Jiangmiao Pang†

Code Paper Simpler-Instruct Benchmark

ICLR 2026

X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model

Jinliang Zheng*, Jianxiong Li*, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, Ya-Qin Zhang, Jiangmiao Pang, Jingjing Liu, Tai Wang, Xianyuan Zhan

Project Paper Code

AAAI 2026 Oral

CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation

Hao Li*, Shuai Yang*, Yilun Chen†, Xinyi Chen, Xiaoda Yang, Yang Tian, Hanqing Wang, Tai Wang, Dahua Lin, Feng Zhao, Jiangmiao Pang†

Code Paper Simpler-OR Benchmark

CVPR 2025

GenManip: LLM-driven Simulation for Generalizable Instruction-Following Manipulation

Ning Gao*, Yilun Chen*, Shuai Yang*, Xinyi Chen*, Yang Tian, Hao Li, Haifeng Huang, Hanqing Wang, Tai Wang, Jiangmiao Pang

Code Project Paper

NeurIPS 2024

Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers

Haifeng Huang*, Yilun Chen*, Zehan Wang*, Rongjie Huang, Runsen Xu, Tai Wang, Yang Zhao, Jiangmiao Pang, Zhou Zhao

Ranked 1st place on the ScanRefer localization benchmark in September 2024.
Ranked 1st place on the Scan2Cap benchmark in September 2024.

Code Paper

ECCV 2024 Oral

PointLLM: Empowering Large Language Models to Understand Point Clouds

Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, Dahua Lin

ECCV 2024 Best Paper Candidate.

Paper Code Project

ICCV 2023

FocalFormer3D: Focusing on Hard Instance for 3D Object Detection

Yilun Chen, Zhiding Yu, Yukang Chen, Shiyi Lan, Anima Anandkumar, Jiaya Jia, Jose M. Alvarez

Ranked 1st place on the nuScenes LiDAR 3D detection leaderboard in March 2023.
Ranked 1st place on the nuScenes LiDAR 3D tracking leaderboard in March 2023.

Paper Code

T-PAMI 2022

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

Yilun Chen, Shijia Huang, Shu Liu, Bei Yu, Jiaya Jia

Ranked 1st among camera-based approaches on the KITTI 3D detection leaderboard in November 2021.
Its multi-modal variant VoCo ranked 1st on the KITTI 3D detection leaderboard for Car in May 2022.

Paper Code

CVPR 2020

DSGN: Deep Stereo Geometry Network for 3D Object Detection

Yilun Chen, Shu Liu, Xiaoyong Shen, Jiaya Jia

Ranked 1st among camera-based approaches on the KITTI 3D detection leaderboard in November 2019.

Paper Project Code

CVPR 2018

Cascaded Pyramid Network for Multi-Person Pose Estimation

Yilun Chen*, Zhicheng Wang*, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, Jian Sun

Champion of the MS-COCO 2017 Keypoint Detection Challenge.
Ranked 1st on the COCO keypoint detection leaderboard in October 2017.

Paper Code BibTeX

Technical Report, 2025

InternVLA-M1: A Spatially Grounded Foundation Framework for Generalist Robot Policy

InternVLA-M1 Team, Yilun Chen†

Dominated Hugging Face Robotics Trending with 6 of the top 8 models in September 2025.

Project Paper Code

ICRA 2026

Re³Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation

Xiaoshen Han, Minghuan Liu, Yilun Chen†, Junqiu Yu, Xiaoyang Lyu, Yang Tian, Bolun Wang, Weinan Zhang, Jiangmiao Pang

Project Paper Code

ICLR 2026

RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation

Hao Li*, Ziqin Wang*, Zi-han Ding, Shuai Yang, Yilun Chen†, Yang Tian, Xiaolin Hu, Tai Wang, Dahua Lin, Feng Zhao, Si Liu, Jiangmiao Pang

Project Paper Code

ICLR 2026

ST4VLA: Spatially Guided Training for Vision-Language-Action Models

Jinhui Ye*, Fangjing Wang*, Ning Gao*, Junqiu Yu*, Yangkun Zhu, Bin Wang, Jinyu Zhang, Weiyang Jin, Yanwei Fu, Feng Zheng, Yilun Chen†, Jiangmiao Pang

Project Paper

ICLR 2026

InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation

Shuai Yang*, Hao Li*, Bin Wang, Yilun Chen†, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, Yiyi Liao, Jiangmiao Pang†

Code Paper Simpler-Instruct Benchmark

ICLR 2026

X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model

Project Paper Code

AAAI 2026 Oral

CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation

Hao Li*, Shuai Yang*, Yilun Chen†, Xinyi Chen, Xiaoda Yang, Yang Tian, Hanqing Wang, Tai Wang, Dahua Lin, Feng Zhao, Jiangmiao Pang†

Code Paper Simpler-OR Benchmark

CVPR 2025

GenManip: LLM-driven Simulation for Generalizable Instruction-Following Manipulation

Ning Gao*, Yilun Chen*, Shuai Yang*, Xinyi Chen*, Yang Tian, Hao Li, Haifeng Huang, Hanqing Wang, Tai Wang, Jiangmiao Pang

Code Project Paper

NeurIPS 2024

Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers

Haifeng Huang*, Yilun Chen*, Zehan Wang*, Rongjie Huang, Runsen Xu, Tai Wang, Yang Zhao, Jiangmiao Pang, Zhou Zhao

Ranked 1st place on the ScanRefer localization benchmark in September 2024.
Ranked 1st place on the Scan2Cap benchmark in September 2024.

Code Paper

ECCV 2024 Oral

PointLLM: Empowering Large Language Models to Understand Point Clouds

Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, Dahua Lin

ECCV 2024 Best Paper Candidate.

Paper Code Project

ICCV 2023

FocalFormer3D: Focusing on Hard Instance for 3D Object Detection

Yilun Chen, Zhiding Yu, Yukang Chen, Shiyi Lan, Anima Anandkumar, Jiaya Jia, Jose M. Alvarez

Ranked 1st place on the nuScenes LiDAR 3D detection leaderboard in March 2023.
Ranked 1st place on the nuScenes LiDAR 3D tracking leaderboard in March 2023.

Paper Code

T-PAMI 2022

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

Yilun Chen, Shijia Huang, Shu Liu, Bei Yu, Jiaya Jia

Ranked 1st among camera-based approaches on the KITTI 3D detection leaderboard in November 2021.
Its multi-modal variant VoCo ranked 1st on the KITTI 3D detection leaderboard for Car in May 2022.

Paper Code

NeurIPS 2022

Unifying Voxel-based Representation with Transformer for 3D Object Detection

Yanwei Li, Yilun Chen, Xiaojuan Qi, Zeming Li, Jian Sun, Jiaya Jia

Paper Code

CVPR 2022

Multi-View Transformer for 3D Visual Grounding

Shijia Huang, Yilun Chen, Jiaya Jia, Liwei Wang

Paper Code

CVPR 2022

Efficient Neural Radiance Fields

Tao Hu, Shu Liu, Yilun Chen, Tiancheng Shen, Jiaya Jia

Paper Code

CVPR 2020

DSGN: Deep Stereo Geometry Network for 3D Object Detection

Yilun Chen, Shu Liu, Xiaoyong Shen, Jiaya Jia

Ranked 1st among camera-based approaches on the KITTI 3D detection leaderboard in November 2019.

Paper Project Code

ICCV 2019

Fast Point R-CNN

Yilun Chen, Shu Liu, Xiaoyong Shen, Jiaya Jia

Paper BibTeX

CVPR 2018

Cascaded Pyramid Network for Multi-Person Pose Estimation

Yilun Chen*, Zhicheng Wang*, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, Jian Sun

Champion of the MS-COCO 2017 Keypoint Detection Challenge.
Ranked 1st on the COCO keypoint detection leaderboard in October 2017.

Paper Code BibTeX

AAAI 2018 Oral

R-FCN++: Towards Accurate Region-based Fully Convolutional Networks for Object Detection

Zeming Li, Yilun Chen, Gang Yu, Yangdong Deng

Paper BibTeX

Experience

Shanghai AI Laboratory Research Scientist, Mar. 2023 - Present
NVIDIA Research Research Intern, Jun. 2022 - Feb. 2023
Mentors: Zhiding Yu, Jose M. Alvarez
SmartMore Inc. Research Intern, Mar. 2020 - Jun. 2022
Mentor: Shu Liu
Tencent Youtu Lab Research Intern, Mar. 2018 - Jan. 2020
Mentor: Shu Liu
Megvii Face++ Research Intern, Nov. 2016 - Nov. 2017
Mentor: Gang Yu

Education

The Chinese University of Hong Kong Ph.D., Computer Science and Engineering, 2018 - 2022
Beihang University Bachelor, Computer Science and Engineering, 2013 - 2017

Service

Conference Reviewer
CVPR, ECCV, ICCV, ICLR, NeurIPS, ICML, CoRL, IROS, ICRA
Journal Reviewer
T-PAMI, IJCV, RA-L
Teaching
CSCI3310, CSCI3180, CSCI1120, ENGG1100

Yilun Chen

Open Positions

Recent Highlights

Selected Publications

InternVLA-M1: A Spatially Grounded Foundation Framework for Generalist Robot Policy

Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation

RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation

ST4VLA: Spatially Guided Training for Vision-Language-Action Models

InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation

X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model

CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation

GenManip: LLM-driven Simulation for Generalizable Instruction-Following Manipulation

Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers

PointLLM: Empowering Large Language Models to Understand Point Clouds

FocalFormer3D: Focusing on Hard Instance for 3D Object Detection

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

DSGN: Deep Stereo Geometry Network for 3D Object Detection

Cascaded Pyramid Network for Multi-Person Pose Estimation

InternVLA-M1: A Spatially Grounded Foundation Framework for Generalist Robot Policy

Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation

RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation

ST4VLA: Spatially Guided Training for Vision-Language-Action Models

InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation

X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model

CronusVLA: Transferring Latent Motion Across Time for Multi-Frame Prediction in Manipulation

GenManip: LLM-driven Simulation for Generalizable Instruction-Following Manipulation

Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers

PointLLM: Empowering Large Language Models to Understand Point Clouds

FocalFormer3D: Focusing on Hard Instance for 3D Object Detection

DSGN++: Exploiting Visual-Spatial Relation for Stereo-based 3D Detectors

Unifying Voxel-based Representation with Transformer for 3D Object Detection

Multi-View Transformer for 3D Visual Grounding

Efficient Neural Radiance Fields

DSGN: Deep Stereo Geometry Network for 3D Object Detection

Fast Point R-CNN

Cascaded Pyramid Network for Multi-Person Pose Estimation

R-FCN++: Towards Accurate Region-based Fully Convolutional Networks for Object Detection

Experience

Education

Service

Re³Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation

Re³Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation