Jinyi Hu

Jinyi (James) Hu

I am a researcher at DeepSeek AI. I received my Ph.D. from the THUNLP Lab at Tsinghua University, advised by Prof. Maosong Sun.

Previously, I spent six wonderful months as a visiting student at MIT Han Lab, advised by Prof. Song Han and Yao Lu. I also interned at ByteDance Seed, working with Mingxuan Wang. During my undergraduate studies, I interned at Mila Québec, advised by Jian Tang, and at Alibaba DAMO Academy, advised by Boxing Chen.

Please feel free to reach out to me via email.

Email / CV / Google Scholar / Github

Education

B.E. Department of Computer Science and Technology, Tsinghua University, 2017-2021

Ph.D. Department of Computer Science and Technology, Tsinghua University, 2021-2026

News

[2024.6] Open embodied agent platform LEGENT is accepted to ACL 2024 as demo paper.

[2024.1] VisCPM is accepted to ICLR 2024 as a spotlight (5%).

[2023.6] SOTA Zh-En Multimodal Model VisCPM is open-source!

[2022.10] Two papers are accepted to EMNLP 2022.

[2022.4] Della is accepted to NAACL 2022.

[2020.9] One paper is accepted to NeurIPS 2020.

Publication

(*indicates equal contribution)

Embodied AI

LEGENT: Open Platform for Embodied Agents
Zhili Cheng, Jinyi Hu, Zhitong Wang, Shengding Hu, An Liu, Yuge Tu, Pengkai Li, Lei Shi, Zhiyuan Liu, Maosong Sun
ACL 2024 demo
[paper] [code] [doc]

Multimodal Learning

	ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformer Jinyi Hu, Shengding Hu, Yuxuan Song, Yufei Huang, Mingxuan Wang, Hao Zhou, Zhiyuan Liu, Wei-Ying Ma, Maosong Sun arxiv 2024 [paper] [code]
	Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages Jinyi Hu, Yuan Yao, Chongyi Wang, Shan Wang, Yinxu Pan, Qianyu Chen, Tianyu Yu, Hanghao Wu, Yue Zhao, Haoye Zhang, Xu Han, Yankai Lin, Jiao Xue, Dahai Li, Zhiyuan Liu, Maosong Sun ICLR 2024 (spotlight, top 5% ) [paper] [code (VisCPM)] [code (MiniCPM-V)]
	RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun Tat-Seng Chua CVPR 2024 [paper] [project]
	Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis Zanlin Ni, Yulin Wang, Renping Zhou, Jiayi Guo, Jinyi Hu, Zhiyuan Liu, Shiji Song, Yuan Yao, Gao Huang CVPR 2024 [paper]
	AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation Zanlin Ni, Yulin Wang, Renping Zhou, Rui Lu, Jiayi Guo, Jinyi Hu, Zhiyuan Liu, Yuan Yao, Gao Huang ECCV 2024 [paper]
	GUICourse: From General Vision Language Model to Versatile GUI Agent Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, Yuan Yao, Yankai Lin, Zhiyuan Liu, Maosong Sun arxiv* 2024 [paper]
	Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistants Tianyu Yu, Jinyi Hu, Yuan Yao, Haoye Zhang, Yue Zhao, Chongyi Wang, Shan Wang, Yinxv Pan, Jiao Xue, Dahai Li, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun arxiv, 2023 [paper] [code (Muffin)] [dataset (UniMM-Chat)]
	Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivots Jinyi Hu, Xu Han, Xiaoyuan Yi, Yutong Chen, Wenhao Li, Zhiyuan Liu, Maosong Sun arxiv, 2023 [paper]

Text Reasoning and Generation

	OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun ACL 2024 [paper] [code]
	Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, Xing Xie NAACL 2022 (oral) [paper] [code (DELLA)]
	Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, Xing Xie Findings of EMNLP 2022 [paper]
	Towards interpretable natural language understanding with explanations as latent variables Jinyi Hu, Wangchunshu Zhou, Hanlin Zhang, Xiaodan Liang, Maosong Sun, Chenyan Xiong, Jian Tang, NeurIPS* 2020, cite by CoT, STaR [paper] [code]
	Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention Wenhao Li, Xiaoyuan Yi, Jinyi Hu, Maosong Sun, Xing Xie EMNLP 2022 [paper]
	Aspect-level sentiment-controllable review generation with mutual learning framework Huimin Chen, Yankai Lin, Fanchao Qi, Jinyi Hu, Peng Li, Jie Zhou, Maosong Sun AAAI 2021 [paper]

Service

Reviewer: NeurIPS, ICLR, CVPR, ACL, EMNLP, NAACL

Honors & Awards

Comprehensive Scholarship, 2023
Longhu Scholarship, 2023
Comprehensive Scholarship, 2022
Merit Student in Beijing, 2020
Tang Lixin Scholarship, 2020
Evergrande Scholarship, 2018

Miscellaneous

I usually go to the gym during the spare time. I also love NBA, Snooker and Tennis, a big fan of LeBron James, Ronnie O'Sullivan and Roger Federer. I love the number 7.