I am a PhD student at the Department of Computer Science and Engineering, The Chinese University of Hong Kong (CUHK), advised by Prof. Jiaya Jia and Prof. Bei Yu .
Prior to that, I obtained my B.Eng. degree in Computer Science from Sun Yat-Sen University (SYSU).
I am particular interested in building Multimodal Intelligence that can actively interact with the physical world and continuously learning from interaction.
My recent research mainly focuses on the the former part, building Multi-modal Large Language Models (MLLMs) to empower intelligence agents to interact with human and the physical world.
Representative works includes LLaMA-VID , Mini-Gemini and MGM-Omni .
Prior to that, I also had some experience on visual perception and representation learning.
I am seeking Research Scientist / MTS opportunities in industry on Multimodal Foundation Models and Agents for 2026 Fall or later, open to any location. Feel free to reach out if there is a good match.
Happy to chat about any topics, research, open source projects, life, career, investment. Feel free to email me and set up a coffee chat.
[2024-12] We release Lyra , an open source multi-modal large language models that support long speech comprehension, omni understanding and cross-modality efficiency.
[2024-07] LLaMA-VID is accepted in ECCV 2024, Milano.
[2024-03] We release Mini-Gemini , an open source vision-language models that support high-resolution image understanding and reasoning image generation.
