Fanyi Pu (濮凡轶)

Fanyi Pu
College of Computing and Data Science
Nanyang Technological University, Singapore
+65 8398-1454 | FPU001@e.ntu.edu.sg

Abstract

I am currently a final-year undergraduate student at Nanyang Technological University, specializing in Data Science and Artificial Intelligence.

I am currently doing research at SenseTime and MMLab@NTU. I am fortunate to be supervised by Prof. Ziwei Liu, I am also grateful for the extensive help I have received from Bo Li, Zhongang Cai and Lei Yang. My research interests currently lie in multi-modality (unified) models and spatial intelligence.

During my time at Shaoxing No.1 High School, I participated in the Olympiad in Informatics. Additionally, I competed in the ICPC competition, where I earned a gold medal in the 2022 Kunming Asia Regional Contest and a silver medal in the 2022 Nanjing Asia Regional Contest. I am a contributor to the Nanyang Programming Contest.

I like to play chess, my Lichess ID is @pufanyi, my current rating is 2086 (rapid).

Keywords: Spatial Intelligence · Unified Multimodal Models · Vision-Language Models · Deep Learning

1   Education

Nanyang Technological University2022 – 2026
B.Sc. in Data Science and Artificial Intelligence, Singapore
  • Expected Honours (Highest Distinction), Current CGPA: 4.63 / 5.00
  • Core courses: Reinforcement Learning, Deep Learning, Data Structures and Algorithms, Probability and Statistics, Cryptography
University of California, BerkeleyJun 2024 – Aug 2024
Summer Session, Berkeley, CA
  • Studied Computer Security and Game Theory, GPA: 4.00 / 4.00

2   Publications & Research Projects

SenseNova-SI: Scaling Spatial Intelligence with Multimodal Foundation ModelsSep 2025 – Present
Preprint, Co-first author [Paper] [Project Page]
  • A foundation model designed to scale spatial intelligence, achieved state-of-the-art on key spatial benchmarks
  • Executed full-stack training pipeline for Qwen-based variants with LMMs-Engine on 100+ GPUs
LMMs-Eval: Reality Check on the Evaluation of Large Multimodal ModelsJan 2024 – Present
NAACL (Findings), Co-first author [Paper] [Project Page]
  • 3.3K GitHub stars multimodal evaluation framework
  • Developed low-cost automatic generation pipeline for Multi-modal LiveBench
Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional VideosSep 2024 – Feb 2025
Preprint, Third Author [Paper] [Project Page]
  • Built evaluation set with 300 expert-level videos and 900 human-annotated questions
  • Cited by Gemini 3 Pro (Google) and GPT-5 (OpenAI)
Otter & MIMIC-IT: Multi-Modal In-Context Instruction TuningJun 2023 – Dec 2023
IEEE TPAMI, Co-first author [Paper] [Project Page]
  • Early experiment on vision-language-agent (VLA) model with 3.3K GitHub stars
  • Generated 2.8M multimodal instruction tuning data using pure language models
LMMs-Engine: A Simple, Unified Multimodal Models Training EngineAug 2025 – Present
Research Project, Core Developer
  • Lean and flexible training framework for rapid research prototyping and large-scale production
VLoRP: Memory-Efficient LLM Training by Low-Rank Projection of GradientsAug 2024 – Jan 2025
Preprint, Fourth author [Paper]
  • Implemented training framework with DeepSpeed integration; reproduced baselines (LoRA, GaLore, MeZO)

3   Experience

SenseTime ResearchSep 2025 – Present
Research Intern (Spatial Intelligence), Singapore
  • Conducted research on spatial intelligence, focusing on unified multimodal model optimization
  • Delivered SenseNova-SI, establishing new state-of-the-art for spatial understanding
LMMs-LabJan 2024 – Present
Core Member, Singapore
  • Core member of non-profit initiative democratizing Large Multimodal Models
  • Spearheaded development of LMMs-Eval, LMMs-Engine, and Video-MMMU
Synvo AIJan 2025 – Jul 2025
Core Contributor, Singapore
  • Architected and implemented Synvo File System for unstructured multimodal data storage
MMLab@NTUJan 2025 – May 2025
Research Intern, supervised by Prof. Liu Ziwei
  • Focused on multimodal language models and unified multimodal models

4   Competitions

4.1   ICPC (International Collegiate Programming Contest)

  • Ranked 22nd, ICPC Asia Pacific Championship, 2024
  • Ranked 13th, ICPC Asia Jakarta Regional, 2023
  • Ranked 6th, ICPC Asia Manila Regional, 2022
  • Gold Medal, ICPC Asia Kunming Regional, 2021
  • Silver Medal, ICPC Asia Nanjing Regional, 2021

4.2   Simon Marais Mathematics Competition

  • Best-in-University Prize, Nanyang Technological University, Dec 2022

5   Teaching & Activities

Tutorial Lecturer, NTU SC1008 C & C++ ProgrammingJan 2026 – Present
  • Instructed a cohort of 50 beginners in core C/C++ syntax and methodologies
Nanyang Programming Contest OrganizerJan 2025 – Present
  • Organized 5 competitions and tutorials for non-ICPC students
NTU Students' Computing and Data Science ClubNov 2024 – Present
  • Managed telegram group with 130+ members; share AI/ML learning resources
Teaching Assistant, NTU SC1003 Introduction to Computational ThinkingAug 2024 – Dec 2024
  • Instructed a cohort of 20 beginners in Python syntax and data manipulation

6   Miscellaneous

Hobbies: Physics, Chess (Lichess rating: 2084), Calligraphy, Table tennis