DS-GA 3001.003 / CSCI-GA 3033.115 Embodied Learning and Vision (Spring 2026)

Table of contents

1. DS-GA 3001.003 / CSCI-GA 3033.115 Embodied Learning and Vision (Spring 2026)
   1. Course Syllabus
   2. About
   3. Recommended Prerequisites
   4. Logistics
   5. Grading
   6. Course Work

---

You are visiting the course webpage of DS-GA 3001.003 / CSCI-GA 3033.115 Embodied Learning and Vision (Spring 2026) offerred in the NYU Data Science and Computer Science department taught by Prof. Mengye Ren.

Past year website: Spring 2025.

Course Syllabus

The syllabus document can be accessed here (TBD) (requires NYU login).

About

How do we build end-to-end learning agents for embodied AI? This graduate level course covers advanced topics in embodied visual learning, perception and planning, with applications to robotics and self-driving. Topics include deep learning, computer vision, 3D perception, unsupervised 3D learning, self-supervised representation learning, continual learning, foundation model agents, etc. The goals of the course are:

• Understand and apply computer vision and deep learning in the context of embodied agent learning by applying spatial geometric priors, model inductive biases, design multi-task network architectures, and embodied foundation models.
• Learn to formulate a variety of computer vision and robotics problems using deep learning tools.
• Develop a deep understanding of supervised and self-supervised representation learning for downstream perception and planning tasks.
• Develop hands-on skill of implementing embodied learning systems in simulator environments.

Recommended Prerequisites

• Machine Learning (DS-GA 1003 or CSCI-GA 2565)
• Computer Vision (CSCI-GA 2271)
• Deep Learning (DS-GA 1008)
• Proficiency in Python Programming

Logistics

• Lectures: Tuesday 2:45pm - 5:45pm (including 1hr recitation)
• Location 194 Mercer St Room 206
• Office Hours: See Staff page.
• Communication: We will use Campuswire as our main communication tool for announcements and answering questions related to the lectures, assignments, and projects. The registration link is available on Brightspace.

Grading

• In-Class Participation (10%): Marks will be given on the amount of in-class participation during the discussion and Q&A period.
• Paper Review (15%): Students need to submit a paper review on one of the selected readings every week.
• Topic Presentation (30%): Marks will be given on the prepared content, depth, analysis, and presentation quality.
• Project (45%):
  • Project Proposal (10%): Project proposal is due in the 6th week. Student groups will need to schedule a mandatory office hour to discuss the project with the instructor.
  • Project Meetings (10%): Each team is required to meet with their mentor every two weeks. To get full marks, team members need to show reasonable progress.
  • Presentation (10%): Project presentations are conducted in the final two weeks.
  • Report (15%): Project report is due in the final week.

Course Work

• See Syllabus for details on course works of 1) paper reviews 2) topic presentation and 3) research project.