Course Description

This course teaches advanced principles for endowing mobile autonomous robots with capabilities to autonomously learn new skills and to physically interact with the environment and with humans. Concepts that will be covered in the course are: Reinforcement Learning (RL) and its relationship to optimal control, contact and dynamics models for prehensile and non-prehensile robot manipulation, as well as imitation learning and human intent inference. Students will learn the theoretical foundations for these concepts. Prerequisites: CS106A or equivalent, CME 100 or equivalent (for linear algebra), CME 106 or equivalent (for probability theory), and AA 174A/274A.

Instructors

Prof. Jeannette Bohg	Prof. Marco Pavone	Prof. Dorsa Sadigh

Course Assistants

Aditya Dutt	Suneel Belkhale	Chris Agia	Roger Dai

Meeting Times

Lectures meet on Mondays and Wednesdays from 1:30pm to 2:50pm at Skilling Auditorium.

Prof. Bohg's office hours are Wednesdays, 9:00am - 10:00am in Gates 244 and on Zoom.
Prof. Pavone's office hours are Tuesdays 1:00pm - 2:00pm in Durand 261.
Prof. Sadigh's office hours are Fridays 9:00am - 10:00am in Gates 246 or by appointment.
CA office hours are (starting Week 2):

• Mondays from 2:30pm to 4:00pm (Suneel, Zoom)
• Mondays from 4:15pm to 5:30pm (Chris, Gates 100)
• Tuesdays from 9:00am to 10:30am (Roger, Huang Basement)
• Tuesdays from 4:30pm to 6:00pm (Aditya, Gates 100)
• Thursdays from 2:30pm to 4:00pm (Suneel, Zoom)
• Fridays 2:30pm to 4:00pm (Aditya, Gates 100)

All up-to-date OH times and events can be found on our Google Calendar

Syllabus

The class syllabus can be found here.

Project Report

For students taking the course for 4 units, you are required to complete a project report by the end of the quarter. Project report guidelines can be found here.

Schedule

Subject to change. Lecture recordings will be posted on Canvas.

Blue: learning-based control and perception
Red: interaction with the physical environment
Green: interaction with humans

Lecture 12+

Week	Topic	Lecture Slides	Lecture Notes
1	(Jan 6) Course overview, intro to ML for robotics (Jan 8) Neural networks and PyTorch tutorial (Jan 10) HW1 out	Lecture 1 Lecture 2	Lecture 1 Colab notebook
2	(Jan 13) Markov decision processes (Jan 15) Intro to RL	Lecture 3 Lecture 4	Lecture 3 & 4 Lecture 3 & 4
3	(Jan 20) Martin Luther King Jr. Day (no class) (Jan 22) Model-based and model-free RL for robot control	Lecture 5	Lecture 5 & 6
4	(Jan 27) Learning-based perception (Jan 29) Fundamentals of grasping and manipulation I (Jan 31) HW1 due, HW2 out	Lecture 6 Lecture 7	Lecture 5 & 6 Lecture 7 & 8 & 9
5	(Feb 3) Fundamentals of grasping and manipulation II (Feb 5) Learning-based grasping and manipulation (Feb 7) Exam 1	Lecture 8 Lecture 9	Lecture 7 & 8 & 9 Lecture 7 & 8 & 9
6	(Feb 10) Learning-based Manipulation (Feb 12) Interactive Perception (Feb 14) HW2 due, HW3 out	Lecture 10 Lecture 11
7	(Feb 17) Presidents' Day (no class) (Feb 19) Imitation learning I (Feb 21) Exam 2	Lecture 12	Lecture 12+
8	(Feb 24) Imitation learning II (Feb 26) Learning from human feedback	Lecture 13 Lecture 14	Lecture 12+
9	(Mar 03) Interaction-aware learning, planning, and control (Mar 05) Shared autonomy (Mar 07) HW3 due	Lecture 15 Lecture 16
10	(Mar 10) Guest lecture (Erdem Bıyık) (Mar 12) Guest lecture 2 (TBD) (Mar 14) Exam 3