Deepak Pathak

@deepakpathak

Co-Founder & CEO @SkildAI, Faculty @CarnegieMellon. PhD @UCBerkeley; BTech @IITKanpur I study topics in AI (robotics, machine learning & computer vision).

Pittsburgh, PA

cs.cmu.edu/~dpathak/

Replies Reposts Media

Pinned
Deepak Pathak
@deepakpathak
Aug 25
In-context learning for robotics is here. - Long-horizon tasks over 10 minutes long - Never seen during pre-training - Prompted with one video, no fine-tuning We are building intelligence from the foundations up, not from the top down.
Skild AI
@SkildAI
Aug 25
Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:
00:00
35
Deepak Pathak
@deepakpathak
11h
If S1 makes a mistake, it tries again. For example, the input video prompt here attached the wheel just once. During deployment the wheel didn't align properly, so S1 re-aligned it. By watching the prompt, S1 understands the goal and improvises to get there:
00:00
ADRIAN | PROFESOR IA
@TUPROFESORIA
Aug 25
Replying to @SkildAI
what happens if the robot accidentally drops something, does it panic or retry?
28
Deepak Pathak
@deepakpathak
Sep 8
"Does S1 exhibit physical prompt steerability: different prompts induce distinct behavior in the same environment?" Yes! Watch S1 follow 4 different video prompt recipes in the same kitchen. The first one is something far out of distribution -- "putting a plate in a toaster".
00:00
Skild AI
@SkildAI
Aug 25
05:14
Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:
46
Deepak Pathak
@deepakpathak
Sep 3
You don't have to post-train ChatGPT on every user. If you did, it would never have taken off. Yet this is exactly how robotics works today For robots to take off, they need to learn in-context. Great article from @chris_j_paxton on this new paradigm and S1's place in it:
Chris Paxton
@chris_j_paxton
Aug 28
In-Context Learning Results Hint at a “GPT Moment” for Robotics For general-purpose robots to be useful and economical, you need to be able to teach them new skills on the fly. And now we can see how this will work, with in-context long horizon video demonstrations shown by
4
Deepak Pathak
@deepakpathak
Sep 3
Big day for open source. Congratulations to @huggingface and @nvidia!
Jensen Huang
@JensenHuang
Sep 3
Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI. Thank you
1