user avatar
Deepak Pathak
@pathak2206
Co-Founder & CEO @SkildAI, Faculty @CarnegieMellon. PhD @UCBerkeley; BTech @IITKanpur I study topics in AI (robotics, machine learning & computer vision).
Pittsburgh, PA
Joined May 2013
Posts
  • Pinned
    user avatar
    We hosted Prof. Alyosha Efros (UC Berkeley) at @SkildAI! He didn't believe that robots could actually cook eggs reliably. :) Tested back-to-back 5times without fail! One batch of scrambled eggs every ~2.5mins nonstop. The same model assembles a GPU on a server rack too.
    Image
    00:00
  • user avatar
    Even after 4yrs of locomotion research, we keep getting surprised by how far we can push the limits of legged robots! We report a major update 🚀🤖 Extreme Parkour: extremely long & high jumps, ramp, handstand, etc. all with a single neural net! extreme-parkour.github.io 🧵(1/n)
    Image
    00:00
  • user avatar
    🤖 Robotics often faces a chicken and egg problem: no web-scale robot data for training (unlike CV or NLP) b/c robots aren't deployed yet & vice-versa. Introducing VRB: Use large-scale human videos to train a *general-purpose* affordance model to jumpstart any robotics paradigm!
    Image
    00:00
  • user avatar
    LLMs like GPT-3 and Codex contain rich world knowledge. In this fun study, we ask if GPT like models can plan actions for embodied agents. Turns out, with apt sanity checks, even vanilla LLMs without any finetuning can generate good high-level plans given a low-level controller.
    Image
    00:00
  • user avatar
    RL gets specific to the robot it is trained on. Can a policy be trained to control many agents? Turns out, training (shared) policy for each motor instead of whole robot not only achieves SOTA at train but also transfers to unseen agents w/o fine-tuning! huangwl18.github.io/modular-rl
    Image
    00:00
  • user avatar
    After 3yrs of locomotion research, we report a major update in our #CoRL2022 (Oral) paper: vision-based locomotion. Our small, safe, low-cost robot can walk almost any terrain: high stairs, stepping stones, gaps, rocks. Stair for this robot is like climbing walls for humans.
    Image
    00:00
  • user avatar
    Thrilled to announce @SkildAI! Over the past year, @gupta_abhinav_ and I have been working with our top-tier team to build an AI foundation model grounded in the physical world. Today, we’re taking Skild AI out of stealth with $300M in Series A funding, led by @lightspeedvp,
  • user avatar
    How can we enable robots to perform diverse tasks? Designing rewards or demos for each task is not scalable. We propose WHIRL which learns by watching a single human video followed by autonomous exploration *directly* in the real world (no simulation)! human2robot.github.io
    Image
    00:00
  • user avatar
    Robotic hands are daunting -- costly yet super fragile. After yrs of frustration, we decided to make one that is better, stronger & anyone can build! Open sourcing LEAP Hand 🚀🤖 - low cost ($2K) - 3D printed. Easy to assemble (3hr) - sim2real code etc. leaphand.com
    Image
    00:00
  • user avatar
    RL agents get specific to tasks they are trained on. What if we remove the task itself during training? Turns out, a self-supervised planning agent can both explore efficiently & achieve SOTA on test tasks w/ zero or few samples in DMControl from images! ramanans1.github.io/plan2explore
    Image
    00:00
  • user avatar
    Thrilled to announce @SkildAI! Over the past year, @gupta_abhinav_ and I have been working with our top-tier team to build an AI foundation model grounded in the physical world. Today, we’re taking Skild AI out of stealth with $300M in Series A funding: forbes.com/sites/rashishr…
  • user avatar
    I'm excited to share that I'll be joining CMU (@CarnegieMellon) as an Assistant Professor in the School of Computer Science (@SCSatCMU) in 2020! Grateful to my mentors and friends in making this possible. Looking forward to the exciting years ahead at @CMU_Robotics and @mldcmu!
    Image
  • user avatar
    In-context learning for Robotics is here. Skild Brain adapts to new bodies and survives extreme shifts — without ever being trained on any of these robots.
    We built a robot brain that nothing can stop. Shattered limbs? Jammed motors? If the bot can move, the Brain will move it— even if it’s an entirely new robot body. Meet the omni-bodied Skild Brain:
    Image
    00:00
  • user avatar
    How could we enable an agent to perform many tasks? Supervising for every new task is impractical. We present Latent Explorer Achiever (LEXA) that explores by discovering goals far beyond the frontier and then achieves test tasks, specified via images, in a zero-shot manner.
    Image
    00:00