Kenny Shaw (@kenny_

Kenny Shaw

845 posts

Kenny Shaw

@kenny__shaw

Incoming asst professor at UIUC MechSE. My research focuses on learning dexterity from humans as well as building low-cost LEAP Hands.

Pittsburgh PA

Joined June 2014

Pinned
Kenny Shaw
@kenny__shaw
Nov 19, 2024
Teaching bimanual robot hands to perform very complex tasks has been notoriously challenging. In our work, Bidex: Bimanual Dexterity for Complex Tasks, we’ve developed a low-cost system that completes a wide range of highly dexterous tasks in real-time. bidex-teleop.github.io
00:00
39K
Kenny Shaw
@kenny__shaw
Oct 31, 2023
How can robot hands learn to do kitchen tasks from safe, continual real-world interaction? In our new CoRL 2023 paper DEFT, we learn policy priors from human videos and use our soft robot hand to safely adapt the policy with real world practice. 1/
00:00
19K
Kenny Shaw
@kenny__shaw
Sep 21, 2023
Want to get into dexterous manipulation but are frustrated with no good hardware options? You should try LEAP Hand, our dexterous, durable, $2k robot hand that can be assembled in 3 hours. We even open-sourced our sim2real code! leaphand.com cs.cmu.edu/news/2023/leap…
11K
Kenny Shaw
@kenny__shaw
Nov 9, 2023
Visit us demoing at #CoRL2023 Poster 5 and afternoon demos today! @anag004 uses LEAP Hand for sim2real dexterous, functional grasping dexfunc.github.io @aditya__kannan uses DASH Hand to learn from internet videos and real-world fine-tuning dexterous-finetuning.github.io
00:00
18K
Kenny Shaw
@kenny__shaw
Apr 13, 2024
Happy to share that our paper, Videodex, was featured on the front page of IJRR for April 2024.😀 I believe learning from human videos for robot hands will be very important especially in the era of foundation models that need a lot of data. journals.sagepub.com/toc/ijra/43/4
6.6K
Kenny Shaw
@kenny__shaw
May 15, 2024
Cool work using our open source LEAP Hand on a small robot arm! Excited to see where your project goes!
Brian Smith
@sirwart
May 15, 2024
Playing around with a modified version of the LEAP hand by @kenny__shaw. Still need to train it to shoot hoops on its own though
00:00
5.1K
Kenny Shaw
@kenny__shaw
May 24, 2024
Great investigative work by Toru and co! Gloves are definitely a great way to collect robot hand teleop data. Our lab has also been also building a big setup which we will demo and open-source at RSS 2024 :)
Toru
@ToruO_O
May 24, 2024
A common question we get for HATO (toruowo.github.io/hato) is: can it be more dexterous? Yes! The first iteration of our system actually achieves this -- by capturing finger poses with mocap gloves and remapping them to robot hands. [video taken in late 2023 (with @YuZhang)]
00:00
00:00
5.2K
Kenny Shaw
@kenny__shaw
May 5, 2024
Interesting....same as our upcoming low-cost and open-sourced hand. 😉
Elon Musk
@elonmusk
May 5, 2024
Replying to @DrJimFan and @Tesla_Optimus
The new Optimus hand later this year will have 22 DoF
7.9K
Kenny Shaw
@kenny__shaw
Jan 26, 2024
Autonomously opening ANY door in ANY environment is a very hard problem. We 1) Initialize our policy with VLM + priors and adapt continually to the real world using RL. 2) Develop robust, repeatable, low-cost hardware to unlock new robot performance
Haoyu Xiong
@Haoyu_Xiong_
Jan 26, 2024
Introducing Open-World Mobile Manipulation 🦾🌍 – A full-stack approach for operating articulated objects in open-ended unstructured environments: Unlocking doors with lever handles/ round knobs/ spring-loaded hinges 🔓🚪 Opening cabinets, drawers, and refrigerators 🗄️ 👇
00:00
5K
Kenny Shaw
@kenny__shaw
Dec 21, 2023
Great results using our open-source LEAP Hand for functional grasping!
Ananye Agarwal
@anag004
Dec 21, 2023
Robotic intelligence requires dexterous tool use, but generalizing across tools is hard. Our CoRL23 paper combines semantics (affordances) with low-level control (sim2real) to show functional grasping that generalizes to hammers, drills and more! dexfunc.github.io 1/n
00:00
2.1K
Kenny Shaw
@kenny__shaw
Mar 14, 2024
Great project using Mocap gloves to collect accurate human hand data that can transfer to our LEAP Hand. The autonomous results are great! Best of all, everything is open source and very inexpensive to replicate. Congrats @chenwang_j and SAIL team!!!
Chen Wang
@chenwang_j
Mar 14, 2024
Can we use wearable devices to collect robot data without actual robots? Yes! With a pair of gloves🧤! Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands Everything open-sourced
00:00
2.4K
Kenny Shaw
@kenny__shaw
Mar 26, 2024
Another great LEAP Hand paper with deformables! Build your own in only a few hours. Leaphand.com
Jun Yamada
@junjungoal
Mar 25, 2024
We introduce D-Cubed, a novel trajectory optimisation method using a latent diffusion model trained from a task-agnostic play dataset, including only representative hand motions, to solve dexterous deformable object manipulation tasks! (1/N)
00:00
1.5K
Kenny Shaw
@kenny__shaw
Mar 24, 2024
Dynamixels are truly great. It's almost similar to what Python is in software: a great abstraction layer on actuation. It allows us to focus on the robot design and learning itself.
Alexander Koch
@alexkoch_ai
Mar 23, 2024
Open-source robot arm for about $200. It uses five Dynamixel servo motors and weighs slightly over 100g (without the base). The design for a leader arm is included as well so that you can teleoperate it effectively. (Video is at 1x speed) github.com/AlexanderKoch-…
00:00
1.6K
Kenny Shaw
@kenny__shaw
Apr 4, 2024
Demo is all you need right now for the hype 😅
Aidan Gold
@MrGoldBro
Apr 3, 2024
This has to be peak AI frenzy. Cognition Labs potentially raising at a $2B valuation, just weeks after announcing their $21M raise at $350M. Do they even have $1M in revenue?
3K