Karl Pertsch (@KarlPertsch) / X

Karl Pertsch

457 posts

Karl Pertsch

@KarlPertsch

Robot Foundation Models @physical_int

Joined July 2015

Pinned
Karl Pertsch
@KarlPertsch
Mar 3
This one has been a long time coming: today we’re introducing MEM, an approach for giving VLAs short-term and long-term memory. Memory is such an obvious capability, but adding it isn’t easy (most VLAs today are memory-less). A short thread on challenges, solutions, and the new
00:00
9.5K
Karl Pertsch
@KarlPertsch
Jan 16, 2025
Excited to release FAST, our new robot action tokenizer! 🤖 Some highlights: - Simple autoregressive VLAs match diffusion VLA performance - Trains up to 5x faster - Works on all robot datasets we tested - First VLAs that work out-of-the-box in new environments! 🧵/
00:00
91K
Karl Pertsch
@KarlPertsch
Oct 3, 2023
Very excited to release the Open X-Embodiment Dataset today — the largest robot dataset to date with 1M+ trajectories! Robotics needs more data & this is a big step! robotics-transformer-x.github.io There’s lots to unpack here, so let’s do a deep dive into the dataset! 🧵1/15
00:00
111K
Karl Pertsch
@KarlPertsch
Sep 29, 2024
I'll give a talk at the Multimodal Agents workshop at ECCV tomorrow Sept 30, at 2:20pm CET. Excited for my first talk at a vision conference: robotics is increasingly becoming a multi-modal sequence modeling problem w/ lots of potential for LLM/VLM researchers to have big impact!
71K
Karl Pertsch
@KarlPertsch
Jun 20, 2025
We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵
00:00
111K
Karl Pertsch
@KarlPertsch
Dec 14, 2023
3 mo. ago we released the Open X-Embodiment dataset, today we’re doing the next step: Introducing Octo 🐙, a generalist robot policy, trained on 800k robot trajectories, stronger than RT-1X, flexible observation + action spaces, fully open source! 💻: octo-models.github.io /🧵
00:00
127K
Karl Pertsch
@KarlPertsch
Jun 14, 2024
Very excited to release OpenVLA today, a 7B parameter open-source vision-language-action model (VLA). 🦾 SoTA generalist policy (better than Octo & RT-2-X) ⚡️ Easy to run & fine-tune on 1 GPU with quantization and LoRA 💻 Open-source PyTorch codebase 🤗 Models on HuggingFace 1/
00:00
61K
Karl Pertsch
@KarlPertsch
Feb 4, 2025
We are releasing the π₀ model today -- code + weights + fine-tuning instructions, including our recent π₀-FAST model! 🎉 We hope the model will be useful to others! I am really excited about this release because it also marks a shift in how we can *evaluate* policies! Mini 🧵/
Physical Intelligence
@physical_int
Feb 4, 2025
Many of you asked for code & weights for π₀, we are happy to announce that we are releasing π₀ and pre-trained checkpoints in our new openpi repository! We tested the model on a few public robots, and we include code for you to fine-tune it yourself.
00:00
44K
Karl Pertsch
@KarlPertsch
Oct 1, 2024
It was fun giving this talk yesterday! The live talk wasn't recorded, but I just uploaded the recording of a practice run I did the night before (link below). A short thread of key points from the talk 🧵
Karl Pertsch
@KarlPertsch
Sep 29, 2024
I'll give a talk at the Multimodal Agents workshop at ECCV tomorrow Sept 30, at 2:20pm CET. Excited for my first talk at a vision conference: robotics is increasingly becoming a multi-modal sequence modeling problem w/ lots of potential for LLM/VLM researchers to have big impact!
43K
Karl Pertsch
@KarlPertsch
Mar 19, 2024
Access to *diverse* training data is a major bottleneck in robot learning. We're releasing DROID, a large-scale in-the-wild manipulation dataset. 76k trajectories, 500+ scenes, multi-view stereo, language annotations etc Check it out & download today! 💻: droid-dataset.github.io
00:00
40K
Karl Pertsch
@KarlPertsch
Jun 25, 2025
I'll give a talk about benchmarking generalist policies today at RSS (4:30p, RTH 526, in the benchmarking workshop)! I will discuss sim eval, auto eval, and distributed real-world eval (ie RoboArena) -- swing by :)
13K
Karl Pertsch
@KarlPertsch
Jul 9, 2024
Our OpenVLA model has been downloaded more than 20k times in less than a month -- the most for any robotics model on the 🤗 hub by a long shot! Here is a little "cook book" for people who want to get started using OpenVLA! 🧑‍🍳 1/🧵
25K
Karl Pertsch
@KarlPertsch
Nov 5, 2024
I will be at @corl_conf this week, co-presenting 4 papers and one workshop across the full spectrum of scalable robot learning research: data, models & evals! Also happy to chat about research @physical_int! Short 🧵w/ paper pointers 👇
00:00
10K
Karl Pertsch
@KarlPertsch
Jul 15, 2025
We’re organizing the RoboArena Challenge at CoRL this year! Show the performance of your best generalist policy, in a fair, open benchmark for the robotics community! 🤖 Sign up, even if you don’t have a robot! More details in 🧵👇
00:00
27K