Georgia Chalvatzaki
TU Darmstadt
Speaker
The Evolving Role of Structure in Physical Intelligence
Overview
What does it take to achieve the physical intelligence of a five-year-old? Children learn to manipulate new objects and generalize skills from only a handful of interactions. Despite rapid progress in vision and language, today’s robots remain far less data-efficient and lack comparably rich training data.
Structured Physical Intelligence (SPIN) asks how physical priors and structured world representations can close this gap. Relevant approaches include Visual-Language-Action models (VLAs), world models, object-centric and compositional representations, affordances, and particle-based models. Robotics provides a direct test of whether these ideas support efficient generalization for dexterous manipulation, hierarchical control, and few-shot learning.
The workshop brings researchers together to examine recent advances, current limitations, and open problems. The workshop will feature a live dexterous manipulation challenge that puts today’s robots side by side with a child, with invited talks and a panel discussion.
Research Questions
- Q1 — What structure enables data-efficient physical intelligence?
- Q2 — How should physical agents represent the world?
- Q3 — What should be built in, and what should emerge from scale?
- Q4 — How can structured models generalize and adapt?
- Q5 — How should progress be understood and evaluated?
The Child–Robot Dexterity Challenge
How do today’s robots compare with a child on everyday manipulation tasks?
We invite researchers, students, and practitioners from academia and industry to test their robotic systems alongside a child on the same everyday household tasks. We will compare performance task by task to show what today’s robots can and cannot do and help participants learn from one another.
How it works
We will select three to five everyday manipulation tasks that a child can complete with familiar objects. We will share the final task list and simple completion guidelines before the event so teams can prepare.
Key dates and participation details
- Final tasks and rules: To be announced. Example tasks may include grasping and moving everyday objects, untwisting a bottle cap, inserting an object into a matching opening, folding a piece of cloth, and pouring from one container into another.
- Challenge launch: September 17, 2026
- Registration deadline: October 12, 2026
- Live challenge: November 12, 2026 — Griffin Ballroom
Prize pool
The challenge features a $1,000 prize pool, sponsored by Scale AI.
Hardware
We will provide a bimanual system with two YAM arms, based on the MolmoAct 2 setup, together with a dexterous Sharpa hand setup. Sharpa will support up to five teams with remote experiments and data collection.
Evaluation
We will share completion guidelines and details on timing, attempts, resets, and autonomy with teams before the event. We will provide a short, task-by-task summary of the demonstrations, with a focus on friendly comparison and shared learning.
The child and participating robotic systems will try the same tasks, giving the audience a familiar point of comparison.
How to express interest
Email the organizing team at roeiherz@gmail.com, amirb4r@gmail.com, or csferrazza@berkeley.edu with:
- Your name, affiliation, and contact information
- A brief description of your system or proposed contribution
- Which hardware you want to use
- Your space, power, networking, setup, and safety requirements
- Any other constraints we should know about
Speakers & Panelists
The Evolving Role of Structure in Physical Intelligence
Abstract: Scaling data and model capacity has expanded the capabilities of robot learning, but robust sensorimotor behavior remains difficult to acquire. In this talk, I will discuss how the role of structure is evolving as robots increasingly rely on pretrained models. As these models supply more of the representation, structure extends beyond the architecture of a policy or model. It also shapes what a robot observes, what its representations preserve, and how it learns from interaction. Drawing on our work in robotic manipulation, I will examine how geometric and semantic priors complement learned representations and physical feedback. The discussion will focus on how these design choices affect data efficiency, generalization, and adaptation, and on what can be built in, what can be inherited from pretrained models, and what must still be learned or measured through physical interaction.
Jeannette Bohg
Stanford University
Speaker & panelist
Learning by Trying: Physical Reasoning from Puzzles to Contact
Learning by Trying: Physical Reasoning from Puzzles to Contact
Abstract: How much of physical intelligence must be built in, and how much can emerge from scale? In this talk, I will examine this question through three settings that share one requirement: the agent must act to find out. First, physical puzzles: Causal-PIK uses an explicit physics-informed prior to choose informative next trials and can outperform human solvers. Yet a frontier foundation model that observes only pixels now matches or exceeds human performance on some of the same puzzles under an equal attempt budget. Second, robots: The benchmark RoboARCade lets a foundation model control a robot directly, so we can observe how it explores, fails, revises its plan, and tries again, much like a solver working through a physical puzzle. Third, contact: The benchmark ContactBench includes objects that look identical but demand different behavior, so success depends on sensing contact during interaction. Data may eventually close this gap, but today there is neither a clear interface between foundation models and contact sensing nor enough data to learn one. Until then, structure is what bridges it.
Yunzhu Li
Columbia University
Speaker
Structured World Models as Scalable Data Engines for Robotics
Structured World Models as Scalable Data Engines for Robotics
Abstract: Scaling robotic manipulation demands predictive models of how the world evolves under action and scalable sources of diverse training data. In this talk, I will present structured world models that incorporate physical priors for planning and control across rigid, deformable, articulated, and granular objects. I will also introduce a real-to-sim-to-real framework that turns real-world observations into interactive digital twins for scalable data generation, policy training, and evaluation, laying the groundwork for more general and robust robotic foundation models.
Mike Rabbat
AMI Labs
Speaker & panelist
Talk title to be announced
Sergey Zakharov
Toyota Research Institute
Panelist
Yixuan Wang
NVIDIA / Columbia University
Panelist
Schedule
Morning half-day schedule, in person, 8:30 AM–12:30 PM.
| 8:30 | Welcome — 10 min |
| 8:40 |
Keynote Talk 1: Georgia Chalvatzaki — 25 min + 5 min Q&AThe Evolving Role of Structure in Physical IntelligenceAbstract: Scaling data and model capacity has expanded the capabilities of robot learning, but robust sensorimotor behavior remains difficult to acquire. In this talk, I will discuss how the role of structure is evolving as robots increasingly rely on pretrained models. As these models supply more of the representation, structure extends beyond the architecture of a policy or model. It also shapes what a robot observes, what its representations preserve, and how it learns from interaction. Drawing on our work in robotic manipulation, I will examine how geometric and semantic priors complement learned representations and physical feedback. The discussion will focus on how these design choices affect data efficiency, generalization, and adaptation, and on what can be built in, what can be inherited from pretrained models, and what must still be learned or measured through physical interaction. |
| 9:10 |
Keynote Talk 2: Jeannette Bohg — 25 min + 5 min Q&ALearning by Trying: Physical Reasoning from Puzzles to ContactAbstract: How much of physical intelligence must be built in, and how much can emerge from scale? In this talk, I will examine this question through three settings that share one requirement: the agent must act to find out. First, physical puzzles: Causal-PIK uses an explicit physics-informed prior to choose informative next trials and can outperform human solvers. Yet a frontier foundation model that observes only pixels now matches or exceeds human performance on some of the same puzzles under an equal attempt budget. Second, robots: The benchmark RoboARCade lets a foundation model control a robot directly, so we can observe how it explores, fails, revises its plan, and tries again, much like a solver working through a physical puzzle. Third, contact: The benchmark ContactBench includes objects that look identical but demand different behavior, so success depends on sensing contact during interaction. Data may eventually close this gap, but today there is neither a clear interface between foundation models and contact sensing nor enough data to learn one. Until then, structure is what bridges it. |
| 9:40 | Child–Robot Dexterity Challenge and Judging — Griffin Ballroom — 60 min |
| 10:40 |
Keynote Talk 3: Yunzhu Li — 25 min + 5 min Q&AStructured World Models as Scalable Data Engines for RoboticsAbstract: Scaling robotic manipulation demands predictive models of how the world evolves under action and scalable sources of diverse training data. In this talk, I will present structured world models that incorporate physical priors for planning and control across rigid, deformable, articulated, and granular objects. I will also introduce a real-to-sim-to-real framework that turns real-world observations into interactive digital twins for scalable data generation, policy training, and evaluation, laying the groundwork for more general and robust robotic foundation models. |
| 11:10 | Keynote Talk 4: Mike Rabbat — 25 min + 5 min Q&A |
| 11:40 | Break — 10 min |
| 11:50 | Panel: What is Next in Structured Physical Intelligence Models? — 30 min |
| 12:20 | Concluding remarks — 10 min |
| 12:30 | Adjourn |
Organizers
Advisory Board
Contact
Questions about the workshop can go to roeiherz@gmail.com, amirb4r@gmail.com, or csferrazza@berkeley.edu.
Sponsors
We gratefully acknowledge the support of our sponsors.
