Object-centric bimanual manipulation
SimDuoReal
Object-Centric Bimanual Policy
for Emergent Dexterity
Research by
The idea
Specify the object’s motion.
Let the hands discover the coordination.
Trained in simulation, SimDuoReal turns object goals into coordinated action. One policy grasps, reorients, and passes unseen objects between hands—without task-specific fine-tuning.
01 / On real hardware
One Policy, Diverse Tasks, Unseen Objects
The same policy handles precise assembly, object handover, and coordinated tool use. Only the desired object motion changes.
Bring the barcode into view.
Pick up and reorient different objects to present their barcodes to the scanner.
Real-world evaluation uses operator-provided object goals through VR. The policy controls the arms and hands.
02 / Emergent dexterity
Emergent dexterity
The following behaviors emerge from a shared object-centric objective, without prescribed motion sequences.
Handover & in-hand manipulation
1 clipAdjusting the grasp through a handover
Before · During · After
The hands adjust the brush’s orientation before, during, and after the transfer.
In-hand manipulation and handover form one continuous sequence.
Regrasping
2 clipsA new grasp lets the policy recover when a transfer does not go as intended.
Recovering from an unstable handover
Recovering from a failed handover
Pre-grasp manipulation
1 clipReorienting the brush before grasping
Reorienting objects from challenging initial poses makes them graspable.
03 / Interactive demo
Interactive demo
An interactive experience is on its way. Explore the project in a dedicated demo, directly in your browser.
Coming soonOpen interactive demo
04 / The method
Learning one bimanual policy
Train workspace experts, refine their transitions, and distill one policy that follows object trajectories.
Learn through goal sampling
- Shared formulation: object-centric representation and reward.
- Within-workspace goals: grasping and in-hand adjustment.
- Cross-workspace goals: handover.
The blue box marks the workspace in each video.
Left-workspace expert
Grasp and manipulate objects toward goals in the left workspace.
Cross-workspace expert
Transfer a held object to the opposite workspace, coordinating contact and release.
Right-workspace expert
Grasp and manipulate objects toward goals in the right workspace.
Refine behavior transitions
- Start from predecessor end states to practice transitions.
- Keep the original goals and reward while refining each expert.
Distill one policy
- DAgger + reinforcement learning combine the refined experts.
- Train on transfer and two-object scenes to learn behavior composition.
- Deploy one unified policy.
Follow object trajectories
- VR or human video provides object trajectories.
- One or two objects are controlled by the same policy.
- No task-specific fine-tuning on real hardware.
05 / Evaluation
Evaluation results
Real-world success
61 successful trials out of 80.
Four tasks. Unseen objects.
No task-specific fine-tuning.
Controlled simulation benchmark
Handover makes a difference.
Barcode scanning across 3,072 simulated environments with unseen primitive objects.
without handover
unified policy
View simulation comparisons
| Method | Success | Progress | Time (s) |
|---|---|---|---|
| 2 × SimToolReal (no handover) | 60.4% | 79.6% | 9.2 ± 4.9 |
| Chained experts without refinement | 79.4% | 88.1% | 9.1 ± 4.4 |
| Refined expert teachers | 84.3% | 89.8% | 8.7 ± 4.0 |
| Unified policy, DAgger only | 81.6% | 89.5% | 9.0 ± 4.2 |
| Unified policy, composition scenes only | 68.3% | 84.8% | 8.8 ± 4.4 |
| SimDuoReal | 85.0% | 90.8% | 8.6 ± 3.9 |
The system assumes known rigid-object meshes and multi-view pose tracking. Desired pose sequences do not directly specify force- or timing-dependent behavior.
06 / Read the research
SimDuoReal
Object-Centric Bimanual Policy
for Emergent Dexterity
@misc{simduoreal2026,
title = {SimDuoReal: Object-Centric Bimanual
Policy for Emergent Dexterity},
author = {Anonymous},
year = {2026}
}