CALVIN
ROBOT LEARNING LONG-HORIZON MANIPULATION
ReEnvisionFuture-Guided Selective Action Repair
for Long-Horizon Manipulation
Revise the belief. Repair the plan.
Abstract
A local deviation may affect only a few actions.
Why regenerate the entire plan?
Long-horizon robotic manipulation requires robust execution over sequences of interdependent actions. Existing methods typically handle execution deviations by replanning the remaining action chunk or correcting actions step by step as new observations arrive. However, a local deviation may affect only a few actions under distribution shift, yet replanning regenerates the entire chunk from an unfamiliar state, potentially introducing errors into otherwise valid actions. Step-wise correction may react too late, as it does not explicitly anticipate how the deviation affects subsequent actions. We propose ReEnvision, a future-guided selective action repair framework that updates a joint future-action belief using execution feedback. It then learns how the resulting belief revision affects each unexecuted action. The resulting belief revision enables selective repair of only the affected actions within a cached action chunk, preserving valid actions and reserving full replanning for severe deviations. On CALVIN, L-CALVIN, and RLBench, ReEnvision achieves 86.6% 5-step success, improves 10-step success from 56% to 63%, and reaches an average success rate of 88.8%, respectively. On five real-world manipulation tasks, it achieves an average success rate of 82.8%. Controlled repair experiments further demonstrate the benefits of localizing affected actions and preserving valid ones during execution.
Robust execution over longer horizons.
L-CALVIN
10-task success
+7 percentage points over Long-VLARLBench
Mean task success
Across 18 manipulation tasksReal-world
Mean task success
ARX X5 · Five manipulation tasks| Method | 1 task | 2 tasks | 3 tasks | 4 tasks | 5 tasks | Avg. length |
|---|---|---|---|---|---|---|
| π*0.5 | 94.8 | 87.4 | 78.2 | 71.7 | 64.3 | 3.97 |
| VPP | 96.5 | 90.9 | 86.6 | 82.0 | 76.9 | 4.33 |
| REMAC | 96.0 | 91.4 | 88.3 | 84.9 | 78.4 | 4.39 |
| PALM | 96.9 | 93.8 | 89.3 | 85.9 | 82.0 | 4.48 |
| DeFI | 97.9 | 94.2 | 90.7 | 87.0 | 81.2 | 4.51 |
| VITA | 99.1 | 94.9 | 91.2 | 87.8 | 84.5 | 4.57 |
| ReEnvision Ours | 97.8 | 94.5 | 93.0 | 90.1 | 86.6 | 4.62 |
Table 1 · 1,000 five-task sequences; results averaged over three random seeds. Bold indicates the best listed value.
| Method | 5-task success | 10-task success | Avg. length |
|---|---|---|---|
| VLAS | 49 | 16 | 4.90 |
| RoboVLMs | 64 | 34 | 6.04 |
| REMAC | 71 | 41 | 6.61 |
| DeFI | 75 | 44 | 7.30 |
| Long-VLA | 85 | 56 | 8.24 |
| ReEnvision Ours | 89 | 63 | 8.36 |
Table 2 · 100 ten-instruction sequences; results averaged over three random seeds. Success rates shown as percentages.
| Method | Mean success (%) |
|---|---|
| HDFlow | 87.5 |
| TVVE | 86.6 |
| SpatialActor | 87.4 |
| BridgeVLA | 85.7 |
| ReEnvision Ours | 88.8 |
Table 3 · 18 tasks, 50 episodes per task; results averaged over three random seeds.
| Method | Pen | Color | Drawer | Towel | Flower | Average |
|---|---|---|---|---|---|---|
| Diffusion Policy | 78 | 56 | 78 | 80 | 52 | 68.8 |
| π*0.5 | 90 | 70 | 80 | 94 | 70 | 80.8 |
| BridgeVLA | 88 | 70 | 86 | 74 | 68 | 77.2 |
| DeFI | 86 | 68 | 78 | 82 | 66 | 76.0 |
| ReEnvision Ours | 90 | 76 | 84 | 88 | 76 | 82.8 |
Table 4 · Five manipulation tasks on an ARX X5 robot; 50 trials per task for each method. Bold indicates the best listed value.
CONTROLLED DEVIATIONS
Match the intervention
to the change.
Small deviations often leave the cached plan valid. Medium deviations call for localized repair. Severe deviations more often require full replanning.
73% of small-perturbation cases preserve the cached suffix.
Execution strategy by perturbation severity
Experiments
Simulation
5-task sequence 1
3/5 tasks completed
5/5 tasks completed
5-task sequence 2
3/5 tasks completed
5/5 tasks completed
10-task manipulation sequence
5/10 tasks completed
10/10 tasks completed
Open the bottom drawer
Failure
Success
Wine bottle placement
Failure
Target: right of the rack
Success
Target: middle of the rack
Real-world
Pen placement
1/2 tasks completed
2/2 tasks completed
Color sorting
2/3 tasks completed
3/3 tasks completed
Sequential drawer placement
3/6 tasks completed
6/6 tasks completed
Towel folding and placement
4/4 tasks completed
4/4 tasks completed
Sequential flower placement
2/3 tasks completed
3/3 tasks completed
Long sequential flower placement
3:02ReEnvision
Sequentially placing seven flowers into a vase.