ROBOT LEARNING LONG-HORIZON MANIPULATION

ReEnvisionFuture-Guided Selective Action Repair
for Long-Horizon Manipulation

Revise the belief. Repair the plan.

ReEnvision compares expected and observed futures, updates its joint future-action belief, and selectively repairs the affected actions while preserving the rest.
Figure 1: Overview of ReEnvision. ReEnvision maintains a joint future-action belief ht and revises it with new observations. The revision identifies affected actions in the cached chunk, enabling selective repair. Unlike methods that rely on full replanning, ReEnvision repairs only affected actions within the current chunk while preserving unaffected ones.

Abstract

A local deviation may affect only a few actions.
Why regenerate the entire plan?

Long-horizon robotic manipulation requires robust execution over sequences of interdependent actions. Existing methods typically handle execution deviations by replanning the remaining action chunk or correcting actions step by step as new observations arrive. However, a local deviation may affect only a few actions under distribution shift, yet replanning regenerates the entire chunk from an unfamiliar state, potentially introducing errors into otherwise valid actions. Step-wise correction may react too late, as it does not explicitly anticipate how the deviation affects subsequent actions. We propose ReEnvision, a future-guided selective action repair framework that updates a joint future-action belief using execution feedback. It then learns how the resulting belief revision affects each unexecuted action. The resulting belief revision enables selective repair of only the affected actions within a cached action chunk, preserving valid actions and reserving full replanning for severe deviations. On CALVIN, L-CALVIN, and RLBench, ReEnvision achieves 86.6% 5-step success, improves 10-step success from 56% to 63%, and reaches an average success rate of 88.8%, respectively. On five real-world manipulation tasks, it achieves an average success rate of 82.8%. Controlled repair experiments further demonstrate the benefits of localizing affected actions and preserving valid ones during execution.

Robust execution over longer horizons.

CALVIN

86.6%

5-task success

4.62 / 5 average completed tasks

L-CALVIN

63%

10-task success

+7 percentage points over Long-VLA

RLBench

88.8%

Mean task success

Across 18 manipulation tasks

Real-world

82.8%

Mean task success

ARX X5 · Five manipulation tasks
CALVIN · ABC → D · Success rate (%)
Method1 task2 tasks3 tasks4 tasks5 tasksAvg. length
π*0.594.887.478.271.764.33.97
VPP96.590.986.682.076.94.33
REMAC96.091.488.384.978.44.39
PALM96.993.889.385.982.04.48
DeFI97.994.290.787.081.24.51
VITA99.194.991.287.884.54.57
ReEnvision Ours97.894.593.090.186.64.62

Table 1 · 1,000 five-task sequences; results averaged over three random seeds. Bold indicates the best listed value.

CONTROLLED DEVIATIONS

Match the intervention
to the change.

Small deviations often leave the cached plan valid. Medium deviations call for localized repair. Severe deviations more often require full replanning.

73% of small-perturbation cases preserve the cached suffix.

Execution strategy by perturbation severity

KeepSelective repairReplan
Small
Medium
Severe
Figure 4 · Execution decisions under controlled perturbations. Each row sums to 100%.

Experiments

Simulation

5-task sequence 1

DeFI0:18

3/5 tasks completed

ReEnvision0:12

5/5 tasks completed

5-task sequence 2

DeFI0:18

3/5 tasks completed

ReEnvision0:10

5/5 tasks completed

Real-world

Pen placement

DeFI0:31

1/2 tasks completed

ReEnvision0:31

2/2 tasks completed

Color sorting

DeFI0:43

2/3 tasks completed

ReEnvision0:43

3/3 tasks completed

Sequential drawer placement

DeFI1:06

3/6 tasks completed

ReEnvision1:35

6/6 tasks completed

Towel folding and placement

DeFI0:58

4/4 tasks completed

ReEnvision1:03

4/4 tasks completed

Sequential flower placement

DeFI1:11

2/3 tasks completed

ReEnvision1:46

3/3 tasks completed