2026

Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation

Xiaoman Li^1,2 Zhiqiang Yang^2,3 Jiayu Hu^2,4 Jipeng Kong^2,5

Yang Zhang^2,6 Jingwei Zhang^2,3 Fangzheng Yan² Chunhua Shen^1† Chenjia Bai^2,7†

¹ Zhejiang University, State Key Lab of CAD & CG

² The Institute of Artificial Intelligence, China Telecom (TeleAI)

³ South China University of Technology⁴ University of Science and Technology of China

⁵ ShanghaiTech University⁶ Tsinghua University⁷ Gamma Robotics (γ)

^† Corresponding Authors

Find the action chunks that actually move the task forward

arXiv

Code

RedNote

Feishu Group

Demonstrations Interventions Failed rollouts

Project video

GR-BC in the real world.

A two-minute view of precise manipulation, backbone generalization, iterative training, and robustness evaluations. Draw back the task curtain to watch the demo, or reveal it to explore twenty key moments across four manipulation tasks.

Four tasks. Twenty moments of precision.

Explore the four task sequences

Assembly model, graphics card insertion, screw tightening, and memory stick insertion task sequences

Motivation

Long-horizon manipulation data is rarely clean. Even successful demonstrations can contain hesitations, overshoots, failed attempts, and imperfect recoveries. Intervention-assisted rollouts mix autonomous mistakes with delayed expert corrections, while failed episodes can still contain short segments that reconnect the system to a viable path.

Approach

Pipeline

GR-BC keeps the VLA architecture and supervised objective unchanged. Reverse search on a task-conditioned goal-reachability graph estimates remaining steps to success and reweights observed action chunks by their progress. New rollouts refresh the graph and weights for the next offline round.

GR-BC framework: recoverability graph, remaining-step search, weighted VLA training, and new rollouts — **From mixed experience to better supervision.**
Hover a part to follow its role in the learning cycle.

A
Connect compatible experience
Build a conservative path to success.
§4.1 ↗

Demonstrations, policy rollouts, and human recoveries share one task-conditioned graph. Frozen MCR features encode three camera views. Temporal edges preserve executed transitions; cross-trajectory bridges require mutual neighbors in every view, matching gripper states, and locally normalized distances.

Pass forward A goal-reachability graph that can reconnect failed trajectories to successful paths.

B
Retrieve remaining steps
Search backward. Make progress dense.
§4.2 ↗

Successful endpoints connect to a shared goal sink. Reverse Dijkstra search retrieves the minimum task-completion cost supported by the graph; failed endpoints must reconnect through valid bridges. Linear interpolation extends sparse estimates to every frame. States without a goal path receive zero training weight.

Pass forward Dense remaining-step estimates for evaluating each observed action chunk.

C
Learn from the consequence
Imitate chunks that make stable progress.
§4.3 ↗

Normalize remaining steps into progress, then evaluate both the net gain across an executed chunk and its deepest temporary regression. Chunks that pass both thresholds receive bounded positive weights; the rest receive zero. These weights scale the original supervised VLA loss, while all states remain available for graph connectivity.

Pass forward A post-trained VLA policy, using the original architecture and observed action targets.

D
Collect, reconnect, repeat
Turn the next rollout into new evidence.
§4.3 ↗

Run the updated policy in the real world and collect successful and failed rollouts, including human interventions and recoveries. Add these trajectories to the accumulated data pool, extend the graph, and rerun reverse search to refresh training weights for the next offline round.

Back to A New experience updates the graph and supervision for the next training round.

Graph-retrieved progress

How states become progress.

Three camera views and a separate gripper state become a graph node, shown as a simple three-node trajectory. — Frozen features · gripper state kept separate.

Aligned multi-view neighborhoods and locally normalized cross-trajectory connections. — Local agreement · matching gripper states.

Reverse search → remaining cost → progress.

p (s) = clip (1 - \frac{{\hat{R}}_{rec} (s)}{R_{max}}, 0, 1)

Paper §4.1–4.3 ↗

Stable progress → bounded positive weightInsufficient net progress → zero weightExcess regression → zero weight

INTERACTIVE TRAJECTORIES

Progress, under
the microscope.

A successful trajectory is rarely a straight line. Open the collection, follow four real-world rollouts from three synchronized cameras, and unfold the details of every setback.

Loading the trajectory observatory…

Reading the signal. Graph progress is a graph-derived estimate, not human-annotated ground truth. The default curve uses a centered 15-frame moving average; select Raw trace to inspect the original values. Dip values follow the selected curve. Transition weights use their own 0–1.5 scale. Pale orange marks recorded human-intervention frames; the rose-tinted dip windows follow the supplied event annotations. Episode annotations describe the motion, alignment corrections, and human–policy control handoffs associated with each dip.

Progress evidence

Experiment

GRP follows task completion while dropping at detours, reorientation, and slip-outs — the local failures that a monotonic trajectory score can miss.

GRP progress curve compared with RoboMeter, SARM, Robo-Dopamine, and ReWiND — Five progress estimators, with GRP highlighting local regressions.

Progress estimator comparison

Method	Δ_SF ↑Success–failure gap	ρ_dev ↑Deviation detection	Class. SE ↑Successful episodes	Class. FE ↑Failed episodes	Failure data	Progress labels
ReWiND	0.003	0	30/30	1/30	—	—
SARM	0.24	0.48	26/30	14/30	—	✓
Robo-Dopamine	0.29	0.59	29/30	8/30	—	✓
RoboMeter	0.159	0.69	30/30	6/30	✓	✓
GRP (ours)	0.97	0.92	30/30	30/30	✓	—

Policy improvement

91% average final success.

In the iterative X-VLA evaluation, GR-BC reaches the highest final success rate on all four tasks and improves more consistently than DAgger, AWR, and reward-aware BC.

Success rates for DAgger, AWR, RA-BC, and GR-BC on four tasks — Policy improvement across offline post-training rounds with X-VLA.

Graph ablation Why the conservative graph matters

98.3%failed-rollout node reachability

25.0%zero-weight chunk ratio

0.42pre-intervention / normal weight

0.0%gripper-mismatched edges

Cross-trajectory edges recover goal-reachable states from failed rollouts. The gripper-state constraint removes incompatible visual shortcuts while preserving useful recovery connections.

Real-world rollouts

Robustness

Four tasks, in motion.

Graphics card

0:35

Memory stick

0:36

Screw tightening

1:31

Assembly model

0:53

Scroll sideways to explore

Robustness videos: Graphics card · Memory stick · Screw tightening · Assembly model.

Citation

Cite this work.

Use the BibTeX below to cite this work.

@inproceedings{li2026grbc,
  title     = {Graph-Reweighted Behavior Cloning with Human Interventions
               for Precise Robotic Manipulation},
  author    = {Li, Xiaoman and Yang, Zhiqiang and Hu, Jiayu and Kong, Jipeng and
               Zhang, Yang and Zhang, Jingwei and Yan, Fangzheng and
               Shen, Chunhua and Bai, Chenjia},
  booktitle = {Conference on Robot Learning},
  year      = {2026}
}

Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation

GR-BC in the real world.

Motivation

Pipeline

Build a conservative path to success.

Search backward. Make progress dense.

Imitate chunks that make stable progress.