This work introduces Super-Mimic, a hierarchical framework that goes beyond simple imitation of human actions, enabling task modifications and new skill transfers, and operates in a zero-shot manner without requiring task-specific training data.
Generalizing to long-horizon manipulation tasks in a zero-shot setting remains a central challenge in robotics. Current multimodal foundation based approaches, despite their capabilities, typically fail to decompose high-level commands into executable action sequences from static visual input alone. To address this challenge, we introduce Super-Mimic, a hierarchical framework that enables zero-shot robotic imitation by directly inferring procedural intent from unscripted human demonstration videos. Our framework is composed of two sequential modules. First, a Human Intent Translator (HIT) parses the input video using multimodal reasoning to produce a sequence of language-grounded subtasks. These subtasks then condition a Future Dynamics Predictor (FDP), which employs a generative model that synthesizes a physically plausible video rollout for each step. The resulting visual trajectories are dynamics-aware, explicitly modeling crucial object interactions and contact points to guide the low-level controller. We validate this approach through extensive experiments on a suite of long-horizon manipulation tasks, where Super-Mimic significantly outperforms state-of-the-art zero-shot methods by over 20%. These results establish that coupling video-driven intent parsing with prospective dynamics modeling is a highly effective strategy for developing general-purpose robotic systems.
We propose Super-Mimic, a hierarchical framework that combines the HIT module for task planning and the FDP module for subtask execution to tackle zero-shot long-horizon robotic manipulation.
The Human Intent Translator (HIT): This module parses complex human actions into transferable, language-guided robotic plans. It extracts key action frames from human videos and generates language-conditioned, transferable subtask plans through multimodal reasoning.
The Future Dynamics Predictor (FDP): This module employs video generation dynamics-aware visual trajectories that capture object interactions and contact constraints.
These imagined futures guide low-level control with enhanced robustness in cluttered, open-world environments.
We evaluate Super-Mimic on three challenging long-horizon scenarios, each comprising 5 to 8 subtasks:
Our experimental platform: a 7-DoF xArm7 arm and a third-view Orbbec RGB-D camera.
Our experiments demonstrate that Super-Mimic significantly outperforms state-of-the-art zero-shot methods by over 20% on these tasks.
The green lines in the trajectory visualization represent the RDP-optimized 2D trajectories extracted from the generated future videos, used for precise robotic control and capturing object interactions and contact constraints.
We evaluate Super-Mimic on three challenging long-horizon scenarios and compare against state-of-the-art zero-shot methods ReKep and MOKA. The results show that Super-Mimic significantly outperforms the baselines by over 20%.
Key Results:
@article{ye2025watch,
title={From Watch to Imagine: Steering Long-horizon Manipulation via Human Demonstration and Future Envisionment},
author={Ye, Ke and Zhou, Jiaming and Qiu, Yuanfeng and Liu, Jiayi and Zhou, Shihui and Lin, Kun-Yu and Liang, Junwei},
journal={arXiv preprint arXiv:2509.22205},
year={2025}
}