Scaling Online Exploration for Robot Agents
*Equal contribution†Project lead‡Core contributor
Real-robot sessions
Each video is one full session. The agent explores on the real robot, fixes its own mistakes, and turns what works into programs. No demonstrations, no teleoperation, no policy training. The raw logs of every session, with the agent's transcripts, robot tool calls, state logs and camera recordings, are on Hugging Face.
Simulation
Method
A frozen LLM coding agent drives the robot through its perception and control interfaces. What it learns lives in its workspace: notes, programs and verified skills.
Exploration compiles into code
Contribute your robot's skills
Every skill a RAX agent learns on a real robot is an executable procedure, stored with its preconditions, checks, failure modes and the runs that verify it, so a later session on another robot can reuse and compose it. The RAX skill library collects them, one library per embodiment, starting with the Rainbow RB-Y1 and the Franka FR3. If your agent has learned a task on a real robot, please contribute it: add the skill to your robot's library by following the contributing guide, or bring a new robot with its own library.
Case studies
Robot rollout sessions
3D model built by the agent
No model of the air fryer was given. From what it measured in its first manual run, the agent wrote an articulated model, a fixed body and a drawer on a prismatic joint, placed it on the point cloud of each new scene, and planned its actions on it. Drag to orbit, scroll to zoom.
- c, d
- Axis from a circle fit to the lid; pull direction from the handle’s points. Estimated in every scene.
- opening
- Measured from the shift of the handle face after the pull.
- sizes
- Radii, heights and handle size, fixed from the agent’s own measurements.
- checks
- Every planned motion is checked for clearance against the model at the current opening.
Abstract
Robot learning has advanced significantly by scaling demonstration datasets and model capacity; however, deploying a new robot embodiment or task typically necessitates a dedicated teleoperation corpus. Furthermore, resulting policies remain bounded by their imitation distributions, limiting generalization and continual self-improvement. We introduce RAX (Robot Agent eXploration), an open-source framework for robot agents centered on scaling exploration: enabling agents to improve by acquiring online experience and synthesizing executable programs. In RAX, a general-purpose coding agent generates robot control code directly, operating without action demonstration data, hand-engineered action primitives, or value-function guidance. Through online exploration, the agent identifies decision sequences that achieve task goals and compiles them into modular programs encapsulating the full perception-planning-control loop for fast inference. Using RAX, we present a systematic multi-embodiment study on scaling online exploration across fine-grained precision and long-horizon manipulation tasks using single-arm, bimanual, dexterous, mobile, and humanoid systems in simulation and on real robots. Generalization performance on unseen environments improves consistently as exploration scales. In simulation, RAX outperforms top baseline models by +37.4% on average. On real robots, RAX autonomously learns memory-dependent, articulated, high-precision, deformable, and long-horizon mobile manipulation tasks without human demonstrations, continually improving performance by composing verified skills from prior exploration sessions. As exploration scales, we observe emergent system capabilities including active perception, error recovery, safety guardrails, and human-robot agent collaboration. We open-source RAX as a generalist robot-agent platform that allows new robot embodiments to be integrated solely via API interfaces, bypassing teleoperated data collection and policy training entirely.
BibTeX
@article{huang2026scaling,
title={Scaling Online Exploration for Robot Agents},
author={Huang, Runhan and Zhang, Xiangcheng and Chen, Haoyu and Li, Yitong and Wang, Yen-Jen and Lyu, Bohan and Ye, Haotian and Hong, Chuye and Wang, Yixuan and Su, Chloe and others},
journal={arXiv preprint arXiv:2610.22496},
year={2026}
}
