A strong grounding foundation model.
We introduce GroundingPI, a 4B model built on visual primitives, with a staged training recipe and state-of-the-art grounding performance.
GroundingPI
A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Transfer toward physical intelligence.
Autonomous driving and robotic manipulation evaluations demonstrate the value of this perceptual foundation, including strong ID and OOD performance and improved action-data efficiency.
Insights into perceptual pretraining and future embodied paradigms.
We analyze how pretraining scale and data composition shape grounding and transfer, and discuss implications for System-1 foundation-model design and its complementary role in future embodied systems.
News
- GroundingPI weights released on ModelScope.
- GroundingPI weights released on Hugging Face.
Abstract
Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%).
As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.
Demo Video
GroundingPIOpen video ↗
Method
Results
Select a model or capability to explore the values.
Physical Intelligence Performance
We evaluate autonomous driving on nuScenes and manipulation on RoboTwin 2.0 and RoboCasa-GR1. The results show lower driving error at every reported horizon and the highest success rate in five of six manipulation settings, including all four OOD settings. These results highlight the value of grounding perception for physical intelligence across distinct action learners and embodiments.
Manipulation success rates and reciprocal nuScenes open-loop L2 errors (higher is better in all panels). Manipulation comparisons fix the action expert, action data, and training budget within each benchmark.
Open figure ↗Value details
| Benchmark | Model | Score | Metric |
|---|---|---|---|
| RoboTwin2.0 ID | GroundingPI-4B | 66.20 | Success rate (%) |
| RoboTwin2.0 ID | RynnBrain1.1-2B | 10.45 | Success rate (%) |
| RoboTwin2.0 ID | PaliGemma-3B | 41.54 | Success rate (%) |
| RoboTwin2.0 ID | Cosmos-Predict2.5-2B | 47.50 | Success rate (%) |
| RoboTwin2.0 ID | Wan2.2-TI2V-5B | 51.32 | Success rate (%) |
| RoboTwin2.0 ID | Rex-Omni-3B | 54.76 | Success rate (%) |
| RoboTwin2.0 ID | Qwen3-VL-4B | 58.74 | Success rate (%) |
| RoboTwin2.0 ID | RynnBrain-2B | 59.12 | Success rate (%) |
| RoboTwin2.0 ID | LocateAnything-3B | 60.62 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | GroundingPI-4B | 17.60 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | Cosmos-Predict2.5-2B | 2.76 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | Wan2.2-TI2V-5B | 2.76 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | PaliGemma-3B | 5.10 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | LocateAnything-3B | 6.10 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | RynnBrain1.1-2B | 8.77 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | RynnBrain-2B | 9.76 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | Qwen3-VL-4B | 12.20 | Success rate (%) |
| RoboTwin2.0-Clean2Random OOD | Rex-Omni-3B | 14.10 | Success rate (%) |
| 1/nuScenes L2 1s | GroundingPI-4B | 7.52 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 1s | PaliGemma-3B | 6.28 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 1s | Rex-Omni-3B | 6.78 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 1s | RynnBrain-2B | 6.90 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 1s | RynnBrain1.1-2B | 7.19 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 1s | Qwen3-VL-4B | 7.41 | Reciprocal L2 (1/m) |
| RoboCasa-GR1 ID | GroundingPI-4B | 37.75 | Success rate (%) |
| RoboCasa-GR1 ID | Rex-Omni-3B | 25.17 | Success rate (%) |
| RoboCasa-GR1 ID | RynnBrain1.1-2B | 26.67 | Success rate (%) |
| RoboCasa-GR1 ID | Wan2.2-TI2V-5B | 32.17 | Success rate (%) |
| RoboCasa-GR1 ID | PaliGemma-3B | 32.67 | Success rate (%) |
| RoboCasa-GR1 ID | Qwen3-VL-4B | 34.67 | Success rate (%) |
| RoboCasa-GR1 ID | LocateAnything-3B | 35.08 | Success rate (%) |
| RoboCasa-GR1 ID | Cosmos-Predict2.5-2B | 36.67 | Success rate (%) |
| RoboCasa-GR1 ID | RynnBrain-2B | 39.00 | Success rate (%) |
| RoboCasa-GR1 Type OOD | GroundingPI-4B | 28.94 | Success rate (%) |
| RoboCasa-GR1 Type OOD | RynnBrain1.1-2B | 18.69 | Success rate (%) |
| RoboCasa-GR1 Type OOD | Rex-Omni-3B | 18.75 | Success rate (%) |
| RoboCasa-GR1 Type OOD | Wan2.2-TI2V-5B | 22.69 | Success rate (%) |
| RoboCasa-GR1 Type OOD | PaliGemma-3B | 24.00 | Success rate (%) |
| RoboCasa-GR1 Type OOD | Cosmos-Predict2.5-2B | 24.06 | Success rate (%) |
| RoboCasa-GR1 Type OOD | LocateAnything-3B | 26.25 | Success rate (%) |
| RoboCasa-GR1 Type OOD | Qwen3-VL-4B | 26.38 | Success rate (%) |
| RoboCasa-GR1 Type OOD | RynnBrain-2B | 28.06 | Success rate (%) |
| 1/nuScenes L2 2s | GroundingPI-4B | 3.76 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 2s | PaliGemma-3B | 3.13 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 2s | Rex-Omni-3B | 3.46 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 2s | RynnBrain-2B | 3.54 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 2s | RynnBrain1.1-2B | 3.61 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 2s | Qwen3-VL-4B | 3.72 | Reciprocal L2 (1/m) |
| RoboCasa-GR1 Appearance OOD | GroundingPI-4B | 37.78 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | RynnBrain1.1-2B | 21.20 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | Rex-Omni-3B | 22.28 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | Wan2.2-TI2V-5B | 29.68 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | LocateAnything-3B | 32.75 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | Cosmos-Predict2.5-2B | 33.45 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | Qwen3-VL-4B | 33.98 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | PaliGemma-3B | 34.53 | Success rate (%) |
| RoboCasa-GR1 Appearance OOD | RynnBrain-2B | 36.93 | Success rate (%) |
| RoboCasa-GR1 Container OOD | GroundingPI-4B | 37.86 | Success rate (%) |
| RoboCasa-GR1 Container OOD | RynnBrain1.1-2B | 21.92 | Success rate (%) |
| RoboCasa-GR1 Container OOD | Rex-Omni-3B | 27.54 | Success rate (%) |
| RoboCasa-GR1 Container OOD | LocateAnything-3B | 30.58 | Success rate (%) |
| RoboCasa-GR1 Container OOD | Qwen3-VL-4B | 32.63 | Success rate (%) |
| RoboCasa-GR1 Container OOD | Cosmos-Predict2.5-2B | 33.63 | Success rate (%) |
| RoboCasa-GR1 Container OOD | Wan2.2-TI2V-5B | 33.79 | Success rate (%) |
| RoboCasa-GR1 Container OOD | PaliGemma-3B | 34.96 | Success rate (%) |
| RoboCasa-GR1 Container OOD | RynnBrain-2B | 37.42 | Success rate (%) |
| 1/nuScenes L2 3s | GroundingPI-4B | 2.04 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 3s | PaliGemma-3B | 1.72 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 3s | Rex-Omni-3B | 1.90 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 3s | RynnBrain-2B | 1.92 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 3s | RynnBrain1.1-2B | 1.97 | Reciprocal L2 (1/m) |
| 1/nuScenes L2 3s | Qwen3-VL-4B | 2.00 | Reciprocal L2 (1/m) |
Autonomous Driving Performance
GroundingPI attains an average open-loop L2 error of 0.296 m, improving on Qwen3-VL-4B (0.301 m) and RynnBrain (0.308 m). This modest, consistent gain suggests that embodied specialization alone need not supply the perception most useful for driving. Reading road signs also motivates accurate and timely OCR in vision-based driving, although this evaluation does not isolate sign reading or measure its latency. For latency-sensitive execution, these results motivate compact perceptual foundations; they do not establish closed-loop performance or a deployment limit for larger models.
Robot Manipulation Performance
We report task success rate (SR) across two representative embodiments: fixed-base bimanual tabletop manipulation on RoboTwin2.0 and humanoid dexterous-hand manipulation on RoboCasa-GR1. In-distribution (ID) evaluation follows the full training and evaluation protocol of each benchmark. For out-of-distribution (OOD) evaluation, policies are trained on ID data and tested under held-out conditions.
Controlled Backbone Comparison and Evaluation Protocol
We evaluate vision-language and video-generation foundations spanning the backbone families used in VLA and WAM systems. To compare their contribution to action learning under controlled downstream conditions, we couple every backbone to the same layer-wise Action DiT. Following the conditioning design of π0, intermediate features from a single backbone forward pass are projected and resampled to a common interface, then injected into the corresponding cross-attention blocks of the action expert. Within each benchmark, all runs keep the Action DiT architecture, action-training data, optimization budget, action representation, and inference procedure fixed, and vary only the pretrained foundation. This controlled comparison allows us to assess how effectively each foundation supports downstream action learning and how well the resulting policies generalize out of distribution. The shared configuration and matched training budget are detailed below.
Fixed-base bimanual — RoboTwin2.0. This widely adopted benchmark spans 50 tabletop manipulation tasks. RoboTwin2.0-Full trains and evaluates on both Clean and Randomized data, whereas RoboTwin2.0-Clean2Random trains only on Clean demonstrations and evaluates on Randomized scenes. The released Randomized setting retains the same manipulation tasks but varies scene clutter, lighting, table and background textures, tabletop height, and language instructions. It therefore primarily evaluates scene robustness under environmental variation.
Humanoid dexterous-hand — RoboCasa-GR1. This benchmark evaluates fine-grained manipulation from head-camera observations without wrist cameras. Its full protocol measures ID performance, while OOD evaluation uses three held-out suites: Unseen Appearance applies novel textures to familiar object–container pairs; Unseen Combinations places seen objects in novel container pairings; and Unseen Object Types introduces novel object categories. These suites evaluate entity generalization across object appearances, object categories, and object–container combinations.
Together, the two protocols evaluate scene robustness and entity generalization, two complementary requirements of an embodied foundation model. Both require stable, task-conditioned perception before action learning, making them direct tests of whether a pretrained foundation provides reusable representations for manipulation.
Shared Configuration
| Component | Shared configuration |
|---|---|
| Backbone conditions | 8 normalized-depth hidden states |
| Condition interface | 64 tokens per depth, width 1024 |
| Action topology | 16 atomic blocks: 8 cross-attention and 8 self-attention blocks, interleaved from cross-attention |
| Residual / attention width | 1024; 16 heads; head dimension 64 |
| Feed-forward width | 4096 |
| Planning tokens | 32 |
| Context bypass | None |
| Action horizon | 16 |
| Time sampling | Beta base distribution (α,β)=(1.5,1.0), s=0.999, 1,000 timestep buckets |
| Inference | 4 Euler steps; one cached backbone forward per observation |
Matched Training Budget
| Item | Configuration |
|---|---|
| Optimization length | 40,000 updates |
| Hardware / precision | 40 accelerators; bfloat16; DeepSpeed ZeRO-2 |
| Batch size | 32 per device; 1280 global |
| Optimizer | AdamW, β1=0.95, β2=0.999, ϵ=10−8 |
| Weight decay / clipping | 10−5; gradient norm 1.0 |
| Learning rates | 10−4 for the action expert and interface; 10−5 for adapted backbone parameters |
| Schedule | Cosine decay; 2,000 warmup steps; minimum LR 5×10−7 |
| Flow samples | 2 noise samples per training observation |
| RoboTwin 2.0 data | Full: Clean and Randomized; Clean2Random: Clean training, Randomized evaluation |
| RoboCasa-GR1 data | Identical demonstration pool across foundations at each data fraction |
Overall Performance and Generalization under Complementary Shifts
Under the matched downstream architecture and training protocol, we compare backbones with distinct pretraining objectives. We organize the compared backbones into four groups: general-purpose VLMs (Qwen3-VL-4B and PaliGemma-3B), video-generation backbones (Wan2.2-TI2V-5B and Cosmos-Predict2.5-2B), grounding models (LocateAnything-3B and Rex-Omni-3B), and embodied foundation models (RynnBrain and RynnBrain1.1). Across these heterogeneous pretraining sources, GroundingPI ranks first in five of the six evaluations, including RoboTwin2.0 Full and every OOD setting. It achieves 66.20% on RoboTwin2.0 Full and remains competitive on RoboCasa-GR1 Full at 37.75%, compared with RynnBrain’s 39.00%. These results support precise spatial perception as an effective foundation for action learning across two robot embodiments, with consistent performance under distribution shift.
RoboTwin2.0-Clean2Random evaluates scene robustness, while RoboCasa-GR1 evaluates entity generalization across appearances, object types, and object–container relations. GroundingPI ranks first in all four OOD settings. On RoboTwin2.0, it reaches 17.60%, with Rex-Omni second at 14.10%. On RoboCasa-GR1, RynnBrain ranks second across Type, Appearance, and Container OOD. Rex-Omni’s strength in scene robustness is consistent with its emphasis on detection, referring, and coordinate prediction. RynnBrain’s strength in entity generalization is consistent with its broader language-conditioned embodied and semantic understanding, including spatiotemporal localization and physically grounded reasoning. These observations highlight the complementary strengths of precise perceptual grounding and broader embodied semantic understanding.
Benchmark results
Loading results…
| # | Model | Score | Zero-shot |
|---|
Unreported and protocol-mismatched results are excluded.
Zero-shot: YES / NO / UNK (unspecified). External references are listed separately.
Details ↗Value details
Perception Before Action
The diagram illustrates a conceptual progression from action-only adaptation of general VLMs, through perceptual supervision in the π family, to grounding as a native foundation capability. GroundingPI retains broad VQA-style, language-conditioned semantic knowledge while emphasizing precise spatial perception through grounding and pointing supervision. Visual primitives provide an interface for physical prompting, specifying targets, locations, and structures alongside language. Together with precise spatial perception, this interface supports scene robustness and entity generalization. Our transfer results evaluate the pretrained foundation for physical intelligence rather than a separate prompting intervention.
Data Efficiency
We evaluate data efficiency under the RoboCasa-GR1 Full/ID protocol using 25%, 50%, 75%, and 100% of the robot demonstrations. We vary only the amount of action data, keeping the architecture, action-learning approach, and other training choices unchanged. GroundingPI leads all three reduced-data settings. At 50% data, its 28.75% SR exceeds every baseline at 75%, where RynnBrain achieves the highest score of 27.75%.
These results support prioritizing precise spatial perception before action learning. The same physical-prompting interface allows scarce action demonstrations to focus on control rather than relearning basic perception. Together with the preceding results, this suggests that precise grounding supports effective and generalizable action learning as well as more data-efficient adaptation.
Success rate at four demonstration fractions with the remaining action-learning setup fixed.
Open figure ↗Value details
| Model | Demonstrations | Success rate |
|---|---|---|
| RynnBrain-2B | 75% | 27.75% |
| RynnBrain-2B | 100% | 39.00% |
| GroundingPI-4B | 50% | 28.75% |
| GroundingPI-4B | 100% | 37.75% |
Effects of Grounding Training
Value details
| Exposure order | Grounding Avg | RoboCasa-GR1 ID SR | RoboCasa-GR1 OOD Avg SR |
|---|---|---|---|
| 1 | |||
| 2 | |||
| 3 | |||
| 4 (full) |
QuestionHow does pretraining scale affect perceptual capability and its transfer to action learning?
FindingScaling grounding pretraining strengthens both grounding quality and downstream action learning, with generalization tracking the perceptual gains most closely.
What Transfers from Grounding to Action?
Value details
| Included groups | RoboCasa-GR1 ID | RoboCasa-GR1 OOD Avg | nuScenes L2 Avg |
|---|---|---|---|
| A+D+F | |||
| A+B+C+F | |||
| A+B+C+D+F | |||
| A+B+D+E+F | |||
| A+C+D+E+F | |||
| A+B+C+D+E | |||
| F | |||
| All six |
QuestionWhich grounding data compositions best support effective and generalizable downstream action learning?
FindingDense grounding provides the strongest spatial core in the available leave-one-out comparisons, while auxiliary OCR, layout, and GUI supervision is most valuable in combination; the full mixture performs best across manipulation and driving.
TakeawayGrounding transfer depends on both scale and composition: scale strengthens the perceptual foundation, while complementary supervision determines how effectively it supports downstream action.
Resources
Models & Inference
Evaluation & Training
Citation
@misc{yu2026groundingpigroundingfoundationmodel,
title = {{GroundingPI}: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives},
author = {Qize Yu and Lianrui Fan and Boyu Chen and Jiaqi Liang and Xini Ding and Yue Chen and Zetian Song and Yuran Wang and Yi Zou and Kaixuan Wang and Tianxing Chen and Wenxuan Song and Bohan Zhou and Mingleyang Li and Siqiao Huang and Yuqi Ye and Caigao Jiang and Wei Wei and Ruihai Wu and Hang Zhang and Yixiao Ge and Shuchang Zhou and Shilong Liu and Xianming Liu and Ping Luo and Shiyu Huang},
year = {2026},
eprint = {2609.39601},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2609.39601},
}