Abstract
Embodied navigation connects intelligent agents with the physical world and serves as a fundamental capability for achieving general robotic intelligence. However, the availability and quality of navigation data have long limited the generalization ability of Vision-and-Language Navigation (VLN) systems and their capacity to handle long-horizon tasks. To bridge this gap, we curate diverse 3D scene resources and develop an automated pipeline for large-scale navigation data generation, resulting in the GN-Matrix dataset. Building upon a 3D Gaussian Splatting (3DGS) rendering engine, we further introduce a high-fidelity simulation platform that supports interactive roaming and collision-aware navigation. Furthermore, we propose GN-Bench, the first benchmark to provide BEV-based evaluation, incorporating dynamic 3DGS avatars to establish new standards for human-robot interaction evaluation.
To fully leverage the interactive nature of the simulator, we develop an RL-driven navigation foundation model, Break and Establish (BAE), after supervised learning, DAgger exposes the model to rollout-induced states and recovery behaviors beyond idealized offline demonstrations, breaking the narrow expert-centric distribution inherited from supervision and providing a better starting point for downstream RL exploration. This unified VLN paradigm seamlessly integrates both map-based and map-free settings, enabling a broad spectrum of tasks including instruction following, human following, and goal navigation. To the best of our knowledge, this work is the first to formalize high-fidelity 3DGS-rendered Bird’s Eye View (BEV) representations as a compact and efficient memory mechanism, thereby unlocking the latent spatial reasoning capabilities of Vision-and-Language Models (VLMs). The dual-system architecture of GN0 ensures both flexibility and high-efficiency deployment in real-world scenarios.
Extensive evaluations on GN-Bench and VLN-CE, encompassing both quantitative and qualitative analyses, demonstrate that the proposed GN0 significantly outperforms existing state-of-the-art VLN methods. Overall, this work, GN-Matrix, presents a unified framework spanning data, simulation, and learning, offering new insights for advancing embodied navigation in both academic research and industrial applications.
GN0
Break and Establish (BAE) is a unified embodied navigation foundation model for instruction following, human following, and goal-directed navigation. It supports FPV, BEV, and hybrid visual inputs, while jointly modeling both discrete action sequences and continuous trajectories within a single framework. At its core, BAE adopts a trajectory tokenization scheme that unifies planning representation and improves training efficiency across diverse navigation settings.
Combined with high-fidelity rendered data and supervised fine-tuning, this design effectively unlocks the navigation capability of the underlying vision-language model. A key component is our closed-loop post-training pipeline: after supervised learning, DAgger exposes the model to rollout-induced states and recovery behaviors beyond idealized offline demonstrations, breaking the narrow expert-centric distribution inherited from supervision and providing a better starting point for downstream exploration.
Building on this initialization, DAPO further refines the policy with execution-oriented rewards, enabling more effective exploration, stronger long-horizon decision-making, and more reliable action generation during rollout. Lightweight Action Experts further bridge high-level planning and executable control for practical deployment.
Navigation Data Generation
Navigation Task Generation Pipeline
To support large-scale embodied navigation in photorealistic 3D Gaussian Splatting (3DGS) environments, we construct GN-Matrix , a large-scale navigation dataset covering diverse indoor scenes, dynamic human-centered environments, and benchmark-compatible data formats for multiple navigation settings. To build GN-Matrix, we develop a unified data generation pipeline that integrates multi-source 3DGS scene assets, animated 3DGS human avatars, task synthesis, and multimodal annotation. This pipeline is designed not only to increase the scale of navigation data, but also to improve scene diversity, task coverage, and realism.
The key characteristics of GN-Matrix are summarized as follows:
Closed-loop Simulation and Benchmark
GN-Bench is a high-fidelity embodied navigation benchmark built on 3D Gaussian Splatting indoor environments, designed to provide a unified platform for both policy evaluation and data generation under realistic closed-loop interaction. Beyond photorealistic egocentric RGB observations, GN-Bench additionally supports metric-consistent Bird's Eye View (BEV) rendering, enabling a unified benchmark for both map-free and map-conditioned navigation paradigms. The simulator integrates rendering, motion execution, collision handling, and geodesic planning into a single interactive loop, allowing agents to be evaluated under physically constrained online execution rather than static offline replay. Importantly, GN-Bench also provides oracle planning and online relabeling capabilities, which make it possible to construct DAgger-style corrective supervision directly from policy rollouts. This allows the benchmark to serve not only as an evaluation substrate, but also as a practical training environment for studying recovery behaviors, covariate-shift mitigation, and closed-loop policy improvement in embodied navigation.
GN-Bench Experiments
On GN-Bench, the Stage-1 pretrained BAE already performs slightly better than strong baselines such as fine-tuned UniNaVid and InternNav S2. With the full closed-loop pipeline, DAgger and DAPO bring substantial gains, improving both SR and SPL by more than 10 points over the Stage-1 model. Our ablations also reveal that BEV understanding does not emerge automatically from Stage 1 alone: models with BEV input can initially underperform their FPV-only counterparts, but after DAgger and DAPO, the advantage of BEV becomes increasingly clear. Finally, we find that directly applying GRPO-style optimization after Stage 1 is unstable. Without the entropy increase introduced by DAgger, the initial per-token exploration entropy remains extremely low (0.003 vs. 0.06 nats/token), causing DAPO training to collapse and even degrade performance below the Stage-1 model.
VLN-CE Experiments
We evaluate GN-BAE on the standard R2R-CE benchmark using the Validation Unseen split. As shown in table below, GN-BAE achieves the best overall performance, reaching 3.50 NE, 75.6 OS, 67.7 SR, and 63.4 SPL using only single-view RGB input. Compared with the strongest prior RGB-only baseline, GN-BAE reduces NE by 17.5%(from 4.24 to 3.50m) while improving SR and SPL by 2.6 and 1.1 points, respectively, indicating better goal localization and trajectory efficiency.
Supported Robots
TeleBot-W
TeleBot-L
TeleBot-M
