DualMap is an online open-vocabulary mapping system that enables robots to understand and navigate dynamic 3D environments using natural language.

The system supports multiple input sources, including offline datasets (Dataset Mode), ROS streams & rosbag files (ROS Mode), and iPhone video streams (Record3d Mode). We provide examples for each input type.

[2025.12] We have added detailed experiment results of dynamic navigation for better reproduction. Check Table IX and X in the updated Appendix.

[2025.08] Full code released! 🎉 Welcome to use, share feedback, and contribute.

✅ Tested on Ubuntu 22.04 with ROS 2 Humble and Python 3.10

git clone --branch main --single-branch --recurse-submodules [email protected]:Eku127/DualMap.git
cd DualMap

Make sure to use --recurse-submodules to get mobileclip.

conda env create -f environment.yml
conda activate dualmap

cd 3rdparty/mobileclip
pip install -e . --no-deps
cd ../..

The system currently defaults to MobileCLIP-v1, and all reported results are based on v1. Since August 2025, MobileCLIP-v2 has been released, and the system also supports v2. You can set up the v2 environment by following the instructions in Apple’s MobileCLIP repository.

Setting up ROS2 environment for ROS support and applications. We recommend ROS 2 Humble. Once installed, activate the environment:

source /opt/ros/humble/setup.bash

DualMap’s navigation functionality and real-world integration are based on ROS 2. Installation is strongly recommended.

ROS1 noetic is also supported, you can setup the ROS 1 in Ubuntu 22.04 by follow this guide.

Habitat Data Collector is a tool built on top of the Habitat-sim. It supports agent control, object manipulation, dataset and ROS2 bag recording, as well as navigation through external ROS2 topics. DualMap subscribes to live ROS2 topics from the collector for real-time mapping and language-guided querying, and publishes navigation trajectories for the agent to follow.

For the best DualMap experience (especially interactive mapping and navigation), we strongly recommend setting up the Habitat Data Collector. See the repo for installation and usage details.

Here's a quick overview of the requirements for each application type:

• ROS: Please install either ROS1 or ROS2 based on your needs.
• Habitat Data Collector: Currently, it only supports ROS2.

DualMap supports running with offline datasets. Currently supported datasets include:

1. Replica Dataset
2. ScanNet Dataset
3. TUM RGB-D Dataset
4. Self-collected data using Habitat Data Collector

For data collected from your own platform, you can organize it in a similar format to run the system.

Follow the Dataset Runner Guide to arrange datasets, run DualMap with these datasets and reproduce our offline mapping results in Table II in our paper.

DualMap supports input from both ROS1 and ROS2. You can run the system with offline rosbags or in online mode with real robots.

Follow the ROS Runner Guide to get started with running DualMap using ROS1/ROS2 rosbags or live ROS streams.

DualMap supports online interactive mapping and object navigation in simulation via the Habitat Data Collector.

Follow the Online Mapping and Navigation Guide to get started with running DualMap in interactive simulation scenes and to reproduce the navigation results (both static and dynamic) in Table III in our paper.

DualMap supports real-time data streaming from the Record3D app on iPhone.

Follow the iPhone Runner Guide to get started with setting up Record3D, streaming data to DualMap, and mapping with your own iPhone!

We provide two prebuilt map examples for offline querying: one from iPhone data and one from Replica Room 0.

Follow the Offline Query Guide to run the query application.

The system supports both Rerun and Rviz visualization. When running with ROS, you can switch the visualizaiton via use_rerun and use_rviz option in config/runner_ros.yaml

If you find our work helpful, please consider starring this repo 🌟 and cite:

@ARTICLE{jiang2025dualmap,
  author={Jiang, Jiajun and Zhu, Yiming and Wu, Zirui and Song, Jie},
  journal={IEEE Robotics and Automation Letters},
  title={DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes},
  year={2025},
  volume={10},
  number={12},
  pages={12612--12619},
  doi={10.1109/LRA.2025.3621942}
}

For technical questions, please create an issue. For other questions, please contact the first author: jjiang127 [at] connect.hkust-gz.edu.cn

We are grateful to the authors of HOVSG and ConceptGraphs for their contributions and inspiration.

Special thanks to @TOM-Huang for his valuable advice and support throughout the development of this project.

We also thank the developers of MobileCLIP, CLIP, Segment Anything (SAM), MobileSAM, FastSAM, and YOLO-World for their excellent open-source work, which provided strong technical foundations for this project.