Paper

1. setup python env & clone this repo

   conda create -n descrl python=3.9 cmake=3.14.0 -y
   conda activate descrl
   git clone https://github.com/ak-lab-titech/DescRL
   

2. setup habitat-lab

   cd habitat-lab
   python setup.py develop --all
   

3. setup habitat-sim (reference)
4. setup soundspaces

   cd sound-spaces
   pip install -e .
   

   • And also you need to download some datasets (reference)

1. Pre-train ADGenerator
2. edit xgenerator/shs/train.sh
3. run the followings:

   cd xgenerator
   conda activate descrl
   bash ./shs/train.sh
   

4. Pre-train ADPredictor
   1. edit sound-spaces/train.sh
      • here, MODEL should be savi-iprl-pre-offpolicy
   2. run the followings:

      cd sound-spaces
      conda activate descrl
      bash ./train.sh
      

5. Jointly training
   1. run the followings:

      cd sound-spaces
      conda activate descrl
      bash ./train.sh
      

Edit sound-spaces/eval.sh and run the followings:

cd sound-spaces
conda activate descrl
bash ./eval.sh

Edit sound-spaces/test.sh and run the followings:

cd sound-spaces
conda activate descrl
bash ./test.sh

@inproceedings{kondoh2025descrl,
  title={Embodied Navigation with Auxiliary Task of Action Description Prediction},
  author={Kondoh, Haru and Kanezaki, Asako},
  booktitle={Proceedings of International Conference on Computer Vision (ICCV)},
  year={2025},
}