SpotSound

This repository provides the official PyTorch implementation of "SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding".

🌐 Project Page $\cdot$ 📄 Paper $\cdot$ 🤗 Model $\cdot$ 🤗 Benchmark

• [2026.07] SpotSound accepted to ACM MM 2026!
• [2026.04] Released the inference code.
• [2026.04] Preprint available on arXiv.

conda create -n SpotSound python=3.10
conda activate SpotSound
pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

1. Download Model Checkpoints

   • Obtain the pretrained checkpoints from Audio Flamingo 3 and SpotSound.
   • Set pretrain_model to your local path for Audio Flamingo 3, and checkpoint to your SpotSound checkpoint.

2. Run Inference

   • Execute the following command to perform audio temporal grounding inference.

   export CUDA_VISIBLE_DEVICES=0 
   python inference.py --pretrain_model path_to_audioflamingo3 \
        --checkpoint ckpt/spotsound \
        --audio_path data/audio.wav \
        --query dog barking

3. Dataset Structure

   • All samples are stored in a list, where each sample is a dictionary.

   [{
      "audio_path": "_Uro9suV3xU_130_187.wav",
      "caption": "hair dryer drying",
      "answer": [[8.1, 10.9]]
   },...]
   

4. Train

   • Execute the following command to perform audio temporal grounding model training.

   export CUDA_VISIBLE_DEVICES=0 
   python train.py --pretrain_model path_to_audioflamingo3 \
        --output_dir path_to_save \
        --train_data path_to_train_data_json_file \
        --val_data path_to_valid_data_json_file 

If you use this code and data for your research or project, please cite:

@article{sun2026spotsound,
  title={SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding},
  author={Sun, Luoyi and Zhou, Xiao and Li, Zeqian and Zhang, Ya and Wang, Yanfeng and Xie, Weidi},
  journal={arXiv preprint arXiv:2604.13023},
  year={2026}
}

This project builds upon several excellent open-source efforts:

• Audio Flamingo 3
• UniTime.

For questions, please contact: loiesun411@gmail.com.