• [2025.09.30] 📣 ITTO is released.
• [2025.09.18] 🥳 ITTO accepted at NeurIPS 2025!

ITTO is a challenging new benchmark suite for evaluating and diagnosing tracking point tracking methods. Our videos are sourced from existing datasets and egocentric real-world recordings, with high-quality human annotations collected through a multi-stage pipeline. ITTO captures the motion complexity, occlusion patterns, and object diversity characteristic of real-world scenes.

We provide:

1. Benchmark Dataset: A long-range and real-world tracking benchmark with novel motion complexity to evaluate state-of-the-art tracking methods.
2. Evaluation: A rigorous evaluation protocol along defined axes of motion complexity.
3. Annotation Pipeline: A plug-and-play annotation pipeline for collecting high-quality video track annotations.

1. Clone the dataset repository:

   git clone https://huggingface.co/datasets/your-username/your-dataset
   cd your-dataset

2. Follow the download instructions in the HuggingFace repo to get the full dataset
3. Then use our dataloaders and evaluation scripts:

We provide dataloaders for loading ITTO annotations. You can test your dataset installation by running our example dataloader script:

python dataloaders/check_dataloaders.py

We provide model evaluation scripts that were used to produce the numbers reported in the paper in the evaluation_scripts folder, which also contains per-model installation instructions.

See evaluation_scripts.md for instructions on how to run model evals. We provide separate, instructions for each of the models we evaluate in the ITTO paper:

• CoTracker3: evaluation_scripts/cotracker
• DELTA: evaluation_scripts/delta
• LocoTracker: evaluation_scripts/locotracker
• SceneTracker: evaluation_scripts/scenetracker
• SpatialTracker: evaluation_scripts/spatialtracker
• TapNet/TapNext/BootsTAP/BootsTAPIR: evaluation_scripts/tapnet
• TapTr: evaluation_scripts/taptr

Our model evals are based on CoTracker3, DELTA, LocoTracker, SceneTracker, SpatialTracker, TapNet, and TapTr. We thank the authors for their excellent work!

This code is released for academic and research purposes only under CC BY-NC 4.0. Commercial use is prohibited due to dependencies on non-commercial third-party code. See LICENSE.md for full details.

@inproceedings{Demler_2025_Neurips,
    author    = {Demler, Ilona and Chauhan, Saumya and Gkioxari, Georgia},
    title     = {Is This Tracker On? A Benchmark Protocol for Dynamic Tracking},
    booktitle = {39th Conference on Neural Information Processing Systems (NeurIPS 2025) Track on Datasets and Benchmarks},
    month     = {December},
    year      = {2025},
    pages     = {19446-19455}
}