Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports

Haopeng Li¹, Andong Deng², Jun Liu*³, Hossein Rahmani³, Yulan Guo⁴, Bernt Schiele⁵, Mohammed Bennamoun⁶, Qiuhong Ke*⁷

¹University of Melbourne, ²University of Central Florida, ³Lancaster University, ⁴Sun Yat-sen University, ⁵Max Planck Institute for Informatics, ⁶University of Western Australia, ⁶Monash University,

Reasoning over sports videos for question answering is an important task with numerous applications, such as player training and information retrieval. However, this task has not been explored due to the lack of relevant datasets and the challenging nature it presents. Most datasets for video question answering (VideoQA) focus mainly on general and coarse-grained understanding of daily-life videos, which is not applicable to sports scenarios requiring professional action understanding and fine-grained motion analysis. In this paper, we introduce the first dataset, named Sports-QA, specifically designed for the sports VideoQA task. The Sports-QA dataset includes various types of questions, such as descriptions, chronologies, causalities, and counterfactual conditions, covering multiple sports.

The numbers of QA pairs for different types and different sports.

The distributions of answer classes broken down by question types.

Examples from Sports-QA.

Benchmark on Sports-QA

• Videos: HuggingFace
• QA pairs (meta-data.zip) and extracted features (*.h5): figshare

If you find our work useful in your research, please consider giving a star and citation.

@article{li2024sports,
  title={Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports},
  author={Li, Haopeng and Deng, Andong and Ke, Qiuhong and Liu, Jun and Rahmani, Hossein and Guo, Yulan and Schiele, Bernt and Chen, Chen},
  journal={arXiv preprint arXiv:2401.01505},
  year={2024}
}

This work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!

• MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions
• FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding
• IGV: Invariant Grounding for Video Question Answering

For further questions, please contact Haopeng Li (haopeng.li@student.unimelb.edu.au).