DeTAL: Open-Vocabulary Temporal Action Localization with Decoupled Networks

This repository contains the code for DeTAL: Open-Vocabulary Temporal Action Localization with Decoupled Networks paper.

The project was modified from ActionFormer, thanks for their wonderful work!

• Recommended Environment: python >= 3.8, cuda 11.3, PyTorch 1.10.2
• Install dependencies: pip install -r requirements.txt
• Install NMS(Non-Maximum Suppression) by following steps: cd ./libs/utils; python setup.py install --user; cd ../..

Download features and annotations from Thumos_i3d

Download features and annotations from ActivityNet_i3d

Unpack the zip folder and modified as './data', the folder structure should look like

DeTAL-release/
  ├── data
  │   ├── anet_1.3
  │   │   ├── annotations
  │   │   ├── i3d_features
  │   │   ├── tsp_features
  │   └── thumos
  │       ├── annotations
  │       ├── i3d_features
  ├── libs
  ├── tools
  ├── eval.py
  ├── train.py
  └── ...

Modified the config file './configs/thumos_i3d.yaml'

python ./train.py ./configs/thumos_i3d.yaml --output rpn_split_x

python ./eval.py ./configs/thumos_i3d.yaml ./ckpt_base/thumos_i3d_rpn_split_x/

ModiFied the file_path in 'cls_heads.py'

python ./eval.py ./configs/thumos_i3d.yaml ./ckpt_base/thumos_i3d_rpn_split_x/