Dongyue Lu^1,2   Lingdong Kong¹   Gim Hee Lee¹   Camille Simon Chane³   Wei Tsang Ooi^1,2

¹National University of Singapore   ²IPAL, CNRS IRL 2955, Singapore
³ETIS UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France

FlexEvent is an event-frame object detector designed to work across varying operational frequencies. It combines the temporal resolution of event cameras with the semantic detail of RGB frames, allowing the same detector to operate from standard frame rates up to 180 Hz.

Our main contributions are:

• FlexEvent, a flexible event-frame detection framework for real-world sensing at varying operational frequencies.
• FlexFuse, an adaptive fusion module that combines recurrent event features with multi-scale RGB features.
• FlexTune, a frequency-adaptive fine-tuning method that uses frequency-adjusted labels to improve temporal generalization.

conda create -n flexevent python=3.9 -y
conda activate flexevent

python -m pip install \
  torch==2.1.2 torchvision==0.16.2 \
  --index-url https://download.pytorch.org/whl/cu121
python -m pip install -r requirements.txt

Follow the data preparation instructions in DSEC-Detection.

Then generate events_2x.h5 for every sequence with the script included here:

export DSEC_ROOT=/path/to/dsec
for split in train test; do
  for sequence in "$DSEC_ROOT/$split"/*; do
    python scripts/downsample_events.py \
      "$sequence/events/left/events.h5" \
      "$sequence/events/left/events_2x.h5"
  done
done

Your final data directory should look like this:

DSEC_ROOT/
├── train/<sequence>/
└── test/<sequence>/
    ├── images/
    │   ├── timestamps.txt
    │   └── left/distorted/*.png
    ├── events/left/events_2x.h5
    └── object_detections/left/tracks.npy

We provide two label settings:

• 2class: car and pedestrian. The original car, bus, and truck categories are merged into car.
• 8class: the native DSEC categories. Since the train category has no annotations in the DSEC-Det split, the released model evaluates the seven non-empty categories.

Download the released weights from Hugging Face and place them in checkpoints/.

For each label setting, we provide two variants:

• Global is a FlexFuse model trained only with ground-truth labels. It generally performs best at the standard 20 Hz operating frequency.
• Balanced adds FlexTune and is further trained with pseudo-labels at intermediate timestamps. It performs better at high frequencies and is more consistent across the full frequency range.

All values below are COCO AP@[0.50:0.95].

python test_global.py \
  dataset=dsec \
  dataset.path=/path/to/dsec \
  dataset.class_mode=2class \
  checkpoint=checkpoints/flexevent_2class_global.pth \
  hardware.gpus=0

Use dataset.class_mode=8class with an 8-class checkpoint.

Our interframe time sampling and interpolated labels follow DAGR. It reports the empty-event point and nine event rates from 180 Hz to 20 Hz. Intermediate annotations are generated online from the official DSEC-Det tracks by matching track IDs and interpolating the bounding boxes between two annotated frames. No additional label-generation step is required.

python test_interframe.py \
  dataset=dsec \
  dataset.path=/path/to/dsec \
  dataset.class_mode=2class \
  checkpoint=checkpoints/flexevent_2class_balanced.pth \
  hardware.gpus=0 \
  validation.interframe=true

visualize.py runs one test sequence and renders RGB, events, ground truth and predictions together. Use an image extension for one frame or a video extension for a sequence:

python visualize.py \
  dataset=dsec \
  dataset.path=/path/to/dsec \
  dataset.class_mode=2class \
  checkpoint=checkpoints/flexevent_2class_balanced.pth \
  hardware.gpus=0 \
  visualization.sequence=thun_01_a \
  visualization.output=demo.mp4

This code builds on RVT, DSEC-Detection, and DAGR.

Training code will be released soon.