By Kaixaun Lu, Mehmet Onurcan Kaya, Dim P. Papadopoulos.
This repository is an official implementation of the paper AutoQ-VIS: Improving Unsupervised Video Instance Segmentation via Automatic Quality Assessment and Boosting Unsupervised Video Instance Segmentation with Automatic Quality-Guided Self-Training.
TL; DR. We propose AutoQ-VIS, an unsupervised video instance segmentation framework that eliminates manual labeling via automatic quality assessment and self-training loops.
Abstract. Video Instance Segmentation (VIS) faces significant annotation challenges due to its dual requirements of pixel-level masks and temporal consistency labels. While recent unsupervised methods like VideoCutLER eliminate optical flow dependencies through synthetic data, they remain constrained by the synthetic-to-real domain gap. We present AutoQ-VIS, a novel unsupervised framework that bridges this gap through quality-guided self-training. Our approach establishes a closed-loop system between pseudo-label generation and automatic quality assessment, enabling progressive adaptation from synthetic to real videos. Experiments demonstrate state-of-the-art performance with 52.6
See installation instructions.
See Preparing Datasets for AutoQ-VIS.
Download the pre-trained VideoCutLer from this link and then place it in the AutoQ-VIS home directory.
Run script train_Qpredictor.py to initialize the quality predictor. You can download our trained weight from here.
Run script train_Mask2former.py to train the VIS model. You can download our trained weight from here.
Run scrip eval.py to evaluate the model.
