If you have any question (code sharing, weight, data split etc), please feel free to contact me via email at sunoh0131@gmail.com.
Open-World Object Detection method enhanced by Vision Foundation Models for better detection performance and effective feature similarity recognition among detected instances.
Open World Object Detection (OWOD) addresses realistic scenarios where unseen object classes emerge, enabling detectors trained on known classes to detect unknown objects and incrementally incorporate the knowledge they provide. While existing OWOD methods primarily focus on detecting unknown objects, they often overlook the rich semantic relationships between detected objects, which are essential for scene understanding and applications in open-world environments (e.g., open-world tracking and novel class discovery). In this paper, we extend the OWOD framework to jointly detect unknown objects and learn semantically rich instance embeddings, enabling the detector to capture fine-grained semantic relationships between instances. To this end, we propose two modules that leverage the rich and generalizable knowledge of Vision Foundation Models (VFM). First, the Unknown Box Refine Module uses instance masks from the Segment Anything Model to accurately localize unknown objects. The Embedding Transfer Module then distills instance-wise semantic similarities from VFM features to the detector’s embeddings via a relaxed contrastive loss, enabling the detector to learn a semantically meaningful and generalizable instance feature. Extensive experiments show that our method significantly improves both unknown object detection and instance embedding quality, while also enhancing performance in downstream tasks such as open-world tracking.
2025/11/11: 🚀 OW-Rep was accepted to WACV 2026
Here are some example outputs:
Option A: Using Docker (Recommended) You can easily set up the environment by pulling the pre-built Docker image:
docker pull sunoh0131/ow-rep:latest
Note: Alternatively, you can manually install the environment by following the "Requirements and Installation" guide in the PROB GitHub repository.
You can download coco dataset and pascal dataset into the data/ directory.
You can refer to "Dataset Preparation" part in the PROB GitHub repository.
Previous and our dataset splits are in the data/OWOD/ImageSets directory.
Dataset folder structure:
workspace/
└── data/
└── OWOD/
├── JPEGImages
├── Annotations
└── ImageSets
├── OWDETR
├── TOWOD
└── VOC2007
You can download the self-supervised backbone model from here and place the .pth file inside the models/ directory.
- Tested in A100 80GB
- For inference, SAM and DINO features or their models are not required.
If you find this code useful for your research, please consider citing us:
@inproceedings{lee2026ow,
title={OW-Rep: Open World Object Detection with Instance Representation Learning},
author={Lee, Sunoh and Jeon, Minsik and Min, Jihong and Seo, Junwon},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
pages={339--349},
year={2026}
}
If you have any question, please contact me via email at sunoh0131@gmail.com.
OW-Rep builds on previous works' code bases such as PROB, Deformable DETR, Detreg, LabelRelaxation and OWOD. If you found OW-Rep useful please consider citing these works as well.
