[Paper Review] Revisiting Open World Object Detection

Xiaowei Zhao, Xianglong Liu+4 more|arXiv (Cornell University)|Jan 3, 2022

Advanced Neural Network Applications4 citations

TL;DR

This paper revisits Open World Object Detection (OWOD) by identifying flaws in the sole prior benchmark and proposing five foundational principles for fair benchmark construction. It introduces two novel metrics—Unknown Detection Recall (UDR) and Unknown Detection Precision (UDP)—and a new framework with a non-parametric Proposal ADvisor (PAD) and Class-specific Expelling Classifier (CEC) that improves unknown detection and reduces overconfidence. Experiments on the proposed fair benchmark show state-of-the-art performance across both standard and new metrics.

ABSTRACT

Open World Object Detection (OWOD), simulating the real dynamic world where knowledge grows continuously, attempts to detect both known and unknown classes and incrementally learn the identified unknown ones. We find that although the only previous OWOD work constructively puts forward to the OWOD definition, the experimental settings are unreasonable with the illogical benchmark, confusing metric calculation, and inappropriate method. In this paper, we rethink the OWOD experimental setting and propose five fundamental benchmark principles to guide the OWOD benchmark construction. Moreover, we design two fair evaluation protocols specific to the OWOD problem, filling the void of evaluating from the perspective of unknown classes. Furthermore, we introduce a novel and effective OWOD framework containing an auxiliary Proposal ADvisor (PAD) and a Class-specific Expelling Classifier (CEC). The non-parametric PAD could assist the RPN in identifying accurate unknown proposals without supervision, while CEC calibrates the over-confident activation boundary and filters out confusing predictions through a class-specific expelling function. Comprehensive experiments conducted on our fair benchmark demonstrate that our method outperforms other state-of-the-art object detection approaches in terms of both existing and our new metrics. Our benchmark and code are available at https://github.com/RE-OWOD/RE-OWOD.

Motivation & Objective

• To identify and correct fundamental flaws in the only existing OWOD benchmark, including illogical construction, confusing metrics, and inappropriate method implementation.
• To establish five core benchmarking principles that ensure fairness and validity in OWOD evaluation.
• To propose two new evaluation protocols—UDR and UDP—specifically designed to assess detection performance from the perspective of unknown classes.
• To develop a novel, effective OWOD framework that enhances detection of unknown instances while maintaining known class performance.
• To validate the proposed framework and metrics through comprehensive experiments on a newly constructed, fair benchmark.

Proposed method

• Introduces a non-parametric Proposal ADvisor (PAD) module that assists the RPN in identifying accurate unknown proposals without supervision, improving localization of unknown objects.
• Proposes a Class-specific Expelling Classifier (CEC) that uses a class-specific expelling function to recalibrate activation boundaries and filter out confusing predictions.
• Employs a confirmation and guiding mechanism in PAD to refine proposal quality and reduce background misclassification.
• Designs a two-stage training protocol where the RPN is guided by PAD to better distinguish unknowns from background, while CEC reduces overconfidence in known class predictions.
• Uses a hyper-parameter sensitivity analysis to tune the CEC’s IOU threshold (φ=0.9) and confidence threshold (α=0.5) for optimal balance between precision and mAP.
• Constructs a new, fair benchmark based on the five proposed principles, ensuring logical class progression and consistent evaluation across incremental tasks.

Experimental results

Research questions

• RQ1 What are the fundamental flaws in the existing OWOD benchmark and evaluation protocol that undermine the validity of prior results?
• RQ2 How can a fair and logically consistent benchmark be constructed for OWOD, ensuring alignment with the open-world learning definition?
• RQ3 What new evaluation metrics are needed to fairly assess detection performance from the perspective of unknown classes?
• RQ4 Can a non-parametric module like PAD improve unknown proposal detection without supervision?
• RQ5 Can a class-specific expelling mechanism (CEC) effectively reduce overconfidence and improve discrimination between unknowns and known classes?

Key findings

• The proposed framework achieves a UDR of 21.20 and UDP of 36.74 on the new benchmark, significantly outperforming the baseline ORE model (UDR: 20.57, UDP: 16.43).
• The PAD module alone improves UDR by 0.63 points, demonstrating its effectiveness in identifying accurate unknown proposals.
• The CEC module improves UDP by 14.31 points over the baseline, showing strong enhancement in unknown discrimination.
• The ablation study confirms that both PAD and CEC contribute independently and synergistically, with the full model achieving peak performance.
• Sensitivity analysis shows that with α=0.5 and φ=0.9, the model maintains mAP within 1% drop while achieving a 14.31-point UDP gain.
• The new benchmark and code are publicly released at https://github.com/RE-OWOD/RE-OWOD to support reproducible and fair future research.