[Paper Review] NOD: Taking a Closer Look at Detection under Extreme Low-Light Conditions with Night Object Detection Dataset

Igor Morawski, Yuan Chen+2 more|arXiv (Cornell University)|Oct 20, 2021

Impact of Light on Environment and Health13 citations

TL;DR

This paper introduces the NOD dataset, a large-scale, high-quality collection of night-time urban scenes with instance-level annotations for extreme and non-extreme low-light conditions. It proposes a detection-with-enhancement framework that integrates a learning-based image enhancement module trained jointly with the object detector, along with novel data augmentation techniques, achieving consistent performance gains across multiple detectors and low-light benchmarks, especially under extreme illumination challenges.

ABSTRACT

Recent work indicates that, besides being a challenge in producing perceptually pleasing images, low light proves more difficult for machine cognition than previously thought. In our work, we take a closer look at object detection in low light. First, to support the development and evaluation of new methods in this domain, we present a high-quality large-scale Night Object Detection (NOD) dataset showing dynamic scenes captured on the streets at night. Next, we directly link the lighting conditions to perceptual difficulty and identify what makes low light problematic for machine cognition. Accordingly, we provide instance-level annotation for a subset of the dataset for an in-depth evaluation of future methods. We also present an analysis of the baseline model performance to highlight opportunities for future research and show that low light is a non-trivial problem that requires special attention from the researchers. Further, to address the issues caused by low light, we propose to incorporate an image enhancement module into the object detection framework and two novel data augmentation techniques. Our image enhancement module is trained under the guidance of the object detector to learn image representation optimal for machine cognition rather than for the human visual system. Finally, experimental results confirm that the proposed method shows consistent improvement of the performance on low-light datasets.

Motivation & Objective

• To address the non-trivial challenge of object detection under extreme low-light conditions, which current models fail to handle uniformly.
• To provide a large-scale, high-quality dataset with fine-grained lighting condition annotations (extreme vs. non-extreme low-light) for benchmarking.
• To analyze why low-light conditions degrade detector performance beyond simple data scarcity, identifying fundamental perceptual difficulties.
• To develop a joint learning framework that enhances images for machine cognition rather than human perception, improving robustness in low-light scenarios.
• To propose data augmentation techniques tailored for low-light object detection to improve generalization and performance.

Proposed method

• The NOD dataset is collected from real-world night scenes, annotated with instances of people, bicycles, and cars, and includes instance-level lighting condition labels for extreme and non-extreme low-light.
• A novel image enhancement module is trained end-to-end with the object detector, optimizing for detection performance rather than perceptual quality, using only bounding box annotations.
• Two new data augmentation techniques are introduced: block shuffle augmentation and patch-wise light augmentation, designed to simulate realistic low-light variations during training.
• The enhancement module is integrated into the object detection pipeline (e.g., RetinaNet, Faster R-CNN, FCOS, PAA), enabling joint optimization of image representation and detection head.
• The framework is trained and evaluated on the Sony and Nikon subsets of NOD and the ExDark benchmark, with performance measured using standard AP metrics.
• The method avoids reliance on paired low-light/normal-light image data, making it suitable for dynamic, real-world scenes.

Experimental results

Research questions

• RQ1 What specific lighting conditions—extreme vs. non-extreme low-light—pose distinct challenges for object detectors?
• RQ2 Why does training on low-light data alone fail to close the performance gap between extreme and non-extreme low-light conditions?
• RQ3 Can a learning-based image enhancement module trained jointly with an object detector improve detection accuracy under extreme low-light conditions?
• RQ4 How do novel data augmentation strategies like block shuffle and patch-wise light augmentation affect model generalization in low-light detection?
• RQ5 Is the proposed detection-with-enhancement framework effective across different object detection architectures?

Key findings

• The performance gap between extreme and non-extreme low-light conditions remains significant even after fine-tuning on the NOD dataset, indicating that extreme low-light is a distinct and challenging problem.
• The proposed detection-with-enhancement framework achieves a 1.3% absolute improvement in AP50 (from 72.8% to 74.1%) on the Sony subset when using the enhancement module with RetinaNet.
• The method consistently improves performance across multiple detectors, including Faster R-CNN (+2.6% AP50), FCOS (+1.9% AP50), and PAA (+1.4% AP50).
• The enhancement module outperforms state-of-the-art low-light enhancement methods like KinD++ in joint detection performance, despite being trained without paired data.
• The block shuffle and patch-wise light augmentation techniques contribute to improved robustness and generalization, especially under extreme lighting.
• The study confirms that low-light object detection is not merely a subset of standard detection and requires specialized modeling, particularly for extreme illumination conditions.