[Paper Review] CPA-Enhancer: Chain-of-Thought Prompted Adaptive Enhancer for Object Detection under Unknown Degradations

Yu-Wei Zhang, Yan Wu+2 more|arXiv (Cornell University)|Mar 17, 2024

CCD and CMOS Imaging Sensors4 citations

TL;DR

CPA-Enhancer is a plug-and-play, chain-of-thought (CoT) prompted adaptive enhancer that improves object detection under unknown, multiple degradations without prior degradation knowledge. It uses a CoT-prompt generation module (CGM) and content-driven prompt block (CPB) to dynamically adapt enhancement strategies, achieving state-of-the-art mAP gains across diverse benchmarks like VM, RTTS, and ExDarkA, while boosting performance on semantic segmentation tasks.

ABSTRACT

Object detection methods under known single degradations have been extensively investigated. However, existing approaches require prior knowledge of the degradation type and train a separate model for each, limiting their practical applications in unpredictable environments. To address this challenge, we propose a chain-of-thought (CoT) prompted adaptive enhancer, CPA-Enhancer, for object detection under unknown degradations. Specifically, CPA-Enhancer progressively adapts its enhancement strategy under the step-by-step guidance of CoT prompts, that encode degradation-related information. To the best of our knowledge, it's the first work that exploits CoT prompting for object detection tasks. Overall, CPA-Enhancer is a plug-and-play enhancement model that can be integrated into any generic detectors to achieve substantial gains on degraded images, without knowing the degradation type priorly. Experimental results demonstrate that CPA-Enhancer not only sets the new state of the art for object detection but also boosts the performance of other downstream vision tasks under unknown degradations.

Motivation & Objective

• To address the limitation of existing object detectors that require prior knowledge of degradation types and are ineffective under unknown or multiple degradations.
• To develop a unified, plug-and-play enhancement module that can be integrated with any generic detector without retraining for each degradation type.
• To explore the feasibility and effectiveness of chain-of-thought (CoT) prompting in vision tasks, particularly for adaptive image enhancement in object detection.
• To improve detection robustness and generalization across diverse, unseen degradation types such as noise, blur, fog, and low-light conditions.

Proposed method

• The model employs a CoT-prompt generation module (CGM) that generates step-by-step, degradation-informed prompts based on image features, enabling progressive reasoning about the degradation type.
• A content-driven prompt block (CPB) dynamically fuses input features with CoT prompts through learnable attention mechanisms, allowing adaptive enhancement per feature map.
• The CPB splits the feature map into n parts, each processed with a dedicated prompt, enabling fine-grained, localized enhancement strategy adaptation.
• The entire CPA-Enhancer is trained end-to-end with the detector (e.g., YOLOv3), enabling joint optimization of detection and enhancement without paired degradation data.
• The method leverages few-shot or zero-shot generalization by learning to infer degradation patterns from visual cues, not labeled degradation types.
• The framework is modular and compatible with various detectors and downstream tasks, including semantic segmentation.

Experimental results

Research questions

• RQ1 Can chain-of-thought prompting be effectively adapted to vision tasks like object detection for degradation-aware enhancement?
• RQ2 How can a unified enhancement model dynamically adapt to unknown or multiple degradation types without prior knowledge?
• RQ3 What is the impact of CoT prompting versus independent prompting on detection performance under diverse degradations?
• RQ4 How does the CPB module compare to simpler prompt fusion mechanisms in enhancing feature quality for detection?
• RQ5 To what extent can CPA-Enhancer generalize to unseen degradation types such as varying noise levels or rare real-world distortions?

Key findings

• CPA-Enhancer achieves 82.62% mAP50 on the VM dataset, outperforming the baseline YOLOv3 by 2.35% and setting a new SOTA under unknown degradations.
• On the RTTS dataset, CPA-Enhancer achieves 56.35% mAP50, a 0.89% improvement over the baseline, demonstrating robustness to real-world degradation.
• On ExDarkA, CPA-Enhancer achieves 61.08% mAP50, a 1.56% gain over the baseline, confirming strong generalization to low-light and complex scenes.
• The model generalizes well to unseen noise levels, achieving 82.34% mAP50 at σ=15, 81.95% at σ=25, and 80.62% at σ=50, outperforming YOLOv3 across all noise intensities.
• In semantic segmentation, CPA-Enhancer boosts DeepLabV3+ to 69.3% mIoU and SegFormer to 75.1% mIoU on the ACDC validation set, surpassing Refign (based on DAFormer) by significant margins.
• The model introduces only 3M trainable parameters and adds just 3ms inference time per image on an RTX 4090, demonstrating strong efficiency and practical deployability.