[Paper Review] RestoreDet: Degradation Equivariant Representation for Object Detection in Low Resolution Images

Ziteng Cui, Yingying Zhu+6 more|arXiv (Cornell University)|Jan 7, 2022

Advanced Image Processing Techniques4 citations

TL;DR

RestoreDet is a novel end-to-end framework for object detection in low-resolution, degraded images by learning degradation-equivariant representations through self-supervised contrastive learning across randomly degraded image pairs. It integrates an arbitrary-resolution restoration decoder to preserve high-fidelity details, achieving state-of-the-art detection accuracy and inference speed across diverse degradation conditions without explicit image restoration during inference.

ABSTRACT

Image restoration algorithms such as super resolution (SR) are indispensable pre-processing modules for object detection in degraded images. However, most of these algorithms assume the degradation is fixed and known a priori. When the real degradation is unknown or differs from assumption, both the pre-processing module and the consequent high-level task such as object detection would fail. Here, we propose a novel framework, RestoreDet, to detect objects in degraded low resolution images. RestoreDet utilizes the downsampling degradation as a kind of transformation for self-supervised signals to explore the equivariant representation against various resolutions and other degradation conditions. Specifically, we learn this intrinsic visual structure by encoding and decoding the degradation transformation from a pair of original and randomly degraded images. The framework could further take the advantage of advanced SR architectures with an arbitrary resolution restoring decoder to reconstruct the original correspondence from the degraded input image. Both the representation learning and object detection are optimized jointly in an end-to-end training fashion. RestoreDet is a generic framework that could be implemented on any mainstream object detection architectures. The extensive experiment shows that our framework based on CenterNet has achieved superior performance compared with existing methods when facing variant degradation situations. Our code would be released soon.

Motivation & Objective

• To address the performance drop of object detectors on low-resolution or degraded images due to mismatched degradation assumptions in pre-processing restoration modules.
• To develop a generic, end-to-end framework that jointly optimizes representation learning and object detection without relying on fixed degradation models.
• To leverage self-supervised signals from random down-sampling transformations to learn robust, degradation-invariant feature representations.
• To integrate advanced super-resolution architectures via an arbitrary-resolution restoration decoder (ARRD) that supervises feature learning for improved detection.
• To achieve high detection accuracy and inference efficiency without explicitly reconstructing high-resolution images during inference.

Proposed method

• The framework uses a random degradation transformation $ t $, defined by down-sampling ratio $ s $, blur kernel $ k $, and noise level $ n $, to generate degraded LR images from HR inputs.
• A transformation decoder $ D_t $ is trained to reconstruct the degradation transformation $ t $ from the encoded features of the original and degraded image pair, enforcing equivariance in the representation space.
• An arbitrary-resolution restoration decoder (ARRD) $ D_r $ is introduced to reconstruct the original HR image from the degraded representation, guiding the encoder to preserve fine-grained structural details.
• The encoder $ E $, transformation decoder $ D_t $, restoration decoder $ D_r $, and object detection head $ D_o $ are jointly optimized in an end-to-end manner.
• During inference, only the encoder $ E $ and detection head $ D_o $ are used, avoiding explicit image restoration and enabling faster inference.
• The training data includes randomly degraded images sampled from uniform distributions over $ s \in [1,4] $, $ k \in \{k_{\text{iso}}, k_{\text{aniso}}, k_{\text{none}}\} $, and noise variance $ \sim U(0, 25/255) $.

Experimental results

Research questions

• RQ1 Can self-supervised learning via degradation transformations improve object detection robustness in low-resolution images with unknown or variable degradation?
• RQ2 How does learning degradation-equivariant representations compare to pre-processing with fixed-degradation super-resolution models in terms of detection accuracy and inference efficiency?
• RQ3 To what extent can an arbitrary-resolution restoration decoder improve feature representation quality for downstream detection without explicit image reconstruction?
• RQ4 Does joint optimization of representation learning and detection yield better generalization than sequential pre-processing and detection?
• RQ5 Can the framework generalize to real-world degradation patterns without prior knowledge of the degradation model?

Key findings

• RestoreDet with the transformation decoder (w/ $ D_t $) achieves 80.5% AP on the KITTI dataset, outperforming baseline CenterNet (42.2% AP) and models using only ARRD or $ D_t $.
• The full RestoreDet (w/ $ D_t $ and $ D_r $) achieves the highest detection accuracy and fastest inference speed (FPS) among all compared methods, including those using pre-processing SR modules.
• On COCO with fixed down-sampling (2× and 4×), RestoreDet maintains SOTA performance without retraining, demonstrating strong generalization to unseen degradation levels.
• The ARRD-generated outputs show machine-vision-friendly details with fewer artifacts than standard SR methods, particularly around object centers.
• Abllation studies confirm that both $ D_t $ and $ D_r $ contribute significantly to performance, with $ D_t $ improving robustness and $ D_r $ enhancing feature fidelity.
• The method achieves superior performance on small objects while maintaining strong performance on large objects, unlike pre-processing methods that degrade large-object detection.