[Paper Review] OneRestore: A Universal Restoration Framework for Composite Degradation

Yu Guo, Yuan Gao+4 more|arXiv (Cornell University)|Jul 5, 2024

Conservation Techniques and StudiesArts and Humanities3 citations

TL;DR

OneRestore proposes a universal, transformer-based image restoration framework that handles composite degradations—such as low light, haze, rain, and snow—through adaptive, controllable restoration using scene descriptors. By integrating a cross-attention mechanism and a composite degradation restoration loss with negative samples, it achieves state-of-the-art performance on both synthetic and real-world datasets, significantly outperforming existing methods in complex, multi-factor degradation scenarios.

ABSTRACT

In real-world scenarios, image impairments often manifest as composite degradations, presenting a complex interplay of elements such as low light, haze, rain, and snow. Despite this reality, existing restoration methods typically target isolated degradation types, thereby falling short in environments where multiple degrading factors coexist. To bridge this gap, our study proposes a versatile imaging model that consolidates four physical corruption paradigms to accurately represent complex, composite degradation scenarios. In this context, we propose OneRestore, a novel transformer-based framework designed for adaptive, controllable scene restoration. The proposed framework leverages a unique cross-attention mechanism, merging degraded scene descriptors with image features, allowing for nuanced restoration. Our model allows versatile input scene descriptors, ranging from manual text embeddings to automatic extractions based on visual attributes. Our methodology is further enhanced through a composite degradation restoration loss, using extra degraded images as negative samples to fortify model constraints. Comparative results on synthetic and real-world datasets demonstrate OneRestore as a superior solution, significantly advancing the state-of-the-art in addressing complex, composite degradations.

Motivation & Objective

• To address the gap in existing image restoration methods that focus on isolated degradations rather than composite, real-world scenarios involving multiple impairments.
• To develop a unified framework capable of adaptive, controllable restoration across diverse degradation types without requiring model retraining or manual reconfiguration.
• To enhance model robustness and user control by incorporating scene descriptors (text or visual-based) and a novel composite degradation restoration loss with negative samples.
• To simulate realistic composite degradation scenarios through a new dataset, CDD-11, enabling effective training and evaluation of universal restoration models.

Proposed method

• The framework employs a transformer-based architecture with a novel cross-attention block that fuses degraded scene descriptors with image features to guide restoration.
• Scene descriptors are generated either via manual text embeddings or automatic visual attribute extraction, enabling flexible and controllable input.
• A composite degradation restoration loss is introduced, using additional degraded images as negative samples to enforce a strict lower-bound constraint on the model's output.
• The model is trained on the CDD-11 dataset, a newly constructed benchmark simulating realistic composite degradations by combining physical corruption models for low light, haze, rain, and snow.
• The architecture supports a One-to-Composite restoration strategy, allowing dynamic adaptation to mixed degradation types without requiring specialized sub-models.
• The framework enables precise control over restoration by conditioning the attention mechanism on degradation-specific descriptors, improving alignment with user intent.

Figure 1 : Our OneRestore allows fully controllable image restoration using scene descriptors derived from both automatic visual attribute extraction (top) and manual text embeddings (bottom).

Experimental results

Research questions

• RQ1 How can a single image restoration model effectively handle composite degradations involving multiple simultaneous impairments such as low light, haze, rain, and snow?
• RQ2 Can scene descriptors—either manually provided or automatically extracted—enable controllable and adaptive restoration across diverse degradation types?
• RQ3 How does incorporating a composite degradation restoration loss with negative samples improve model generalization and output fidelity?
• RQ4 To what extent does training on a synthetic composite degradation dataset (CDD-11) generalize to real-world, complex degradation scenarios?

Key findings

• OneRestore achieves state-of-the-art performance on four real-world benchmarks: NPE (low-light), RESIDE RTTS (dehazing), Yang’s dataset (deraining), and Snow100k-R (desnowing), as measured by NIQE and PIQE metrics.
• The model significantly outperforms existing SOTA methods on both synthetic and real-world datasets, demonstrating strong generalization to complex composite degradation scenarios.
• The use of scene descriptors enables precise control over restoration, allowing the model to prioritize specific degradation types based on input descriptions, as validated in visual comparisons.
• The composite degradation restoration loss with negative samples effectively constrains the model, reducing distribution shift and improving output quality by establishing a tighter lower-bound on restoration fidelity.
• Visual results on real-world datasets confirm that OneRestore produces high-quality, detail-rich outputs that closely approximate clean reference images, even under severe composite degradation.
• Failure cases reveal limitations in handling high-density corruption and unmodeled degradation types like raindrops or moiré patterns, indicating room for future improvement.

Figure 2 : Structural comparison between One-to-One, One-to-Many, and our One-to-Composite image restoration methods.