[Paper Review] Unsupervised Single-Image Reflection Separation Using Perceptual Deep Image Priors

Su Hong Kim, Hamed RahmaniKhezri+2 more|arXiv (Cornell University)|Sep 1, 2020

Image Enhancement Techniques37 references4 citations

TL;DR

This paper proposes the first unsupervised single-image reflection separation method using Perceptual Deep Image Priors (Perceptual DIP), where two cross-coupled deep networks optimize jointly to separate background and reflection layers without any training data. By embedding semantic features from a pre-trained classifier and using a multi-loss optimization with cross-feedback and a balancing parameter α, the method achieves state-of-the-art performance on real and synthetic images, outperforming both supervised and unsupervised baselines without requiring labeled data or assumptions about reflection properties.

ABSTRACT

Reflections often degrade the quality of the image by obstructing the background scene. This is not desirable for everyday users, and it negatively impacts the performance of multimedia applications that process images with reflections. Most current methods for removing reflections utilize supervised-learning models. However, these models require an extensive number of image pairs to perform well, especially on natural images with reflection, which is difficult to achieve in practice. In this paper, we propose a novel unsupervised framework for single-image reflection separation. Instead of learning from a large dataset, we optimize the parameters of two cross-coupled deep convolutional networks on a target image to generate two exclusive background and reflection layers. In particular, we design a new architecture of the network to embed semantic features extracted from a pre-trained deep classification network, which gives more meaningful separation similar to human perception. Quantitative and qualitative results on commonly used datasets in the literature show that our method's performance is at least on par with the state-of-the-art supervised methods and, occasionally, better without requiring large training datasets. Our results also show that our method significantly outperforms the closest unsupervised method in the literature for removing reflections from single images.

Motivation & Objective

• To address the challenge of single-image reflection separation without relying on large labeled datasets or additional assumptions.
• To develop a method that leverages deep image priors while incorporating high-level semantic information for more perceptually meaningful separation.
• To eliminate the need for paired training data by using an unsupervised optimization framework based on perceptual and structural priors.
• To improve robustness and consistency of reflection separation across repeated runs and on weak reflection cases.

Proposed method

• Proposes a novel architecture called Perceptual DIP that integrates feature maps from a pre-trained image classification network into the generator network to embed high-level semantic features.
• Employs two cross-coupled Perceptual DIP networks that jointly optimize the background and reflection layers through a feedback mechanism to enhance separation quality.
• Introduces a balancing parameter α to assign roles (background or reflection) to each network, stabilizing training and reducing degeneration, especially for weak reflections.
• Uses a multi-loss optimization scheme including reconstruction loss, exclusion loss, cross-feedback loss, and regularization loss to guide the decomposition process.
• Applies perceptual loss using features from a pre-trained network to ensure structural and semantic consistency in the generated layers.
• Optimizes network parameters directly on the input image without any pre-training or fine-tuning, relying solely on inductive biases of the network architecture and loss functions.

Experimental results

Research questions

• RQ1 Can reflection separation be effectively achieved in an unsupervised manner without any labeled training data?
• RQ2 Can high-level semantic priors improve the perceptual quality of reflection separation compared to low-level image priors alone?
• RQ3 How can cross-coupled optimization between background and reflection networks improve separation robustness and reduce ambiguity?
• RQ4 Can a balancing parameter α stabilize the optimization and prevent degenerate solutions in weak reflection scenarios?
• RQ5 How does the proposed method compare quantitatively and qualitatively to both supervised and unsupervised state-of-the-art methods on real-world and synthetic images?

Key findings

• The proposed method achieves performance on par with or better than state-of-the-art supervised methods on both real and synthetic datasets, despite having no training data.
• The method significantly outperforms the closest unsupervised baseline (Gandelsman et al., 2019), which requires an additional image as input, while the proposed method works with only a single image.
• The inclusion of the balancing parameter α (set to 0.1) stabilizes training and prevents inconsistent outputs across multiple runs, especially for weak reflections.
• The cross-feedback mechanism improves convergence speed and robustness, particularly when gradient information in the reflection layer is sparse.
• The full loss combination (reconstruction, exclusion, cross-feedback, and regularization) yields the best results, with clear separation in color and shape, reducing artifacts compared to partial loss combinations.
• Ablation studies confirm that reconstruction loss is essential, and adding exclusion and cross-feedback losses significantly improves content separation and structural fidelity.