[Paper Review] Exposure: A White-Box Photo Post-Processing Framework

Yuanming Hu, Hao He+3 more|arXiv (Cornell University)|Sep 27, 2017

Advanced Image and Video Retrieval Techniques48 references53 citations

TL;DR

This paper presents Exposure, a white-box, end-to-end photo post-processing system trained on unpaired data, delivering both retouched images and a human-understandable sequence of edits.

ABSTRACT

Retouching can significantly elevate the visual appeal of photos, but many casual photographers lack the expertise to do this well. To address this problem, previous works have proposed automatic retouching systems based on supervised learning from paired training images acquired before and after manual editing. As it is difficult for users to acquire paired images that reflect their retouching preferences, we present in this paper a deep learning approach that is instead trained on unpaired data, namely a set of photographs that exhibits a retouching style the user likes, which is much easier to collect. Our system is formulated using deep convolutional neural networks that learn to apply different retouching operations on an input image. Network training with respect to various types of edits is enabled by modeling these retouching operations in a unified manner as resolution-independent differentiable filters. To apply the filters in a proper sequence and with suitable parameters, we employ a deep reinforcement learning approach that learns to make decisions on what action to take next, given the current state of the image. In contrast to many deep learning systems, ours provides users with an understandable solution in the form of conventional retouching edits, rather than just a "black-box" result. Through quantitative comparisons and user studies, we show that this technique generates retouching results consistent with the provided photo set.

Motivation & Objective

• Address the difficulty of collecting paired retouching data by learning from unpaired photo collections that reflect user style.
• Provide an end-to-end post-processing model that applies multiple edits as a sequence of differentiable filters.
• Offer transparency by outputting a meaningful operation sequence rather than a black-box result.
• Ensure edits are compatible with high-resolution images through resolution-independent filters.
• Enable learning of personal retouching preferences via a GAN-based style model and reinforcement learning.

Proposed method

• Formulate retouching as a sequential decision-making process with states as intermediate images and actions as differentiable filters.
• Model edits as resolution-independent differentiable filters that can be applied to high-resolution RAW data.
• Use a dual-policy network: one for discrete filter selection and one for continuous parameter tuning of the chosen filter.
• Train with policy gradients (actor-critic) guided by a discriminator-based image quality reward from a GAN.
• Incorporate a Wasserstein GAN to align generated edits with a target photo collection without requiring paired data.
• Represent color curves with a differentiable, piecewise-linear formulation to enable gradient-based learning.

Experimental results

Research questions

• RQ1 Can unpaired photo collections be used to learn a user’s retouching style without image pairs?
• RQ2 Can a sequence of differentiable, human-understandable filters emulate professional retouching styles?
• RQ3 Does combining reinforcement learning with a GAN discriminator provide high-quality, style-consistent edits on RAW inputs?
• RQ4 Is the resulting editing sequence interpretable and reversible for users while scaling to high-resolution images.

Key findings

• The system learns to retouch RAW images in a way that matches a target photo collection without paired data.
• The edits are produced as a sequence of conventional, understandable operations rather than a black-box result.
• The framework outputs resolution-independent filter sequences that can be applied to high-resolution images (e.g., RAW to display-ready output).
• User studies and quantitative comparisons show retouching results align with preferred styles in the provided photo collection.
• An end-to-end model combining differentiable filters, RL, and GAN achieves plausible style transfer without image pairs.