[Paper Review] CLEAR: Covariant LEAst-square Re-fitting with applications to image restoration

Charles‐Alban Deledalle, Nicolas Papadakis+2 more|arXiv (Cornell University)|Jun 16, 2016

Image and Signal Denoising Methods42 references26 citations

TL;DR

This paper introduces CLEAR (Covariant LEAst-square Re-fitting), a novel framework to reduce systematic bias in image restoration methods by re-fitting restored signals via a locally affine transformation of the residual. It preserves the Jacobian of the original estimator, ensuring stability and robustness without requiring prior knowledge of support or jump locations, and achieves improved performance across total variation, non-local means, and other variational models with only a modest computational overhead of 2x–3x the original algorithm's cost.

ABSTRACT

In this paper, we propose a new framework to remove parts of the systematic errors affecting popular restoration algorithms, with a special focus for image processing tasks. Generalizing ideas that emerged for $\\ell_1$ regularization, we develop an approach re-fitting the results of standard methods towards the input data. Total variation regularizations and non-local means are special cases of interest. We identify important covariant information that should be preserved by the re-fitting method, and emphasize the importance of preserving the Jacobian (w.r.t. the observed signal) of the original estimator. Then, we provide an approach that has a "twicing" flavor and allows re-fitting the restored signal by adding back a local affine transformation of the residual term. We illustrate the benefits of our method on numerical simulations for image restoration tasks.

Motivation & Objective

• Address the systematic bias introduced by convex relaxations in image restoration, such as ℓ1 regularization and total variation.
• Develop a re-fitting method that enhances restored signals toward the original data while preserving the structural properties enforced by the prior.
• Overcome limitations of classical post-re-fitting techniques that require accurate identification of non-zero coefficients or jump locations.
• Ensure the re-fitting process is covariant—preserving the Jacobian of the original estimator with respect to the observed signal.
• Generalize the re-fitting approach to a wide class of variational models, including Tikhonov, isotropic TV, non-local means, BM3D, and DDID.

Proposed method

• Propose a re-fitting scheme that adds a local affine transformation of the residual to the original restored signal, ensuring consistency with the original estimator’s sensitivity.
• Formulate the re-fitting as a constrained least-squares problem that maintains the structure of the original regularization (e.g., sparsity or total variation).
• Use a primal-dual algorithm with projection onto the active set of the subdifferential to implicitly enforce the re-fitting constraints.
• Leverage the invariance of the re-fitting under small perturbations by ensuring the Jacobian of the estimator is preserved.
• Apply the method iteratively during the solution process of the original algorithm, enabling joint estimation and improving robustness.
• Demonstrate that the re-fitting converges to the correct solution by proving that the projection in the primal-dual algorithm eventually matches the active set of the original solution.

Experimental results

Research questions

• RQ1 Can a re-fitting method be designed that reduces bias in image restoration without requiring explicit identification of non-zero coefficients or jump locations?
• RQ2 How can the re-fitting process be made covariant—preserving the Jacobian of the original estimator with respect to the input signal?
• RQ3 Does the proposed CLEAR framework outperform classical post-re-fitting techniques in terms of stability and accuracy, especially in ill-posed or noisy settings?
• RQ4 Can the re-fitting approach be generalized to a wide range of variational models, including non-convex or non-smooth regularizations?
• RQ5 What is the computational overhead of applying CLEAR compared to the original restoration algorithm?

Key findings

• CLEAR reduces bias in restored images by re-fitting the solution toward the observed data while preserving the structural properties enforced by the prior.
• The method achieves a computational overhead of approximately 2x for ℓ1−ℓ2 analysis models and 3x for more complex models like isotropic TV or non-local means.
• Unlike classical post-re-fitting, CLEAR does not require identifying the support of the solution or jump locations, making it more robust to numerical inaccuracies in iterative solvers.
• For isotropic total variation, CLEAR produces superior results compared to invariant re-fitting, which can amplify oscillations near boundaries.
• The primal-dual algorithm used in CLEAR converges to the correct re-fitted solution, as proven by showing that the projection operator eventually matches the active set of the original solution.
• Numerical simulations demonstrate that CLEAR improves image restoration quality across various tasks, including denoising and deblurring, with consistent gains in PSNR and visual fidelity.