[Paper Review] Bias-Reduction in Variational Regularization

Eva-Maria Brinkmann, Martin Burger+2 more|arXiv (Cornell University)|Jun 16, 2016

Sparse and Compressive Sensing Techniques26 references3 citations

TL;DR

This paper proposes a two-step debiasing method for variational regularization that reduces systematic bias by minimizing data fidelity on a model manifold defined via Bregman distances using the subgradient from the first regularization step. The method is theoretically well-posed and achieves optimal bias reduction, outperforming standard methods and matching Bregman iteration performance in numerical experiments.

ABSTRACT

The aim of this paper is to introduce and study a two-step debiasing method for variational regularization. After solving the standard variational problem, the key idea is to add a consecutive debiasing step minimizing the data fidelity on an appropriate set, the so-called model manifold. The latter is defined by Bregman distances or infimal convolutions thereof, using the (uniquely defined) subgradient appearing in the optimality condition of the variational method. For particular settings, such as anisotropic $\ell^1$ and TV-type regularization, previously used debiasing techniques are shown to be special cases. The proposed approach is however easily applicable to a wider range of regularizations. The two-step debiasing is shown to be well-defined and to optimally reduce bias in a certain setting. In addition to visual and PSNR-based evaluations, different notions of bias and variance decompositions are investigated in numerical studies. The improvements offered by the proposed scheme are demonstrated and its performance is shown to be comparable to optimal results obtained with Bregman iterations.

Motivation & Objective

• To address the well-known bias in variational regularization methods, such as contrast loss in total variation or peak shrinkage in ℓ¹-regularization.
• To unify existing debiasing techniques—like refitting on support and Bregman iterations—under a single, general framework.
• To formalize a two-step approach that leverages the subgradient from the first variational solution to define a model manifold for subsequent debiasing.
• To establish theoretical well-posedness and optimal bias reduction under specific conditions.
• To demonstrate the method’s effectiveness through numerical studies using PSNR, bias decomposition, and variance analysis.

Proposed method

• The method begins with standard variational regularization: minimize H(Au, f) + αJ(u), yielding solution uα and subgradient pα ∈ ∂J(uα).
• A second debiasing step minimizes data fidelity H(Au, f) subject to the constraint that pα ∈ ∂J(u), i.e., u lies in the model manifold defined by the Bregman distance D_J^{pα}(u, uα) = 0.
• The model manifold is defined via the generalized Bregman distance D_J^p(u, v) = J(u) − J(v) − ⟨p, u − v⟩, with p ∈ ∂J(v), ensuring consistency with the subgradient of the first step.
• For one-homogeneous regularizers like ℓ¹ or TV, the method recovers known debiasing schemes (e.g., refitting on support with sign constraints) as special cases.
• The approach is reformulated using infimal convolution of Bregman distances to handle non-smooth and non-strictly convex functionals.
• Theoretical analysis proves the two-step method is well-defined and achieves optimal bias reduction in a specified setting, with equivalence to inverse scale space and Bregman iteration in certain cases.

Experimental results

Research questions

• RQ1 Can a general two-step debiasing framework be developed that unifies existing approaches like refitting, Bregman iterations, and inverse scale space methods?
• RQ2 How can the model manifold defined by Bregman distances be used to optimally reduce bias in variational regularization?
• RQ3 To what extent does the proposed method achieve optimal bias reduction compared to standard regularization and Bregman iterations?
• RQ4 What is the relationship between the subgradient from the first step and the structure of the solution manifold in the second step?
• RQ5 How do different bias and variance decompositions behave under the proposed debiasing scheme in numerical experiments?

Key findings

• The two-step debiasing method is well-defined and theoretically justified for convex, one-homogeneous regularization functionals.
• The method achieves optimal bias reduction in a specific setting, with theoretical guarantees matching those of Bregman iterations.
• For ℓ¹ and TV-type regularization, the method recovers known refitting and sign-constrained debiasing techniques as special cases.
• Numerical results show significant improvements in PSNR and visual quality, with bias decomposition confirming reduced method bias.
• The method performs comparably to Bregman iterations in terms of reconstruction accuracy and bias reduction, without requiring iterative refinement.
• Theoretical equivalence is established between the proposed method and the inverse scale space method under certain conditions, particularly when subgradients are preserved.