[Paper Review] An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration

Alexander Effland, Erich Kobler+2 more|arXiv (Cornell University)|Jul 19, 2019

Sparse and Compressive Sensing Techniques36 references4 citations

TL;DR

This paper proposes an optimal control framework that learns the stopping time of a gradient flow in variational image restoration, treating it as a control variable to improve image quality. By formulating the problem as an optimal control task with a data-driven cost functional, the method achieves superior results in denoising and deblurring, outperforming standard variational methods and revealing interpretable regularization properties via nonlinear spectral analysis.

ABSTRACT

We investigate a well-known phenomenon of variational approaches in image processing, where typically the best image quality is achieved when the gradient flow process is stopped before converging to a stationary point. This paradox originates from a tradeoff between optimization and modelling errors of the underlying variational model and holds true even if deep learning methods are used to learn highly expressive regularizers from data. In this paper, we take advantage of this paradox and introduce an optimal stopping time into the gradient flow process, which in turn is learned from data by means of an optimal control approach. As a result, we obtain highly efficient numerical schemes that achieve competitive results for image denoising and image deblurring. A nonlinear spectral analysis of the gradient of the learned regularizer gives enlightening insights about the different regularization properties.

Motivation & Objective

• To address the paradox that best image restoration quality is achieved before convergence in variational methods, despite the expectation that longer iterations improve results.
• To formalize early stopping as a learned control parameter within a gradient flow framework, rather than a heuristic choice.
• To develop a data-driven optimal control problem where the stopping time T is optimized alongside regularizer parameters to minimize reconstruction error.
• To enable interpretability of learned regularizers through nonlinear spectral analysis of the gradient of the Field of Experts-type regularizer.
• To demonstrate that the optimal stopping time is always finite, contradicting the classical variational paradigm that longer iterations yield better solutions.

Proposed method

• Formulate the image restoration problem as a gradient flow governed by a state equation, with the energy functional composed of a data fidelity term and a Field of Experts-type regularizer with learnable kernels and activation functions.
• Introduce the stopping time T as a control variable in an optimal control problem, where the cost functional measures the distance between the flow trajectory at time T and the ground truth image.
• Discretize the gradient flow using explicit Euler and Heun schemes, resulting in a static variational network architecture that is a subclass of deep learning models with fixed regularizers over time.
• Derive first-order optimality conditions via a Lagrangian approach to enable automated computation of the optimal stopping time T.
• Perform nonlinear spectral analysis on the gradient of the learned regularizer by computing generalized eigenpairs to study local regularization behavior.
• Use accelerated gradient descent with step size control to compute Nv = 64 generalized eigenpairs of size 127×127, initialized with mean-subtracted image patches and smoothed at boundaries.

Experimental results

Research questions

• RQ1 Can the optimal stopping time for a variational image restoration gradient flow be learned from data using optimal control, rather than being set heuristically?
• RQ2 Does learning the stopping time lead to improved image quality compared to standard variational methods that run to convergence?
• RQ3 What are the intrinsic regularization properties of the learned regularizer, and how do they vary across different image structures?
• RQ4 Why is the optimal stopping time always finite, even when the optimization allows for larger T, and how does this contradict classical variational principles?
• RQ5 How does the nonlinear spectral analysis of the learned regularizer's gradient reveal insights into its local behavior on different image features?

Key findings

• The optimal stopping time T is always finite in the learned model, indicating that early stopping is not just a heuristic but a fundamental property of the optimal solution.
• The proposed method achieves competitive image restoration results in denoising and deblurring, with improved PSNR values compared to standard variational approaches that run to convergence.
• Nonlinear spectral analysis reveals that eigenfunctions with small eigenvalues represent cartoon-like structures with clear interfaces, while those with large eigenvalues correspond to textures and noise.
• For denoising, the contrast factor (1 - λjT/S) ranges from 0.368 to 0.999, indicating that the regularizer reduces contrast of high-frequency noise while preserving structural features.
• For deblurring, eigenvalues are small and centered around zero, with contrast factors in [0.992, 1.030], showing the regularizer can both reduce and increase contrast, with a sharpening tendency near image boundaries.
• The learned regularizer exhibits a strong tendency to preserve mid-to-low frequency structures while suppressing high-frequency noise, as confirmed by the spectral analysis of eigenfunctions and contrast factors.