[Paper Review] A GCV based Arnoldi-Tikhonov regularization method

Paolo Novati, Maria Rosaria Russo|arXiv (Cornell University)|Mar 31, 2013

Numerical methods in inverse problems25 references4 citations

TL;DR

This paper proposes a GCV-based Arnoldi-Tikhonov regularization method for solving large-scale linear discrete ill-posed problems, where the Arnoldi process projects the problem onto Krylov subspaces to efficiently approximate the Tikhonov solution with a dynamically updated regularization parameter via Generalized Cross Validation. The method achieves robust convergence and accurate image restoration with minimal iterations, even for high-condition-number systems.

ABSTRACT

For the solution of linear discrete ill-posed problems, in this paper we consider the Arnoldi-Tikhonov method coupled with the Generalized Cross Validation for the computation of the regularization parameter at each iteration. We study the convergence behavior of the Arnoldi method and its properties for the approximation of the (generalized) singular values, under the hypothesis that Picard condition is satisfied. Numerical experiments on classical test problems and on image restoration are presented.

Motivation & Objective

• To address the challenge of selecting an optimal regularization parameter in Tikhonov regularization for large-scale discrete ill-posed problems.
• To develop a computationally efficient method that avoids full SVD/GSVD decomposition by leveraging Krylov subspace projection.
• To integrate Generalized Cross Validation (GCV) as a parameter choice rule within the Arnoldi-Tikhonov framework for noise-free right-hand side scenarios.
• To validate the method’s convergence and accuracy on classical test problems and real-world image restoration tasks.

Proposed method

• The Arnoldi-Tikhonov (AT) method projects the original problem onto Krylov subspaces generated by A and b, using an orthonormal basis W_m to reduce the problem size.
• At each iteration m, the method solves a small-scale Tikhonov problem: min_ym ||H̄_m y_m - ||b|| e_1||² + λ² ||L W_m y_m||², where H̄_m is the projected matrix.
• The regularization parameter λ is selected at each step m via the GCV criterion applied to the projected problem, approximating the full GCV curve.
• The method uses the GCV function G_m(λ) to compute a sequence of regularization parameters {λ_m} that approximate the optimal λ for the full problem.
• Theoretical convergence is supported under the Picard condition, ensuring that the Krylov subspace captures dominant singular components of A.
• The approach avoids explicit GSVD computation of (A, L), reducing computational cost while maintaining accuracy in large-scale problems.

Experimental results

Research questions

• RQ1 Can the Arnoldi-Tikhonov method with GCV parameter selection achieve convergence and stability for large-scale discrete ill-posed problems?
• RQ2 How well does the projected GCV curve approximate the true GCV curve of the full problem?
• RQ3 Does the sequence of regularization parameters {λ_m} computed iteratively yield solutions close to the optimal Tikhonov solution?
• RQ4 Can the method effectively restore blurred and noisy images without prior knowledge of noise level?

Key findings

• The Arnoldi-Tikhonov method with GCV parameter selection converges robustly under the Picard condition, ensuring effective approximation of the true solution.
• The projected GCV curve, computed in small dimensions, closely approximates the full GCV curve, enabling accurate parameter selection with minimal computational cost.
• For the mri.png image restoration problem, the method achieved high-quality results in just 5 iterations, within 0.5 seconds on a single processor.
• The method outperformed standard iterative solvers in terms of speed and accuracy, particularly in handling high-condition-number matrices (e.g., cond(A) ≈ 10^13).
• The sequence of regularization parameters {λ_m} closely tracked the optimal λ_opt, as confirmed by comparison with the true solution in numerical experiments.
• The method demonstrated consistent performance across multiple test problems (BAART, SHAW, FOXGOOD, I_LAPLACE), with low relative error and residual history indicating stable convergence.