[Paper Review] An accelerated forward-backward algorithm for monotone inclusions.

Dirk A. Lorenz, Thomas Pock|arXiv (Cornell University)|Mar 14, 2014

Sparse and Compressive Sensing Techniques34 references9 citations

TL;DR

This paper proposes an inertial forward-backward splitting algorithm for solving monotone inclusion problems involving one co-coercive operator, inspired by Nesterov's accelerated gradient method. It achieves faster convergence than existing first-order methods while maintaining similar per-iteration computational cost, applicable to convex-concave saddle problems and general monotone inclusions in Hilbert spaces.

ABSTRACT

In this paper, we propose an inertial forward backward splitting algorithm to compute a zero of the sum of two monotone operators, with one of the two operators being co-coercive. The algorithm is inspired by the accelerated gradient method of Nesterov, but can be applied to a much larger class of problems including convex-concave saddle point problems and general monotone inclusions. We prove convergence of the algorithm in a Hilbert space setting and show that several recently proposed first-order methods can be obtained as special cases of the general algorithm. Numerical results show that the proposed algorithm converges faster than existing methods, while keeping the computational cost of each iteration basically unchanged.

Motivation & Objective

• To develop an efficient first-order algorithm for computing zeros of the sum of two monotone operators, one of which is co-coercive.
• To extend Nesterov's acceleration technique to monotone inclusion problems beyond smooth convex optimization.
• To unify and generalize several recently proposed first-order methods as special cases of a single framework.
• To ensure convergence in a Hilbert space setting while preserving low per-iteration complexity.

Proposed method

• The algorithm incorporates an inertial term inspired by Nesterov's momentum method to accelerate convergence.
• It applies the forward-backward splitting scheme with a relaxation parameter to handle the sum of monotone operators.
• The co-coercivity of one operator enables stronger convergence guarantees and faster iterates.
• The method is formulated in a Hilbert space, allowing broad applicability to convex-concave saddle problems and monotone variational inequalities.
• The algorithm's convergence is proven using Lyapunov analysis and properties of nonexpansive operators.
• The framework generalizes existing methods by allowing variable parameters and inertial over-relaxation.

Experimental results

Research questions

• RQ1 Can Nesterov-type acceleration be effectively extended to monotone inclusion problems with co-coercive operators?
• RQ2 How does the inertial forward-backward method compare in convergence speed to existing first-order methods for monotone inclusions?
• RQ3 What is the convergence behavior of the algorithm in a general Hilbert space setting?
• RQ4 Which existing first-order methods can be recovered as special cases of the proposed framework?

Key findings

• The proposed algorithm achieves faster convergence than existing first-order methods for monotone inclusions.
• The convergence rate is accelerated compared to standard forward-backward splitting, without increasing per-iteration computational cost.
• Several recently proposed first-order methods are shown to be special cases of the general algorithm.
• The method is applicable to convex-concave saddle point problems and general monotone inclusions in Hilbert spaces.