[论文解读] ALiA: Adaptive Linearized ADMM

Uijeong Jang, Kaizhao Sun+2 more|arXiv (Cornell University)|Feb 16, 2026

Sparse and Compressive Sensing Techniques被引用 0

一句话总结

tldr: ALiA is an adaptive variant of FLiP ADMM that selects stepsizes without backtracking linesearch, with a global convergence guarantee for convex differentiable objectives and improved practical speed.

ABSTRACT

We propose ALiA, a novel adaptive variant of the alternating direction method of multipliers (ADMM). Specifically, ALiA is a variant of function-linearized proximal ADMM (FLiP ADMM), which generalizes the classical ADMM by leveraging the differentiable structure of the objective function, making it highly versatile. Notably, ALiA features an adaptive stepsize selection scheme that eliminates the need for backtracking linesearch. Motivated by recent advances in adaptive gradient and proximal methods, we establish point convergence of ALiA for convex and differentiable objectives. Furthermore, by introducing negligible computational overhead, we develop an alternative stepsize selection scheme for ALiA that improves the convergence speed both theoretically and empirically. Extensive numerical experiments on practical datasets confirm the accelerated performance of ALiA compared to standard FLiP ADMM. Additionally, we demonstrate that ALiA either outperforms or matches the practical performance of existing adaptive methods across problem classes where it is applicable.

研究动机与目标

• Motivate and solve a broad convex optimization problem with both primal and dual variables under linear constraints.
• Introduce adaptivity into primal-dual ADMM without relying on backtracking linesearch.
• Develop adaptive stepsize rules that depend on iterate-dependent quantities to ensure convergence.
• Demonstrate improved practical performance over standard FLiP ADMM and competitive adaptive methods.

提出的方法

• Propose ALiA, an adaptive variant of FLiP ADMM for the general-form problem with differentiable and non-differentiable components.
• Use proximal updates for x and y with gradients of smooth parts and linear operators A and B.
• Update dual variable u first, then compute primal updates, leveraging adaptive subroutines to set gamma_k+1 and the dual direction Δu^{k+1}.
• Subroutine 1 and Subroutine 2 compute adaptive stepsizes and dual updates using iterate-dependent curvature and operator-norm estimates without backtracking.
• Provide proximal-operator based reformulations (u^{k+1}, x^{k+1}, y^{k+1}) and show convergence under convexity and local smoothness assumptions.
• Establish convergence (Theorem 2.1) to a saddle point of the Lagrangian under the proposed adaptive scheme.

实验结果

研究问题

• RQ1 Does ALiA converge globally to a saddle point for the general FLiP-ADMM problem class under convexity and local smoothness?
• RQ2 Can ALiA achieve faster convergence empirically than standard FLiP ADMM and other adaptive primal-dual methods without using backtracking linesearch?
• RQ3 How do the adaptive stepsize rules (Subroutine 1 and Subroutine 2) leverage iterate-dependent quantities to ensure stability and speed?
• RQ4 How does ALiA perform relative to Condat–Vu and other adaptive schemes across problems with differentiable and non-differentiable components?

主要发现

• ALiA is globally convergent to a saddle point for convex and locally smooth objectives under the stated assumptions.
• ALiA preserves convergence without any backtracking linesearch, thanks to adaptive rules that depend on past iterates.
• The paper introduces two adaptive subroutines that yield larger stepsizes and improved convergence speed theoretically and empirically.
• Empirical results on real datasets show accelerated performance of ALiA compared to standard FLiP ADMM and competitive performance against existing adaptive methods.
• ALiA demonstrates superior or comparable performance across problem classes where it is applicable.