[Paper Review] Strong Metric (Sub)regularity of KKT Mappings for Piecewise Linear-Quadratic Convex-Composite Optimization

James V. Burke, Abraham Engle|arXiv (Cornell University)|May 3, 2018

Sparse and Compressive Sensing Techniques30 references7 citations

TL;DR

This paper establishes strong metric subregularity and regularity conditions for Karush-Kuhn-Tucker (KKT) mappings in piecewise linear-quadratic (PLQ) convex-composite optimization, enabling local quadratic convergence of Newton and quasi-Newton methods under conditions analogous to nonlinear programming. The analysis uses generalized equations and active manifold identification to extend second-order convergence theory beyond finite-valued functions to the broader PLQ class.

ABSTRACT

This work concerns the local convergence theory of Newton and quasi-Newton methods for convex-composite optimization: minimize f(x):=h(c(x)), where h is an infinite-valued proper convex function and c is C^2-smooth. We focus on the case where h is infinite-valued piecewise linear-quadratic and convex. Such problems include nonlinear programming, mini-max optimization, estimation of nonlinear dynamics with non-Gaussian noise as well as many modern approaches to large-scale data analysis and machine learning. Our approach embeds the optimality conditions for convex-composite optimization problems into a generalized equation. We establish conditions for strong metric subregularity and strong metric regularity of the corresponding set-valued mappings. This allows us to extend classical convergence of Newton and quasi-Newton methods to the broader class of non-finite valued piecewise linear-quadratic convex-composite optimization problems. In particular we establish local quadratic convergence of the Newton method under conditions that parallel those in nonlinear programming when h is non-finite valued piecewise linear.

Motivation & Objective

• To extend local convergence theory of Newton and quasi-Newton methods to convex-composite problems where the composite function h is infinite-valued piecewise linear-quadratic (PLQ) and convex.
• To establish conditions under which the KKT mapping is strongly metrically subregular and regular, ensuring fast local convergence.
• To bridge the gap between classical nonlinear programming convergence results and non-finite-valued PLQ functions by leveraging generalized equations and active manifold structure.
• To provide a theoretical foundation for second-order methods in modern optimization problems such as robust regression, non-Gaussian estimation, and machine learning.

Proposed method

• Embeds the optimality conditions of convex-composite problems into a generalized equation of the form g(x,y) + G(x,y) ∋ 0, where g is C¹-smooth and G is a set-valued mapping.
• Uses the KKT matrix ∇g(x,y) derived from the Lagrangian structure to define Newton iterates via linearization of the generalized equation.
• Applies tools from variational analysis, including strong metric subregularity and regularity, to analyze local convergence behavior.
• Employs Rockafellar’s partial smoothness and active manifold theory to characterize the geometry of dom(h) and identify the active set structure.
• Imposes second-order sufficient conditions on the Hessian of the Lagrangian and the curvature of the active manifold to ensure local uniqueness and fast convergence.
• Establishes that when the KKT mapping is strongly metrically subregular at a solution, Newton iterates converge quadratically.

Experimental results

Research questions

• RQ1 Under what conditions is the KKT mapping strongly metrically subregular for PLQ-convex-composite problems?
• RQ2 Can local quadratic convergence of Newton methods be established for non-finite-valued PLQ functions, similar to classical nonlinear programming?
• RQ3 How does the active manifold structure of the PLQ function h influence the regularity and convergence of second-order methods?
• RQ4 What role does the generalized equation framework play in unifying convergence analysis across different classes of composite optimization problems?

Key findings

• The KKT mapping is strongly metrically subregular at a solution if the second-order sufficient conditions hold and the active manifold is partially smooth.
• Local quadratic convergence of Newton’s method is achieved when the KKT mapping is strongly metrically subregular, which holds under conditions parallel to those in nonlinear programming.
• The Newton iterate corresponds to solving the linearized generalized equation g(x^k,y^k) + ∇g(x^k,y^k)(x^{k+1}-x^k, y^{k+1}-y^k) = 0, ensuring fast local convergence.
• When h is piecewise linear (a special case of PLQ), the results recover known convergence rates from Womersley (1988), but now extended to non-finite-valued settings.
• The analysis shows that the Newton method converges quadratically in a neighborhood of the solution when the Hessian of the Lagrangian is positive definite on the tangent space of the active manifold.
• The method remains well-defined and convergent even when h is infinite-valued, provided the active manifold structure is preserved locally.