[Paper Review] Locally upper Lipschitz of the perturbed KKT system of Ky Fan $k$-norm matrix conic optimization problems

Yulan Liu, Shaohua Pan|arXiv (Cornell University)|Sep 2, 2015

Sparse and Compressive Sensing Techniques13 references6 citations

TL;DR

This paper establishes the locally upper Lipschitz property of the solution mappings for two types of perturbed KKT systems in Ky Fan $k$-norm matrix conic optimization problems. Under the second-order sufficient condition and strict Robinson’s constraint qualification, it proves a local error bound via an equivalent characterization of the graphical derivative of the normal cone to the $k$-norm matrix cone, which ensures convergence rate analysis for optimization algorithms.

ABSTRACT

This note is concerned with the nonlinear Ky Fan $k$-norm matrix conic optimization problems, which include the nuclear norm regularized minimization problem as a special case. For this class of nonpolyhedral matrix conic optimization problems, under the assumption that a stationary solution satisfies the second-order sufficient condition and the associated Lagrange multiplier satisfies the strict Robinson's CQ, we show that two classes of perturbed KKT systems are locally upper Lipschitz at the origin, which implies a local error bound for the distance from any point in a neighborhood of the corresponding KKT point to the whole set of KKT points.

Motivation & Objective

• To analyze the stability of solution mappings for perturbed KKT systems in Ky Fan $k$-norm matrix conic optimization problems.
• To establish local error bounds for these systems, which are crucial for convergence rate analysis of numerical algorithms.
• To characterize the graphical derivative of the normal cone mapping to the $k$-norm matrix cone, enabling the analysis of solution stability.
• To extend convergence theory to nonpolyhedral matrix conic optimization problems, particularly those involving the nuclear norm as a special case.

Proposed method

• Uses the Karush-Kuhn-Tucker (KKT) system reformulated via the Lagrangian and projection onto the Ky Fan $k$-norm cone.
• Applies an equivalent characterization of the graphical derivative of the normal cone to the $k$-norm matrix cone, derived from singular value decomposition and spectral decomposition.
• Employs second-order sufficient conditions and strict Robinson’s constraint qualification to ensure strong stability of the KKT solution mapping.
• Analyzes two types of perturbed KKT systems: one with perturbations in the KKT residual and another with perturbations in the problem data.
• Utilizes directional derivatives and tangent cone analysis to derive conditions under which the solution mapping is locally upper Lipschitz.
• Relies on advanced tools from variational analysis, including contingent cones and normal cones in convex analysis, to establish the main result.

Experimental results

Research questions

• RQ1 Under what conditions is the solution mapping of the perturbed KKT system for Ky Fan $k$-norm matrix conic optimization locally upper Lipschitz at the origin?
• RQ2 How can the graphical derivative of the normal cone to the $k$-norm matrix cone be characterized in terms of singular values and spectral projections?
• RQ3 What role does the strict Robinson’s constraint qualification play in ensuring local stability of the KKT solution mapping?
• RQ4 How does the second-order sufficient condition contribute to the local error bound and Lipschitz stability of the solution mapping?
• RQ5 Can the local error bound derived from this analysis be used to establish convergence rates for numerical algorithms solving such optimization problems?

Key findings

• The solution mapping of the perturbed KKT system is locally upper Lipschitz at the origin under the second-order sufficient condition and strict Robinson’s CQ.
• An equivalent characterization of the graphical derivative of the normal cone to the $k$-norm matrix cone is derived using spectral decomposition and singular value structure.
• The local error bound holds under the same conditions, which is essential for convergence rate analysis of first-order methods.
• The result applies to nuclear norm regularized problems as a special case when $k = m$, extending stability results to low-rank matrix recovery problems.
• The analysis confirms that the KKT system remains stable under small perturbations in the data, ensuring robustness of numerical solvers.
• The proof technique relies on directional derivatives of the projection operator and tangent cone analysis, providing a rigorous foundation for sensitivity analysis.