[论文解读] On the resolution of l0 minimization problems via alternating Lagrangian schemes

Yue Xie, Uday V. Shanbhag|arXiv (Cornell University)|Oct 12, 2017

Sparse and Compressive Sensing Techniques被引用 1

一句话总结

该论文通过将ℓ₀-最小化问题重新表述为互补约束数学规划（MPCC），提出了解决稀疏恢复和机器学习中关键问题的方法。采用ADMM和增广拉格朗日算法，证明了其收敛至KKT点和局部极小值点，且在低维情况下ADMM可获得近全局解，在高维情况下表现具有竞争力。

ABSTRACT

We consider an $\ell_0$-minimization problem where $f(x) + \|x\|_0$ is minimized over a polyhedral set and the $\ell_0$-norm penalty implicitly emphasizes sparsity of the solution. Such a setting captures a range of problems in image processing and statistical learning. However, given the the nonconvex and discontinuous nature of this norm, convex penalties are often employed as substitutes, and far less is known about directly solving the $\ell_0$-minimization problem. In this paper, inspired by Feng et.al. [20], we consider the resolution of an equivalent formulation of the $\ell_0$-minimization problem as a mathematical program with complementarity constraints (MPCC) and make the following contributions. (i) First, we show that feasible points of this formulation satisfy Guignard constraint qualification. In fact, under suitable convexity assumptions on $f(x)$, KKT conditions are sufficient. (ii) Next, we consider the resolution of the MPCC formulation through two Lagrangian schemes. The first is an ADMM scheme in which we prove that despite the overall nonconvexity, each ADMM subproblem can be solved efficiently recognizing a hidden convexity property. Furthermore, every limit point of the sequence produced by this scheme is a first-order KKT point and a local minimizer, under additional conditions. (iii) The second algorithm is an augmented Lagrangian scheme in which the Lagrangian subproblem is resolved by a proximal alternating algorithm. Under suitable boundedness requirements, the sequence admits a limit point that satisfies the criticality requirement. Preliminary numerics show that solutions of the ADMM scheme are near global in low dimensions and competitive against other methods in high dimensions. Moreover, the augmented Lagrangian scheme often provides solutions of comparable or better quality than the ADMM scheme at some computational cost.

研究动机与目标

• 解决在稀疏信号恢复和统计学习中出现的非凸、不连续ℓ₀-最小化问题的直接求解挑战。
• 通过开发针对原始ℓ₀-范数公式的算法，克服凸松弛方法（如ℓ₁-正则化）的局限性。
• 在适当假设下，建立收敛至一阶最优性条件的理论保证。
• 设计高效算法，利用子问题中隐藏的凸性，尽管整体问题为非凸。
• 比较ADMM和增广拉格朗日算法在解的质量和计算成本方面的性能。

提出的方法

• 将ℓ₀-最小化问题重新表述为互补约束数学规划（MPCC），以应用约束优化技术。
• 应用ADMM算法，其中每个子问题通过识别隐藏的凸性结构被高效求解，尽管整体问题为非凸。
• 证明在额外凸性和有界性假设下，ADMM序列的每个极限点均满足一阶KKT条件，并为局部极小值点。
• 开发增广拉格朗日算法，其中拉格朗日子问题通过一种近端交替算法求解，以处理非凸性。
• 在增广拉格朗日序列有界性要求下，建立收敛至临界点（满足KKT条件）的理论。
• 利用Guignard约束规范性和在凸性假设下的KKT充分性，强化理论保证。

实验结果

研究问题

• RQ1 ℓ₀-最小化问题能否通过MPCC重构有效求解，同时保持理论收敛保证？
• RQ2 尽管存在非凸性，ADMM算法应用于MPCC重构后是否收敛至KKT点和局部极小值点？
• RQ3 ADMM算法在解的质量和计算成本方面与增广拉格朗日算法相比表现如何？
• RQ4 ADMM框架中，何种隐藏凸性特性使得非凸ℓ₀-最小化问题的子问题可高效求解？
• RQ5 在何种条件下，所提算法能产生近全局解或与现有方法具有可比性能？

主要发现

• MPCC重构的可行点满足Guignard约束规范性，且在f(x)具有凸性假设下，KKT条件对最优性充分。
• 在附加条件下，ADMM算法收敛至一阶KKT点和局部极小值点，且由于隐藏凸性，每个子问题均可高效求解。
• 初步数值实验表明，ADMM算法在低维问题中可产生接近全局最优的解。
• 在高维情况下，ADMM算法与现有其他方法相比表现出具有竞争力的性能。
• 增广拉格朗日算法产生的解质量与ADMM相当或更优，但计算成本更高。
• 在适当的有界性假设下，增广拉格朗日算法存在满足临界性要求（即KKT条件）的极限点。