[论文解读] On the Exponential Stability of Projected Primal-Dual Dynamics on a Riemannian Manifold

Prashant Bansode, Venkatesh Chinde+3 more|arXiv (Cornell University)|May 11, 2019

Optimization and Variational Analysis参考文献 32被引用 6

一句话总结

本文提出了一种基于黎曼流形的投影对偶-对偶动态的重新表述，用于线性不等式约束的凸优化问题，通过选择适当的黎曼度量使拉格朗日梯度强单调化，从而实现全局指数稳定性。关键结果是在梯度映射满足利普希茨连续性时，获得全局指数稳定的鞍点收敛，并给出明确的指数衰减界。

ABSTRACT

Equivalence of convex optimization, saddle-point problems, and variational inequalities is a well-established concept. The variational inequality (VI) is a static problem which is studied under dynamical settings using a framework called the projected dynamical system, whose stationary points coincide with the static solutions of the associated VI. VI has rich properties concerning the monotonicity of its vector-valued map and the uniqueness of its solution, which can be extended to convex optimization and saddle-point problems. Moreover, these properties also extend to the representative projected dynamical system. The objective of this paper is to harness rich monotonicity properties of the representative projected dynamical system to develop the solution concepts of the convex optimization problem and the associated saddle-point problem. To this end, this paper studies a linear inequality constrained convex optimization problem and models its equivalent saddle-point problem as a VI. Further, the VI is studied as a projected dynamical system\cite{friesz1994day} which is shown to converge to the saddle-point solution. By considering the monotonicity of the gradient of Lagrangian function as a key factor, this paper establishes exponential convergence and stability results concerning the saddle-points. Our results show that the gradient of the Lagrangian function is just monotone on the Euclidean space, leading to only Lyapunov stability of stationary points of the projected dynamical system. To remedy the situation, the underlying projected dynamical system is formulated on a Riemannian manifold whose Riemannian metric is chosen such that the gradient of the Lagrangian function becomes strongly monotone. Using a suitable Lyapunov function, the stationary points of the projected dynamical system are proved to be globally exponentially stable and convergent to the unique saddle-point.

研究动机与目标

• 为了解决标准投影对偶-对偶动态在欧几里得空间中仅能提供李雅普诺夫稳定性、缺乏显式收敛速率的局限性。
• 在具有线性不等式约束的凸优化中，实现对鞍点动态的全局指数稳定性。
• 通过将动态嵌入黎曼流形，克服欧几里得空间中拉格朗日梯度缺乏强单调性的问题。
• 基于黎曼结构设计的李雅普诺夫函数，建立指数收敛的充分条件。
• 将理论框架扩展至在线和分布式优化场景，以实现有限时间收敛保证。

提出的方法

• 将线性不等式约束凸优化问题的鞍点问题表述为变分不等式。
• 将动态建模为黎曼流形上的投影动态系统，其中黎曼度量被特别选择以诱导拉格朗日梯度的强单调性。
• 定义黎曼投影算子 $ P^r_{\mathcal{M}} $，以在非负正交空间内对对偶变量施加约束。
• 基于黎曼范数 $ \|\cdot\|_r $ 构造李雅普诺夫函数，以分析稳定性与收敛性。
• 利用不等式 $ \|z(t) - z^*\|_r \leq \|z(0) - z^*\|_r e^{-\frac{\alpha\beta(4\nu - \alpha\ell^2)}{8}t} $ 推导指数收敛界，该不等式在 $ \alpha < \frac{4\nu}{\ell^2} $ 时成立。
• 通过动态的欧拉离散化验证理论结果，显示在不同 $ k $ 和 $ \nu $ 下的模拟中呈现几何收敛。

实验结果

研究问题

• RQ1 在黎曼流形上的投影对偶-对偶动态是否能实现全局指数稳定性，而欧几里得形式仅能提供李雅普诺夫稳定性？
• RQ2 如何设计黎曼度量以确保拉格朗日梯度的强单调性，从而实现指数收敛？
• RQ3 步长 $ \alpha, \beta $ 和系统参数 $ \nu, \ell $ 需满足何种条件，才能保证误差向鞍点的指数衰减？
• RQ4 在梯度映射满足利普希茨连续性时，所提出的框架是否仍能保持指数收敛？
• RQ5 通过欧拉离散化实现的离散时间实现中，理论收敛速率是否可被观测到？

主要发现

• 当通过度量设计使拉格朗日梯度强单调化时，黎曼流形上的投影对偶-对偶动态可实现全局指数稳定性。
• 指数收敛速率由界 $ \|z(t) - z^*\|_r \leq \|z(0) - z^*\|_r e^{-\frac{\alpha\beta(4\nu - \alpha\ell^2)}{8}t} $ 明确定义，且在 $ \alpha < \frac{4\nu}{\ell^2} $ 时成立。
• 增大参数 $ k $ 可提高 $ \nu $，从而增强指数衰减系数，如模拟所示，收敛速度加快。
• 仿真结果证实了离散时间投影动态的几何收敛性，验证了连续时间指数稳定性的结论。
• 原始变量和对偶变量收敛至最优解 $ x^* $ 和 $ \lambda^* $，误差如图5所示在 $ L_2 $-正则化最小二乘问题中呈指数衰减。
• 该方法提供的收敛保证强于渐近稳定性，因此适用于实时和在线优化应用。