[Paper Review] Distributed Prediction-Correction ADMM for Time-Varying Convex Optimization

Nicola Bastianello, Andrea Simonetto+1 more|arXiv (Cornell University)|Sep 17, 2020

Distributed Control Multi-Agent Systems39 references11 citations

TL;DR

This paper proposes a distributed, prediction-correction ADMM algorithm with dual regularization for time-varying convex optimization. By predicting future cost changes and applying dual regularization to ensure linear convergence, the method achieves lower asymptotic error and consensus violation than inexact gradient and exact dual decomposition methods in dynamic settings.

ABSTRACT

This paper introduces a dual-regularized ADMM approach to distributed, time-varying optimization. The proposed algorithm is designed in a prediction-correction framework, in which the computing nodes predict the future local costs based on past observations, and exploit this information to solve the time-varying problem more effectively. In order to guarantee linear convergence of the algorithm, a regularization is applied to the dual, yielding a dual-regularized ADMM. We analyze the convergence properties of the time-varying algorithm, as well as the regularization error of the dual-regularized ADMM. Numerical results show that in time-varying settings, despite the regularization error, the performance of the dual-regularized ADMM can outperform inexact gradient-based methods, as well as exact dual decomposition techniques, in terms of asymptotical error and consensus constraint violation.

Motivation & Objective

• To address the challenge of tracking optimal solutions in time-varying distributed optimization under limited computation and communication.
• To develop a distributed algorithm that predicts future cost function changes and corrects errors when new data arrives.
• To improve convergence speed and reduce asymptotic error in time-varying settings through dual regularization.
• To demonstrate that regularization can enhance performance without sacrificing accuracy in dynamic optimization.

Proposed method

• Proposes a dual-regularized ADMM framework that modifies the dual problem by adding a quadratic regularization term to ensure strong convexity.
• Integrates a prediction-correction mechanism where nodes predict future local costs based on past observations to anticipate solution trajectory changes.
• Uses a consensus-based reformulation of the time-varying problem with local variables and auxiliary bridge variables for distributed computation.
• Applies the Peaceman-Rachford splitting method to solve the regularized dual problem in a distributed manner, enabling node-level updates via proximal operators.
• Derives distributed update rules for primal and dual variables using the structure of the graph and neighbor communication only.
• Employs a regularization parameter ϵ to balance convergence speed and solution accuracy, with theoretical bounds on the regularization error.

Experimental results

Research questions

• RQ1 Can dual regularization in ADMM improve convergence rate and reduce asymptotic error in time-varying distributed optimization?
• RQ2 Does a prediction-correction framework enhance tracking performance in dynamic distributed settings compared to standard ADMM?
• RQ3 How does the regularization error of the dual-regularized ADMM affect the final solution quality and consensus violation?
• RQ4 Can the proposed method outperform inexact gradient methods and exact dual decomposition in terms of tracking accuracy and convergence speed?

Key findings

• The dual-regularized ADMM achieves linear convergence, unlike standard ADMM, due to the strong convexity induced by the regularization.
• The regularization error is bounded and decreases with increasing regularization parameter ϵ, ensuring proximity to the true dual solution.
• Numerical results show that the proposed method achieves lower asymptotic error and consensus violation than inexact gradient methods and exact dual decomposition.
• Despite not enforcing consensus constraints exactly, the dual-regularized ADMM maintains better distance from consensus than competing methods.
• The performance gain is attributed to faster convergence from regularization, which reduces the bounded tracking error in time-varying settings.
• The method outperforms primal regularization (e.g., (1/2α)x⊤(I−W)x) in terms of tracking accuracy, demonstrating the superiority of dual regularization in dynamic scenarios.