[Paper Review] ADMM Based Privacy-preserving Decentralized Optimization

Chunlei Zhang, Muaz Ahmad+1 more|arXiv (Cornell University)|Jul 13, 2017

Stochastic Gradient Optimization Techniques49 references4 citations

TL;DR

This paper proposes a novel privacy-preserving decentralized optimization framework using ADMM with partially homomorphic encryption, enabling secure, aggregator-free optimization in multi-agent systems. It introduces a time-varying penalty matrix ADMM variant that achieves an O(1/t) convergence rate while protecting sensitive agent states from eavesdroppers and neighboring agents.

ABSTRACT

Privacy preservation is addressed for decentralized optimization, where $N$ agents cooperatively minimize the sum of $N$ convex functions private to these individual agents. In most existing decentralized optimization approaches, participating agents exchange and disclose states explicitly, which may not be desirable when the states contain sensitive information of individual agents. The problem is more acute when adversaries exist which try to steal information from other participating agents. To address this issue, we propose a privacy-preserving decentralized optimization approach based on ADMM and partially homomorphic cryptography. To our knowledge, this is the first time that cryptographic techniques are incorporated in a fully decentralized setting to enable privacy preservation in decentralized optimization in the absence of any third party or aggregator. To facilitate the incorporation of encryption in a fully decentralized manner, we introduce a new ADMM which allows time-varying penalty matrices and rigorously prove that it has a convergence rate of $O(1/t)$. Numerical and experimental results confirm the effectiveness and low computational complexity of the proposed approach.

Motivation & Objective

• To address privacy leakage in decentralized optimization where agents explicitly exchange sensitive states.
• To enable end-to-end privacy preservation in fully decentralized settings without third-party aggregators.
• To integrate cryptographic techniques—specifically partially homomorphic encryption—into a decentralized ADMM framework.
• To maintain convergence and computational efficiency while ensuring strong privacy guarantees.
• To develop a new ADMM variant with time-varying penalty matrices that supports privacy-preserving computation.

Proposed method

• Introduces a modified ADMM algorithm with time-varying penalty matrices to support secure computation in decentralized networks.
• Employs partially homomorphic encryption to protect agent states during message exchange, preventing eavesdroppers from inferring sensitive data.
• Designs a decentralized encryption scheme where each agent encrypts its local variables before transmission, ensuring no plaintext is exposed.
• Proves convergence of the proposed ADMM variant under mild conditions, showing that the algorithm converges to the optimal solution of the original problem.
• Establishes theoretical convergence rate of O(1/t) by analyzing the dual variable and primal variable updates using Lyapunov-like functions.
• Uses a novel Lyapunov function to analyze convergence, incorporating the time-varying penalty matrices and the structure of the homomorphic encryption.

Experimental results

Research questions

• RQ1 Can privacy be preserved in decentralized optimization without relying on a central aggregator or trusted third party?
• RQ2 How can homomorphic encryption be integrated into a decentralized ADMM framework to protect sensitive agent states?
• RQ3 What convergence guarantees can be established for an ADMM variant with time-varying penalty matrices in a privacy-preserving setting?
• RQ4 Does the proposed method maintain a reasonable convergence rate despite the added cryptographic layer?
• RQ5 Can the proposed approach prevent adversaries from inferring private information through observation of exchanged messages?

Key findings

• The proposed ADMM with time-varying penalty matrices achieves a convergence rate of O(1/t), matching standard ADMM performance.
• The method ensures that no agent reveals its private state in plaintext, even to direct neighbors, due to encryption of all exchanged messages.
• Numerical results confirm that the algorithm converges to the optimal solution with low computational overhead.
• Theoretical analysis proves that the algorithm converges to a saddle point of the Lagrangian function, ensuring optimality of the solution.
• The approach is resilient to eavesdroppers and prevents inference of private information such as agent positions or training data.
• The integration of partially homomorphic encryption into a fully decentralized ADMM framework is shown to be feasible and effective for convex optimization problems.