[Paper Review] Central Server Free Federated Learning over Single-sided Trust Social Networks

Chaoyang He, Conghui Tan+3 more|arXiv (Cornell University)|Oct 11, 2019

Privacy-Preserving Technologies in Data45 references62 citations

TL;DR

This paper introduces Online Push-Sum (OPS), a centralized-server-free federated learning algorithm that works over single-sided trust (asymmetric) social networks with row-stochastic weights, and provides regret analysis and privacy considerations.

ABSTRACT

Federated learning has become increasingly important for modern machine learning, especially for data privacy-sensitive scenarios. Existing federated learning mostly adopts the central server-based architecture or centralized architecture. However, in many social network scenarios, centralized federated learning is not applicable (e.g., a central agent or server connecting all users may not exist, or the communication cost to the central server is not affordable). In this paper, we consider a generic setting: 1) the central server may not exist, and 2) the social network is unidirectional or of single-sided trust (i.e., user A trusts user B but user B may not trust user A). We propose a central server free federated learning algorithm, named Online Push-Sum (OPS) method, to handle this challenging but generic scenario. A rigorous regret analysis is also provided, which shows very interesting results on how users can benefit from communication with trusted users in the federated learning scenario. This work builds upon the fundamental algorithm framework and theoretical guarantees for federated learning in the generic social network scenario.

Motivation & Objective

• Motivate federated learning when a central server is unavailable or communication costs to a central server are prohibitive.
• Enable decentralized learning over directed graphs with single-sided trust (asymmetric topology).
• Develop an online, gradient-free exchange protocol that uses only model parameters to preserve privacy.
• Provide regret guarantees that separate adversarial and stochastic loss components.
• Demonstrate robustness and privacy advantages of OPS under various network settings.

Proposed method

• Propose Online Push-Sum (OPS), a decentralized algorithm that exchanges only model parameters and uses a row-stochastic confusion matrix W.
• Use local updates with gradient-based descent to compute intermediate variables z and a normalizing factor ω, then aggregate via push-sum to obtain updated local models x.
• Relax symmetry/double-stochasticity requirements, enabling arbitrary network topology and single-sided trust.
• Provide regret analysis decomposing losses into adversarial and stochastic components; derive optimal learning rate γ.
• Show that if W is not doubly stochastic, ω ensures unbiased aggregation and reduces to standard online gradient methods when W is doubly stochastic.

Experimental results

Research questions

• RQ1 Can federated learning be effectively performed on networks with single-sided trust edges and no central server?
• RQ2 What are the convergence (regret) properties when communications follow a row-stochastic, directed topology?
• RQ3 How does OPS compare to centralized and other decentralized methods under varying network sizes and densities?
• RQ4 Does OPS preserve privacy by exchanging models rather than gradients or raw data?

Key findings

• OPS achieves regret bounds of the order O(nGR√T + σR(1+√(nC2))√(nT)) with an optimally chosen γ, matching known online optimization rates in special cases.
• Models on all clients converge to the same model with rate O(1/T) under appropriate settings (Theorem 3).
• OPS outperforms decentralized online gradient methods that rely on symmetric/doubly stochastic topologies and benefits from single-sided trust topologies.
• Increasing network density improves performance, and larger networks can provide more stochastic samples that accelerate convergence.
• OPS maintains privacy advantages by exchanging models instead of gradients or data, complicating adversarial inference in asymmetric networks.