[论文解读] Federated Incremental Subgradient Method for Convex Bilevel Optimization Problems

Sudkobfa Boontawee, Mootta Prangprakhon+1 more|arXiv (Cornell University)|Jan 21, 2026

Optimization and Variational Analysis被引用 0

一句话总结

该论文引入了联邦增量次梯度法（FISM）以在联邦设置下解决凸双层优化，证明收敛性并在二元分类和一个位置问题中展示出优于 IR-IG 的性能。

ABSTRACT

In this letter, we consider a bilevel optimization problem in which the outer-level objective function is strongly convex, whereas the inner-level problem consists of a finite sum of convex functions. Bilevel optimization problems arise in situations where the inner-level problem does not have a unique solution. This has led to the idea of introducing an outer-level objective function to select a solution with the specific desired properties. We propose an iterative method that combines an incremental algorithm with a broadcast algorithm, both based on the principles of federated learning. Under appropriate assumptions, we establish the convergence results of the proposed algorithm. To demonstrate its performance, we present two numerical examples related to binary classification and a location problem.

研究动机与目标

• Address bilevel optimization where the outer objective is strongly convex and the inner problem is a finite sum of convex functions.
• Develop a federated incremental subgradient algorithm combining incremental updates with federated learning principles.
• Prove convergence of the proposed method and establish a rate for the inner-level objective.
• Demonstrate effectiveness through numerical experiments on binary classification and a location problem.

提出的方法

• Propose FISM, which uses an outer-level subgradient H and local inner-level subgradients from multiple clients.
• Clients perform local incremental updates with a projected subgradient step and a regularization-like term involving the shared subgradient of H.
• Central server aggregates client updates to form the next global iterate, sharing only the subgradient of H to preserve privacy.
• Provide a convergence analysis under standard assumptions, including nonincreasing step sizes and iterative regularization, and derive a rate for the inner-level objective.

实验结果

研究问题

• RQ1 Can FISM converge to the unique solution of the bilevel problem under federated learning constraints?
• RQ2 What are the convergence guarantees and rate for the inner-level objective when using iterative regularization in FL?
• RQ3 How does FISM compare to IR-IG in terms of convergence, computation, and communication overhead under data decentralization?

主要发现

n	m	IR-IG	S=1	S=2	S=4	S=8
10	500	16.92	15.96	5.95	2.34	1.39
10	1000	42.88	40.51	13.47	5.31	2.76
10	5000	374.42	351.27	120.96	42.67	18.12
10	10000	849.49	800.44	295.64	104.30	38.62
50	500	15.78	14.77	6.13	3.37	2.77
50	1000	35.47	33.20	12.46	5.06	2.96
50	5000	318.56	298.60	106.08	36.96	15.26
50	10000	808.79	760.02	277.83	98.70	36.61
100	500	17.33	16.17	8.31	4.74	3.91
100	1000	34.73	32.44	12.90	5.95	4.10
100	5000	292.83	276.16	97.92	35.52	14.08
100	10000	768.78	724.63	256.56	89.95	35.93

• FISM converges to the unique outer-level solution xH* under stated assumptions.
• A convergence rate for the inner-level objective value of the averaged iterate is established with appropriate step-size schedules.
• Empirical results show FISM outperforming IR-IG in both binary MNIST classification and a location problem, especially as the number of groups S increases.
• FISM achieves faster running times and often better test accuracy and training loss than IR-IG across tested settings.
• Communication-efficient design (sharing only subgradient information H) preserves privacy while enabling FL-style aggregation.
• Asynchronous or heterogeneous settings are proposed as future improvements to mitigate slow-client bottlenecks.