[Paper Review] Bethe-ADMM for Tree Decomposition based Parallel MAP Inference

Qiang Fu, Huahua Wang+1 more|arXiv (Cornell University)|Sep 26, 2013

Error Correcting Code Techniques26 references16 citations

TL;DR

This paper proposes Bethe-ADMM, a parallel MAP inference algorithm for discrete graphical models that combines tree decomposition with an inexact ADMM using a Bethe-divergence-based proximal function. The method enables efficient, scalable, and globally convergent inference by decomposing the graph into trees and solving subproblems in parallel via the sum-product algorithm, achieving near-linear speedup with increasing cores on both synthetic and real-world datasets.

ABSTRACT

We consider the problem of maximum a posteriori (MAP) inference in discrete graphical models. We present a parallel MAP inference algorithm called Bethe-ADMM based on two ideas: tree-decomposition of the graph and the alternating direction method of multipliers (ADMM). However, unlike the standard ADMM, we use an inexact ADMM augmented with a Bethe-divergence based proximal function, which makes each subproblem in ADMM easy to solve in parallel using the sum-product algorithm. We rigorously prove global convergence of Bethe-ADMM. The proposed algorithm is extensively evaluated on both synthetic and real datasets to illustrate its effectiveness. Further, the parallel Bethe-ADMM is shown to scale almost linearly with increasing number of cores.

Motivation & Objective

• To address the challenge of scalable and efficient maximum a posteriori (MAP) inference in large discrete graphical models.
• To enable parallel computation of MAP inference by decomposing the graphical model into tree-structured subproblems.
• To improve convergence and computational efficiency by introducing a Bethe-divergence-based proximal function in ADMM.
• To achieve global convergence of the algorithm despite using inexact subproblem solutions.
• To demonstrate linear scalability of the method with increasing numbers of computational cores.

Proposed method

• The method uses tree decomposition to partition the graphical model into tree-structured subgraphs, enabling parallel processing of subproblems.
• It applies an inexact ADMM framework to coordinate solutions across subproblems, reducing computational overhead.
• A Bethe-divergence-based proximal function is introduced to regularize subproblem updates, making them amenable to efficient solution via the sum-product algorithm.
• Each subproblem is solved in parallel using the sum-product algorithm, which is exact for tree-structured models.
• The algorithm ensures global convergence by leveraging the properties of the Bethe approximation and the ADMM convergence theory.
• The method is implemented with dual ascent and consensus constraints to maintain consistency across overlapping variables in the tree decomposition.

Experimental results

Research questions

• RQ1 Can tree decomposition be effectively combined with ADMM to enable scalable, parallel MAP inference in graphical models?
• RQ2 Does the use of a Bethe-divergence-based proximal function improve convergence and computational efficiency in ADMM-based inference?
• RQ3 Can the proposed Bethe-ADMM achieve near-linear speedup with increasing numbers of cores in practice?
• RQ4 How does Bethe-ADMM compare to existing inference methods in terms of solution quality and runtime on real and synthetic datasets?
• RQ5 Is global convergence guaranteed for the inexact ADMM variant using the Bethe proximal function?

Key findings

• Bethe-ADMM achieves global convergence despite using inexact subproblem solutions, which is a significant theoretical contribution.
• The algorithm demonstrates near-linear scalability with the number of cores, showing strong performance on large-scale inference tasks.
• On synthetic datasets, Bethe-ADMM outperforms standard ADMM and other baselines in convergence speed and solution accuracy.
• On real-world datasets, the method maintains high-quality MAP estimates while significantly reducing runtime through parallelization.
• The use of the Bethe-divergence proximal function enables efficient sum-product algorithm-based subproblem solving, which is critical for scalability.
• Empirical results confirm that the algorithm converges to high-quality solutions even on loopy graphical models, thanks to the tree decomposition and proximal regularization.