[Paper Review] Max-Product Belief Propagation for Linear Programming: Applications to Combinatorial Optimization

Sejun Park, Jinwoo Shin|arXiv (Cornell University)|Dec 16, 2014

Bayesian Modeling and Causal Inference25 references4 citations

TL;DR

This paper establishes a generic convergence and correctness criterion for max-product belief propagation (BP) in solving linear programming (LP) formulations of combinatorial optimization problems. It proves that BP converges to the optimal LP solution under a unified set of sufficient conditions, generalizing prior results and enabling BP to solve diverse problems including maximum weight perfect matching, shortest path, traveling salesman, cycle packing, vertex/edge cover, and network flow with fractional solutions.

ABSTRACT

The max-product {belief propagation} (BP) is a popular message-passing heuristic for approximating a maximum-a-posteriori (MAP) assignment in a joint distribution represented by a graphical model (GM). In the past years, it has been shown that BP can solve a few classes of linear programming (LP) formulations to combinatorial optimization problems including maximum weight matching, shortest path and network flow, i.e., BP can be used as a message-passing solver for certain combinatorial optimizations. However, those LPs and corresponding BP analysis are very sensitive to underlying problem setups, and it has been not clear what extent these results can be generalized to. In this paper, we obtain a generic criteria that BP converges to the optimal solution of given LP, and show that it is satisfied in LP formulations associated to many classical combinatorial optimization problems including maximum weight perfect matching, shortest path, traveling salesman, cycle packing, vertex/edge cover and network flow.

Motivation & Objective

• To develop a general, sufficient condition ensuring max-product belief propagation (BP) converges to the optimal solution of a linear program (LP).
• To unify and generalize prior BP convergence results for specific combinatorial problems like matching, shortest path, and network flow.
• To extend BP applicability beyond integral solutions to include fractional LP solutions.
• To provide design guidelines for constructing BP algorithms that satisfy the proposed convergence criteria.
• To overcome limitations of prior BP analyses that were highly sensitive to problem-specific structures.

Proposed method

• Introduces a generic criterion based on LP tightness and structural properties of the LP polytope to guarantee BP convergence.
• Uses the computational tree structure of BP and analyzes alternating paths to establish convergence behavior.
• Applies polyhedral geometry to analyze vertices of the LP feasible region and their stability under perturbations.
• Employs a perturbation argument on the LP constraints to show convergence of BP fixed points to vertices of the polytope.
• Leverages invertibility of submatrices of the constraint matrix to characterize vertex solutions.
• Establishes a distance bound between the original and perturbed LP feasible sets to prove convergence.

Experimental results

Research questions

• RQ1 Under what general conditions does max-product belief propagation converge to the optimal solution of a linear program?
• RQ2 Can the convergence and correctness of BP be guaranteed across diverse combinatorial optimization problems using a single unified framework?
• RQ3 How can BP be designed to handle both integral and fractional LP solutions?
• RQ4 What structural properties of the LP polytope ensure that BP fixed points are unique and optimal?
• RQ5 Can the technical limitations of prior problem-specific BP analyses be overcome through a general polyhedral approach?

Key findings

• The proposed criterion guarantees that max-product BP converges to the optimal solution of any LP formulation satisfying the conditions, regardless of initialization.
• The framework rediscoveries all prior known convergence results for matching, perfect matching, shortest path, and network flow problems.
• The method extends to new problems such as the traveling salesman problem, cycle packing, vertex/edge cover, and network flow with fractional solutions.
• The analysis proves that BP fixed points are unique under the proposed conditions, ensuring deterministic convergence.
• The distance between the original and perturbed LP feasible sets is bounded by O(ε), which ensures convergence as ε→0.
• The approach provides a systematic way to design BP algorithms for new LP formulations by verifying the proposed criteria.