[Paper Review] On belief propagation guided decimation for random k-SAT

Amin Coja‐Oghlan|arXiv (Cornell University)|Jan 23, 2011

Constraint Satisfaction and Optimization28 references23 citations

TL;DR

This paper provides the first rigorous analysis of belief propagation guided decimation (BPD) for solving random k-SAT problems. It demonstrates that BPD fails to find a satisfying assignment when the clause-to-variable ratio exceeds ρ·rk/k for some constant ρ > 0 independent of k, indicating a fundamental limitation despite empirical success in low-density regimes for small k.

ABSTRACT

Let Φ be a uniformly distributed random k-SAT formula with n variables and m clauses. Non-constructive arguments show that Φ is satisfiable for clause/variable ratios m/n ≤ rk ~ 2k ln 2 with high probability (Achlioptas, Moore: SICOMP 2006; Achlioptas, Peres: J. AMS 2004). Yet no efficient algorithm is know to find a satisfying assignment for densities as low as m/n ~ rk · ln(k)/k with a non-vanishing probability. In fact, the density m/n ~ rk · ln(k)/k seems to form a barrier for a broad class of local search algorithms (Achlioptas, Coja-Oghlan: FOCS 2008). On the basis of deep but non-rigorous statistical mechanics considerations, a message passing algorithm called belief propagation guided decimation for solving random k-SAT has been forward (Mezard, Parisi, Zecchina: Science 2002; Braunstein, Mezard, Zecchina: RSA 2005). Experiments suggest that the algorithm might succeed for densities very close to rk for k = 3, 4, 5 (Kroc, Sabharwal, Selman: SAC 2009). Furnishing the first rigorous analysis of belief propagation guided decimation on random k-SAT, the present paper shows that the algorithm fails to find a satisfying assignment already for m/n ≥ ρ · rk/k, for a constant ρ > 0 independent of k.

Motivation & Objective

• To rigorously analyze the performance of belief propagation guided decimation (BPD) on random k-SAT formulas.
• To determine the maximum clause-to-variable ratio m/n beyond which BPD fails to find a satisfying assignment with non-vanishing probability.
• To identify whether BPD can overcome the known algorithmic barrier at m/n ~ rk·ln(k)/k for large k.
• To establish a theoretical limit on the effectiveness of BPD in the random k-SAT setting using non-rigorous statistical mechanics insights.

Proposed method

• The analysis focuses on uniformly distributed random k-SAT formulas with n variables and m clauses.
• It applies rigorous probabilistic methods to evaluate the behavior of belief propagation guided decimation under varying clause-to-variable ratios.
• The study examines the evolution of variable beliefs and decimation decisions during the BPD process.
• It uses concentration arguments and probabilistic analysis to assess the likelihood of successful decimation to a satisfying assignment.
• The analysis identifies a threshold ρ·rk/k, where ρ > 0 is a constant independent of k, beyond which BPD fails.
• Theoretical bounds are derived based on the structure of the factor graph and the convergence properties of belief propagation in sparse random instances.

Experimental results

Research questions

• RQ1 Can belief propagation guided decimation successfully solve random k-SAT instances at clause-to-variable ratios approaching the satisfiability threshold rk?
• RQ2 What is the maximum clause-to-variable ratio m/n for which BPD can still find a satisfying assignment with non-vanishing probability?
• RQ3 Does BPD overcome the algorithmic barrier at m/n ~ rk·ln(k)/k for large k?
• RQ4 Is there a theoretical limit to the performance of BPD in random k-SAT, and if so, what is its form?

Key findings

• Belief propagation guided decimation fails to find a satisfying assignment for clause-to-variable ratios m/n ≥ ρ·rk/k, where ρ > 0 is a constant independent of k.
• This failure occurs even though empirical results suggest BPD succeeds for densities close to rk for small k (k = 3, 4, 5).
• The result establishes a rigorous upper bound on the performance of BPD, showing it cannot scale to the full satisfiability threshold for large k.
• The failure threshold ρ·rk/k is significantly below the known satisfiability threshold rk, indicating a substantial gap between empirical success and theoretical limits.
• The analysis reveals that BPD is fundamentally limited by the structure of belief propagation in sparse random instances, even when guided by message-passing heuristics.