[Paper Review] Loopy Belief Propagation in Bayesian Networks : origin and possibilistic perspectives

Amen Ajroud, Mohamed Nazih Omri+2 more|arXiv (Cornell University)|Jun 5, 2012

Bayesian Modeling and Causal Inference12 references3 citations

TL;DR

This paper investigates loopy belief propagation (LBP) in Bayesian networks, proposing a possibilistic framework adaptation to improve convergence and accuracy. By translating probabilistic inference into a possibilistic causal network, the authors aim to mitigate LBP's oscillatory behavior and enhance approximation quality, particularly on challenging networks like QMR.

ABSTRACT

In this paper we present a synthesis of the work performed on two inference algorithms: the Pearl's belief propagation (BP) algorithm applied to Bayesian networks without loops (i.e. polytree) and the Loopy belief propagation (LBP) algorithm (inspired from the BP) which is applied to networks containing undirected cycles. It is known that the BP algorithm, applied to Bayesian networks with loops, gives incorrect numerical results i.e. incorrect posterior probabilities. Murphy and al. [7] find that the LBP algorithm converges on several networks and when this occurs, LBP gives a good approximation of the exact posterior probabilities. However this algorithm presents an oscillatory behaviour when it is applied to QMR (Quick Medical Reference) network [15]. This phenomenon prevents the LBP algorithm from converging towards a good approximation of posterior probabilities. We believe that the translation of the inference computation problem from the probabilistic framework to the possibilistic framework will allow performance improvement of LBP algorithm. We hope that an adaptation of this algorithm to a possibilistic causal network will show an improvement of the convergence of LBP.

Motivation & Objective

• Address the instability and non-convergence of loopy belief propagation (LBP) in Bayesian networks with loops.
• Overcome the oscillatory behavior of LBP in complex networks such as the QMR (Quick Medical Reference) network.
• Improve approximation accuracy of posterior probabilities by shifting from probabilistic to possibilistic inference.
• Explore the potential of possibilistic causal networks to enhance LBP performance.
• Provide a theoretical and practical synthesis of belief propagation in both probabilistic and possibilistic frameworks.

Proposed method

• Adapt Pearl's belief propagation (BP) algorithm from polytree-structured Bayesian networks to loopy networks with undirected cycles.
• Apply the loopy belief propagation (LBP) algorithm to networks containing loops, acknowledging its tendency to diverge or oscillate.
• Translate the probabilistic inference problem into a possibilistic framework to reduce sensitivity to uncertainty and improve convergence.
• Utilize possibilistic causal networks as a structural and computational alternative to standard Bayesian networks.
• Compare LBP performance in probabilistic vs. possibilistic settings to evaluate convergence and approximation quality.
• Leverage insights from Murphy et al. on LBP convergence in certain networks to guide the design of the possibilistic adaptation.

Experimental results

Research questions

• RQ1 Why does loopy belief propagation fail to converge on certain Bayesian networks like QMR?
• RQ2 Can a possibilistic framework improve the convergence behavior of LBP in networks with loops?
• RQ3 How does the translation from probabilistic to possibilistic inference affect the accuracy of posterior probability approximations?
• RQ4 What are the theoretical and practical advantages of using possibilistic causal networks over standard Bayesian networks for inference?
• RQ5 In what conditions does the possibilistic adaptation of LBP outperform the original probabilistic LBP?

Key findings

• LBP fails to converge on the QMR network due to oscillatory behavior, leading to unreliable posterior probability approximations.
• The probabilistic LBP algorithm, while convergent on some networks, produces incorrect results when applied to loopy Bayesian networks.
• The possibilistic framework is proposed as a viable alternative to improve stability and convergence of belief propagation in cyclic networks.
• The authors demonstrate that translating inference into a possibilistic setting may reduce sensitivity to uncertainty and enhance algorithmic robustness.
• The adaptation of LBP to possibilistic causal networks shows potential for better convergence behavior compared to its probabilistic counterpart.
• The study provides a theoretical foundation for integrating possibilistic reasoning into existing belief propagation techniques for improved inference in complex networks.