[Paper Review] Consensus Message Passing for Layered Graphical Models

Varun Jampani, S. M. Ali Eslami+3 more|arXiv (Cornell University)|Oct 27, 2014

Gaussian Processes and Bayesian Inference28 references3 citations

TL;DR

This paper introduces Consensus Message Passing (CMP), a technique that improves inference accuracy in layered graphical models—commonly used in computer vision—by training regressors to predict global variables from contextual messages, thereby guiding message passing toward better solutions. CMP significantly outperforms standard Expectation Propagation and Variational Message Passing, and even surpasses competitive bottom-up models, while maintaining computational efficiency.

ABSTRACT

Generative models provide a powerful framework for probabilistic reasoning. However, in many domains their use has been hampered by the practical difficulties of inference. This is particularly the case in computer vision, where models of the imaging process tend to be large, loopy and layered. For this reason bottom-up conditional models have traditionally dominated in such domains. We find that widely-used, general-purpose message passing inference algorithms such as Expectation Propagation (EP) and Variational Message Passing (VMP) fail on the simplest of vision models. With these models in mind, we introduce a modification to message passing that learns to exploit their layered structure by passing 'consensus' messages that guide inference towards good solutions. Experiments on a variety of problems show that the proposed technique leads to significantly more accurate inference results, not only when compared to standard EP and VMP, but also when compared to competitive bottom-up conditional models.

Motivation & Objective

• To address the failure of general-purpose message passing algorithms like EP and VMP in complex, layered vision models.
• To overcome the challenge of inaccurate inference due to poor estimation of a small set of influential global variables.
• To develop a method that leverages the layered structure of vision models to improve inference accuracy and convergence.
• To preserve the computational efficiency of message passing while significantly enhancing its accuracy.
• To enable the use of full generative models in computer vision by overcoming a key bottleneck in inference.

Proposed method

• Train regressors before inference to predict the values of global variables based on messages from the layer below (contextual messages).
• Use these trained regressors to generate consensus messages during inference, sent to target variables to guide the message passing process.
• Integrate consensus messages into standard message passing frameworks, maintaining consistency with the original graphical model's fixed points.
• Apply the method to models with hierarchical, layered structures where higher-level variables depend on lower-level ones through global parameters.
• Use random forests as the regressor architecture, with features designed to be generic and applicable across diverse vision problems.
• Ensure that the consensus messages are informative early in inference, accelerating convergence and improving accuracy.

Experimental results

Research questions

• RQ1 Can message passing inference in layered graphical models be made more accurate by learning to estimate key global variables early in the process?
• RQ2 Does introducing consensus messages based on contextual information improve convergence and accuracy compared to standard EP and VMP?
• RQ3 Can a hybrid approach combining intuitive (consensus) and rational (message passing) inference steps outperform purely bottom-up models in vision tasks?
• RQ4 To what extent does CMP reduce the number of iterations needed for accurate inference in generative models?
• RQ5 In what ways can the framework be generalized to other domains beyond computer vision?

Key findings

• CMP achieves significantly more accurate inference than standard Expectation Propagation and Variational Message Passing on a range of vision problems.
• CMP outperforms competitive bottom-up conditional models, such as the method by Biswas et al. (2009), in both reflectance and light inference tasks.
• The method improves recognition accuracy in reflectance inference, with gains observed both with and without cast shadows.
• CMP reduces the number of iterations required for convergence by providing accurate initial guidance through consensus messages.
• The use of random forest-based regressors for consensus message prediction leads to robust and generalizable performance across different model structures.
• CMP maintains computational efficiency while enabling the use of full generative models, overcoming a major barrier to their adoption in vision.