[Paper Review] A General Framework for Cutting Feedback within Modularised Bayesian Inference

Yang Liu, Robert J. B. Goudie|ArXiv.org|Nov 7, 2022

Bayesian Modeling and Causal Inference4 citations

TL;DR

This paper introduces a formal framework for cutting feedback in modularized Bayesian inference across arbitrary directed acyclic graph (DAG) structures. It defines 'self-contained Bayesian modules' based on observable variables, establishes a parent-child ordering for module inference, and derives cut distributions that minimize KL divergence while excluding influence from suspect modules—extending two-module cut inference to general multiple-module cases via sequential splitting.

ABSTRACT

Standard Bayesian inference can build models that combine information from various sources, but this inference may not be reliable if components of a model are misspecified. Cut inference, as a particular type of modularized Bayesian inference, is an alternative which splits a model into modules and cuts the feedback from the suspect module. Previous studies have focused on a two-module case, but a more general definition of a "module" remains unclear. We present a formal definition of a "module" and discuss its properties. We formulate methods for identifying modules; determining the order of modules; and building the cut distribution that should be used for cut inference within an arbitrary directed acyclic graph structure. We justify the cut distribution by showing that it not only cuts the feedback but also is the best approximation satisfying this condition to the joint distribution in the Kullback-Leibler divergence. We also extend cut inference for the two-module case to a general multiple-module case via a sequential splitting technique and demonstrate this via illustrative applications.

Motivation & Objective

• Address the lack of a general definition of 'module' in modularized Bayesian inference beyond two-module cases.
• Provide a formal definition of a 'self-contained Bayesian module' based on observable random variables rather than parameters.
• Establish a systematic method to identify module structure, determine inference order (parent-child ordering), and construct cut distributions in arbitrary DAGs.
• Extend two-module cut inference to general multiple-module settings using a sequential splitting technique.
• Justify the cut distribution as the optimal approximation to the joint posterior under KL divergence, while excluding feedback from suspect modules.

Proposed method

• Define a 'self-contained Bayesian module' as a set of variables associated with a subset of observable random variables that allow standard Bayesian inference.
• Use the DAG structure to derive a parent-child ordering of modules, ensuring inference flows from ancestors to descendants.
• Formulate the cut distribution as the joint distribution conditioned on cutting feedback from descendant modules, ensuring no influence from suspect components.
• Apply a sequential splitting technique to extend two-module cut inference to multiple modules by iteratively isolating and cutting feedback from each module in order.
• Prove that the cut distribution is the best approximation to the true joint posterior under Kullback-Leibler divergence while satisfying the constraint of zero feedback from suspect modules.
• Demonstrate the method through illustrative applications, showing how it maintains estimation reliability under model misspecification.

Figure 1: Self-contained Bayesian module. Squares denote observable random variables and circles denote parameters. The dashed part is a minimally self-contained Bayesian module (see Definition 2 ).

Experimental results

Research questions

• RQ1 How can a general, formal definition of a 'module' be established in modularized Bayesian inference beyond the two-module case?
• RQ2 What criteria determine the correct order of inference among multiple modules in a DAG structure?
• RQ3 How can the cut distribution be systematically derived for arbitrary DAGs with multiple modules?
• RQ4 What is the theoretical justification for using the cut distribution as the optimal approximation under KL divergence?
• RQ5 How can two-module cut inference be extended to a general multiple-module setting in a principled and scalable way?

Key findings

• The paper formally defines a 'self-contained Bayesian module' based on observable variables, providing a principled foundation for module identification in complex models.
• A parent-child ordering of modules is derived directly from the DAG structure, enabling systematic inference flow from ancestor to descendant modules.
• The cut distribution is mathematically justified as the minimum-KL divergence approximation to the joint posterior that excludes feedback from suspect modules.
• The sequential splitting technique enables a scalable and general extension of cut inference from two modules to multiple modules in arbitrary DAGs.
• The method ensures robust inference by isolating misspecified components, thereby preventing their influence from distorting reliable inferences in other modules.
• Numerical simulations in the supplementary materials show that unlike standard cut inference, the proposed method avoids systematic bias in estimation when feedback is cut, though bias may still exist depending on model structure.

Figure 2: Partitioning the observable random variable $X$ . First $X$ is partitioned into two disjoint group $X_{A}^{\ast}$ and $X_{B}^{\ast}$ . Then $X_{A}^{\ast}$ is enlarged to form $\Psi_{A}=(X_{A},\Theta_{A})$ following Rule 1 , and similarly for $X_{B}^{\ast}$ to form $\Psi_{B}=(X_{B},\Theta_{