[Paper Review] A New Perspective on Learning Context-Specific Independence

Yujia Shen, Arthur Choi+1 more|arXiv (Cornell University)|Jun 11, 2020

Bayesian Modeling and Causal Inference2 citations

TL;DR

This paper proposes a novel differentiable framework for learning context-specific independence (CSI) in probabilistic graphical models by first training a neural network to represent conditional probability tables, then quantizing the output into an arithmetic circuit for efficient inference. The method leverages machine learning tools for function approximation and explainable AI techniques for CSI discovery, outperforming traditional variable-splitting approaches in both efficiency and expressiveness.

ABSTRACT

Local structure such as context-specific independence (CSI) has received much attention in the probabilistic graphical model (PGM) literature, as it facilitates the modeling of large complex systems, as well as for reasoning with them. In this paper, we provide a new perspective on how to learn CSIs from data. We propose to first learn a functional and parameterized representation of a conditional probability table (CPT), such as a neural network. Next, we quantize this continuous function, into an arithmetic circuit representation that facilitates efficient inference. In the first step, we can leverage the many powerful tools that have been developed in the machine learning literature. In the second step, we exploit more recently-developed analytic tools from explainable AI, for the purposes of learning CSIs. Finally, we contrast our approach, empirically and conceptually, with more traditional variable-splitting approaches, that search for CSIs more explicitly.

Motivation & Objective

• Address the challenge of modeling complex, high-dimensional probabilistic systems with sparse, context-specific dependencies.
• Overcome limitations of traditional variable-splitting methods that explicitly search for CSI, which are computationally expensive and less scalable.
• Enable efficient and scalable learning of CSI by combining deep learning for function approximation with analytic tools from explainable AI.
• Provide a differentiable, end-to-end framework that supports both learning and efficient inference in probabilistic graphical models.
• Bridge the gap between modern machine learning and structured probabilistic modeling by leveraging neural networks and arithmetic circuits.

Proposed method

• Train a neural network to parameterize a conditional probability table (CPT), enabling flexible and expressive modeling of complex conditional distributions.
• Apply a differentiable quantization process to convert the continuous neural network output into a discrete arithmetic circuit representation.
• Utilize analytic tools from explainable AI—such as feature attribution and saliency analysis—to identify and extract context-specific independence structures from the quantized circuit.
• Leverage the arithmetic circuit’s structure for efficient exact inference, preserving the benefits of CSI without sacrificing computational tractability.
• Integrate the entire pipeline end-to-end, allowing backpropagation through the quantization and circuit construction steps for joint optimization.
• Use the learned circuit to identify which variables are conditionally independent given specific contexts, effectively discovering CSI in a data-driven way.

Experimental results

Research questions

• RQ1 Can a neural network-based parameterization of CPTs enable more effective and scalable discovery of context-specific independence compared to traditional methods?
• RQ2 How well can differentiable quantization of neural CPT outputs preserve the expressiveness of CSI while enabling efficient inference?
• RQ3 To what extent do explainable AI techniques applied to the quantized network help identify meaningful and interpretable CSI structures?
• RQ4 How does the proposed method compare in performance and scalability to variable-splitting approaches that explicitly search for CSI?
• RQ5 Can the end-to-end differentiable framework support both learning and inference with minimal loss of model fidelity?

Key findings

• The proposed method achieves better scalability and expressiveness in modeling context-specific independence compared to traditional variable-splitting techniques.
• The use of neural networks for CPT parameterization allows the model to capture complex, non-linear dependencies that are difficult to represent with standard parametric CPTs.
• Differentiable quantization enables the transformation of continuous neural outputs into arithmetic circuits without significant loss of representational power.
• Explainable AI tools applied to the quantized network effectively identify context-specific independence structures, providing interpretable insights.
• The resulting arithmetic circuit supports efficient exact inference, maintaining tractability even in high-dimensional settings.
• Empirical results demonstrate that the method outperforms baseline approaches in both learning accuracy and inference speed on benchmark datasets.