[Paper Review] Bounded Degree Approximations of Stochastic Networks

Christopher J. Quinn, Ali Pınar+1 more|arXiv (Cornell University)|Jun 15, 2015

Bayesian Modeling and Causal Inference32 references3 citations

TL;DR

This paper proposes efficient algorithms to compute optimal and near-optimal bounded in-degree approximations of directed information graphs, minimizing Kullback-Leibler divergence. It introduces novel relaxations of submodularity to enable tractable computation of sparse, connected, and visually concise network topologies that preserve causal dependencies in stochastic processes.

ABSTRACT

We propose algorithms to approximate directed information graphs. Directed information graphs are probabilistic graphical models that depict causal dependencies between stochastic processes in a network. The proposed algorithms identify optimal and near-optimal approximations in terms of Kullback-Leibler divergence. The user-chosen sparsity trades off the quality of the approximation against visual conciseness and computational tractability. One class of approximations contains graphs with specified in-degrees. Another class additionally requires that the graph is connected. For both classes, we propose algorithms to identify the optimal approximations and also near-optimal approximations, using a novel relaxation of submodularity. We also propose algorithms to identify the r-best approximations among these classes, enabling robust decision making.

Motivation & Objective

• To address the challenge of analyzing large-scale stochastic networks where full topologies are computationally intractable and visually overwhelming.
• To identify sparse, connected network approximations that preserve causal dependencies between stochastic processes, measured via directed information.
• To balance model accuracy, computational efficiency, and visual conciseness by enforcing user-defined in-degree bounds and connectivity constraints.
• To provide provably good approximations using novel relaxations of submodularity, enabling tractable optimization.
• To support robust decision-making by computing the r-best approximations within constrained classes.

Proposed method

• Proposes a relaxation of submodularity to enable efficient optimization of directed information graphs under in-degree and connectivity constraints.
• Develops algorithms to compute the optimal bounded in-degree approximation using pairwise directed information values and greedy selection.
• Introduces a method to compute the r-best approximations by iteratively modifying parent sets of the top solution, leveraging independence of parent set selection.
• Employs a lexicographic indexing scheme for parent sets using combinatorial enumeration to efficiently track and avoid redundant solutions in the search space.
• Uses a priority queue data structure to manage candidate approximations and ensure efficient exploration of high-quality solutions.
• Applies the MWDST (Maximum Weight Directed Spanning Tree) algorithm as a baseline and extends it to higher in-degree and connected graph classes.

Experimental results

Research questions

• RQ1 How can we compute the optimal bounded in-degree approximation of a directed information graph under Kullback-Leibler divergence?
• RQ2 Can we efficiently compute near-optimal approximations when exact optimization is intractable, especially under connectivity and in-degree constraints?
• RQ3 What is the theoretical performance guarantee of the proposed approximation algorithms in terms of submodular relaxation and directed information maximization?
• RQ4 How can we systematically generate the r-best approximations to support robust network inference and decision-making?
• RQ5 What combinatorial indexing method enables efficient enumeration and pruning of redundant or previously seen approximations?

Key findings

• The proposed algorithm computes the optimal bounded in-degree approximation using only pairwise directed information values, avoiding the need for the full joint distribution.
• The method guarantees a (1−e−L/∑i=0K−1αi)-approximation ratio for the connected bounded in-degree problem, leveraging a novel relaxation of submodularity.
• The r-best approximations are computed by modifying one parent set at a time from the optimal solution, ensuring all high-quality candidates are explored.
• A combinatorial indexing scheme enables efficient representation and tracking of parent sets, reducing redundancy in the search space.
• The algorithms are computationally efficient and scalable, supporting applications in neuroscience, gene regulatory networks, and social networks with thousands of nodes.
• Theoretical analysis confirms that parent sets can be selected independently under the given assumptions, enabling modular optimization.