[Paper Review] DYNOTEARS: Structure Learning from Time-Series Data

Roxana Pamfil, Nisara Sriwattanaworachai+5 more|arXiv (Cornell University)|Feb 2, 2020

Bayesian Modeling and Causal Inference54 references4 citations

TL;DR

DYNOTEARS proposes a score-based method for learning dynamic Bayesian networks (DBNs) that jointly estimates contemporaneous (intra-slice) and time-lagged (inter-slice) relationships in time-series data. By leveraging a smooth algebraic characterization of acyclicity, it formulates structure learning as a constrained optimization problem solvable via standard second-order optimization, achieving state-of-the-art performance in high-dimensional settings on both simulated and real-world datasets from finance and molecular biology.

ABSTRACT

We revisit the structure learning problem for dynamic Bayesian networks and propose a method that simultaneously estimates contemporaneous (intra-slice) and time-lagged (inter-slice) relationships between variables in a time-series. Our approach is score-based, and revolves around minimizing a penalized loss subject to an acyclicity constraint. To solve this problem, we leverage a recent algebraic result characterizing the acyclicity constraint as a smooth equality constraint. The resulting algorithm, which we call DYNOTEARS, outperforms other methods on simulated data, especially in high-dimensions as the number of variables increases. We also apply this algorithm on real datasets from two different domains, finance and molecular biology, and analyze the resulting output. Compared to state-of-the-art methods for learning dynamic Bayesian networks, our method is both scalable and accurate on real data. The simple formulation and competitive performance of our method make it suitable for a variety of problems where one seeks to learn connections between variables across time.

Motivation & Objective

• To address the challenge of learning dynamic Bayesian networks (DBNs) that capture both instantaneous (intra-slice) and delayed (inter-slice) dependencies in time-series data.
• To develop a scalable and accurate method for DBN structure learning in high-dimensional settings where the number of variables exceeds the number of observations (d > n).
• To overcome limitations of existing two-step or independent estimation approaches that fail to jointly model intra- and inter-slice relationships.
• To provide a plug-and-play, optimization-based framework that is robust to complex graph topologies and does not assume bounded in-degree or treewidth.
• To validate the method on real-world datasets from finance and systems biology, demonstrating practical utility and competitive performance against state-of-the-art methods.

Proposed method

• Formulates DBN structure learning as a penalized score optimization problem with an acyclicity constraint, using a smooth equality constraint derived from Zheng et al. (2018) to represent acyclicity algebraically.
• Employs standard second-order optimization algorithms (e.g., L-BFGS) to solve the constrained optimization problem, enabling efficient and scalable learning.
• Separates the weight matrix into two components: W for contemporaneous (intra-slice) dependencies and A for time-lagged (inter-slice) dependencies, both estimated jointly.
• Combines the W and A matrices via element-wise summation to produce a final weighted adjacency matrix representing the full dynamic dependency structure.
• Applies cross-validation to tune hyperparameters on each dataset, ensuring optimal performance on real-world data.
• Uses a differentiable score function with L1 regularization to promote sparsity and improve generalization in high-dimensional settings.

Experimental results

Research questions

• RQ1 Can a unified, score-based optimization framework jointly learn both contemporaneous and time-lagged dependencies in dynamic Bayesian networks more effectively than existing two-step or independent approaches?
• RQ2 How does the proposed method perform in high-dimensional settings (d > n), particularly when traditional methods fail due to the curse of dimensionality?
• RQ3 To what extent does the use of a smooth algebraic acyclicity constraint improve optimization stability and scalability compared to discrete or combinatorial formulations?
• RQ4 How does DYNOTEARS compare to state-of-the-art methods in terms of accuracy and robustness on real-world time-series data from finance and molecular biology?
• RQ5 Can the method reliably recover known biological and financial dependencies in benchmark datasets like DREAM4 and stock return data?

Key findings

• DYNOTEARS achieved an average AUROC of 0.664 and AUPR of 0.173 on the DREAM4 gene regulatory network dataset, ranking 8th and 4th respectively among 18 methods tested.
• On the DREAM4 dataset, DYNOTEARS outperformed most baselines, including G1DBN, ScanBMA, and VBSSM, in AUPR, indicating strong performance in identifying true regulatory interactions.
• In the financial dataset (d=97), DYNOTEARS demonstrated robustness and scalability, successfully learning inter- and intra-slice dependencies without assuming bounded graph complexity.
• The method achieved competitive performance on both simulated and real-world data, particularly excelling in high-dimensional regimes where d > n.
• The use of a smooth acyclicity constraint enabled stable optimization and better convergence compared to discrete or relaxation-based alternatives.
• The joint estimation of intra- and inter-slice relationships led to more accurate and interpretable network structures than methods that treat them separately.