[Paper Review] Estimating Transfer Entropy via Copula Entropy

Jian Ma|arXiv (Cornell University)|Oct 10, 2019

Statistical Methods and Inference47 references15 citations

TL;DR

This paper proposes a non-parametric method to estimate Transfer Entropy (TE) using Copula Entropy (CE), proving TE can be represented solely through CE. Applied to Beijing PM2.5 data, the method effectively infers causal relationships among meteorological factors and air pollution, outperforming Gaussian-assumption-based approaches by avoiding distributional constraints and revealing a 9-hour lagged causal effect of weather on PM2.5.

ABSTRACT

Causal discovery is a fundamental problem in statistics and has wide applications in different fields. Transfer Entropy (TE) is a important notion defined for measuring causality, which is essentially conditional Mutual Information (MI). Copula Entropy (CE) is a theory on measurement of statistical independence and is equivalent to MI. In this paper, we prove that TE can be represented with only CE and then propose a non-parametric method for estimating TE via CE. The proposed method was applied to analyze the Beijing PM2.5 data in the experiments. Experimental results show that the proposed method can infer causality relationships from data effectively and hence help to understand the data better.

Motivation & Objective

• To address the challenge of reliably estimating Transfer Entropy (TE), a key measure of causality in time series, which is notoriously difficult to estimate due to high-dimensional conditional density estimation.
• To establish a theoretical link between TE and Copula Entropy (CE), proving TE can be represented using only CE, thereby enabling a new estimation framework.
• To develop a non-parametric, model-free method for TE estimation that avoids parametric assumptions and improves robustness in non-Gaussian, nonlinear systems.
• To validate the method on real-world environmental data, specifically Beijing's PM2.5 and meteorological time series, to uncover causal relationships in air quality dynamics.

Proposed method

• Theoretical proof is provided that Transfer Entropy (TE) can be expressed exclusively in terms of Copula Entropy (CE), eliminating the need for direct conditional density estimation.
• The method estimates TE via two steps: first estimating CE for the joint and conditional distributions using non-parametric kernel density estimation, then applying the derived CE-based TE representation.
• The approach leverages the equivalence between CE and Mutual Information (MI), allowing CE to serve as a robust, invariant measure of statistical dependence under monotonic transformations.
• The method is applied to observational time series data, using empirical copula density estimation to compute CE, followed by transformation into TE using the derived analytical formula.
• The estimation process is validated under the Markovian and stationarity assumptions, which are justified by empirical analysis of time-lagged dependencies in the data.

Experimental results

Research questions

• RQ1 Can Transfer Entropy be represented solely using Copula Entropy, thereby enabling a new non-parametric estimation framework?
• RQ2 How does the proposed CE-based TE estimation method perform in identifying causal relationships in real, nonlinear, non-Gaussian time series such as air quality data?
• RQ3 What is the temporal lag structure of causal influences between meteorological factors and PM2.5 levels in Beijing?
• RQ4 How does the proposed method compare to existing TE estimation techniques that rely on Gaussian assumptions or parametric models?

Key findings

• The proposed method successfully infers that meteorological factors such as dew point and pressure causally affect PM2.5 levels with a peak causal influence at approximately 9 hours lag.
• Wind speed was found to causally influence temperature and pressure with shorter delays—3 and 5 hours respectively—indicating faster atmospheric feedback mechanisms.
• The method revealed that causal effects accumulate over time, with a two-phase pattern: an initial sharp rise in TE over the first 9 hours, followed by a flatter increase, suggesting a cumulative atmospheric process.
• Unlike correlation, which showed no increase between temperature and PM2.5, TE clearly increased, demonstrating that association does not imply causation and highlighting the method’s ability to detect true causal relationships.
• The method outperformed Gaussian-assumption-based TE estimators by avoiding model misspecification, particularly in non-Gaussian, nonlinear atmospheric systems.