[Paper Review] Exploring and measuring non-linear correlations: Copulas, Lightspeed Transportation and Clustering

Gautier Marti, Sébastien Andler+2 more|arXiv (Cornell University)|Oct 30, 2016

Mental Health Research Topics3 citations

TL;DR

This paper proposes a novel methodology for exploring, measuring, and targeting specific non-linear dependence patterns in multivariate data using copulas, optimal transport, and clustering. By leveraging these tools, the authors introduce a parameterized dependence coefficient (TFDC) that can detect or ignore specific correlation structures, outperforming general-purpose measures in detecting targeted relationships while revealing complex, non-Gaussian dependencies in financial time series.

ABSTRACT

We propose a methodology to explore and measure the pairwise correlations that exist between variables in a dataset. The methodology leverages copulas for encoding dependence between two variables, state-of-the-art optimal transport for providing a relevant geometry to the copulas, and clustering for summarizing the main dependence patterns found between the variables. Some of the clusters centers can be used to parameterize a novel dependence coefficient which can target or forget specific dependence patterns. Finally, we illustrate and benchmark the methodology on several datasets. Code and numerical experiments are available online for reproducible research.

Motivation & Objective

• To address the limitations of traditional correlation measures like Pearson’s, which fail to capture non-linear, non-monotonic, or tail-dependent relationships.
• To develop a method that allows users to target or forget specific dependence patterns—such as upper/lower tail dependence or specific functional forms—rather than detecting all forms of dependence equally.
• To improve correlation-based feature selection by encoding expert knowledge or data-driven patterns into a measurable, interpretable dependence coefficient.
• To provide a practical, computationally feasible framework for dependence analysis using recent advances in optimal transport and clustering of copulas.

Proposed method

• Model pairwise dependence between variables using empirical copulas derived from probability integral transforms of marginal distributions.
• Apply optimal transport (Lightspeed Transportation) to define a meaningful geometry on the space of copulas, enabling distance computation between dependence structures.
• Use clustering (e.g., k-means) on the copula space to identify representative dependence patterns, with cluster centers serving as targets or forget-structures.
• Define a novel dependence coefficient, TFDC (Targeted and Forgetting Dependence Coefficient), parameterized by user-specified target and forget copulas, which measures similarity between empirical copulas and the target.
• Utilize the Wasserstein distance (from optimal transport) as the underlying metric to compare copulas and compute the TFDC score.
• Integrate the framework into a query-driven system where users can pose complex dependence queries (e.g., 'find pairs with upper tail dependence but no linear correlation') and receive data-driven answers.

Experimental results

Research questions

• RQ1 How can we measure non-linear, non-monotonic, and tail-dependent correlations more effectively than with Pearson or mutual information?
• RQ2 Can we design a dependence coefficient that selectively detects or ignores specific types of dependence, such as upper tail dependence in credit default swaps or lower tail dependence in equities?
• RQ3 How do copula-based representations and optimal transport enable a more interpretable and geometrically meaningful analysis of dependence in high-dimensional data?
• RQ4 To what extent can clustering of copulas reveal and summarize the dominant dependence patterns in real-world datasets like financial time series?
• RQ5 Can a query-driven framework using copulas and optimal transport improve feature selection by focusing on domain-relevant dependence structures?

Key findings

• The proposed TFDC method achieves significantly higher statistical power than general-purpose measures like MIC and dCor when detecting specific, targeted dependence patterns, especially under increasing noise levels.
• Empirical copulas of financial assets (e.g., stocks, CDS, FX rates) reveal strong deviations from the Gaussian copula, with distinct tail dependence structures: stocks show lower tail dependence, CDS exhibit upper tail dependence, and FX rates display heterogeneous dependence patterns.
• Clustering of copulas successfully identifies meaningful dependence patterns, such as comonotonicity, countermonotonicity, and independence, which align with domain knowledge in finance.
• The method enables answering complex dependence queries—such as identifying pairs with positive correlation for small changes but no overall correlation—by using handcrafted or data-driven target copulas.
• The framework outperforms standard measures in detecting structured, non-linear relationships, confirming that equitability is not always desirable when domain-specific dependence is the goal.
• The use of optimal transport (Lightspeed Transportation) enables efficient computation of copula distances, making the method scalable and practical for real-world datasets despite prior computational challenges.