[Paper Review] The Randomized Dependence Coefficient

David López-Paz, Philipp Hennig+1 more|arXiv (Cornell University)|Apr 29, 2013

Neural Networks and Applications24 references105 citations

TL;DR

The paper introduces the Randomized Dependence Coefficient (RDC), a scalable, non-linear dependence measure between multivariate random variables that estimates the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient via random non-linear projections of empirical copula transformations. RDC achieves $O(n\log n)$ computational complexity, is invariant to monotonic marginal transformations, and outperforms existing methods in speed and performance on non-functional dependence patterns.

ABSTRACT

We introduce the Randomized Dependence Coefficient (RDC), a measure of non-linear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. RDC is defined in terms of correlation of random non-linear copula projections; it is invariant with respect to marginal distribution transformations, has low computational cost and is easy to implement: just five lines of R code, included at the end of the paper.

Motivation & Objective

• To develop a computationally efficient, non-linear dependence measure that satisfies Rényi's seven fundamental properties of dependence.
• To address the intractability of the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient (HGR) by proposing a practical estimator.
• To create a method that is invariant to monotonic marginal transformations and scalable to high-dimensional and large-sample data.
• To provide a lightweight alternative to existing non-linear dependence measures like dCor, MMD, MIC, and KCCA, which suffer from high computational costs or implementation complexity.

Proposed method

• Apply empirical copula transformation to input data to remove marginal distribution effects, using the probability integral transform.
• Generate $k$ random non-linear projections using sine and cosine basis functions applied to linear combinations of the copula-transformed data.
• Compute the largest canonical correlation between the projected representations of the two random variables.
• Use random projections to approximate the supremum in the HGR coefficient, avoiding the need for exhaustive optimization over infinite-dimensional function spaces.
• Leverage the invariance of canonical correlation under linear transformations and the stability of random projections to ensure robustness.
• Implement the method in just five lines of R code, enabling easy integration into data analysis pipelines.

Experimental results

Research questions

• RQ1 Can a computationally efficient estimator be developed to approximate the intractable Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient?
• RQ2 Does a randomized projection-based approach preserve the theoretical properties of the HGR coefficient while enabling scalability?
• RQ3 How does RDC compare to existing non-linear dependence measures (e.g., dCor, MIC, MMD) in terms of computational efficiency and detection power on non-functional dependence patterns?
• RQ4 To what extent does RDC maintain invariance under monotonic marginal transformations in real-world and synthetic data?
• RQ5 Can RDC effectively identify complex, non-linear dependencies in high-dimensional feature selection tasks?

Key findings

• RDC achieves $O(n\log n)$ computational complexity, significantly outperforming quadratic-cost methods like dCor, MMD, CMMD, and MIC on large datasets.
• On synthetic data, RDC shows strong power in detecting non-functional dependencies (e.g., circular, sinusoidal mixtures), while performing competitively in linear and step-function cases.
• In feature selection tasks across eight real-world datasets, RDC achieved the best or near-best performance in minimizing normalized mean squared error with significantly lower runtime than dCor, MMD, and CMMD.
• RDC scores close to 1.0 on all non-independent association patterns and near 0.0 on independent data, demonstrating strong sensitivity and specificity.
• The method maintains invariance to monotonic marginal transformations, as required by Rényi’s axioms, and performs well under increasing additive noise.
• The RDC implementation requires only five lines of R code, making it highly accessible and easy to integrate into existing data science workflows.