[Paper Review] Toward a generic representation of random variables for machine learning

Gautier Marti, Philippe Very+1 more|arXiv (Cornell University)|Jun 2, 2015

Time Series Analysis and Forecasting26 references3 citations

TL;DR

This paper proposes a generic, non-parametric representation of random variables that decouples dependence and distribution while preserving all information, enabling a novel distance metric (GNPR) that jointly captures both aspects. Applied to i.i.d. time series, particularly financial credit default swap returns, the method yields significantly more stable and homogeneous clustering than standard L2 distance on returns, with an adjusted Rand index of 0.85 vs. 0.64.

ABSTRACT

This paper presents a pre-processing and a distance which improve the performance of machine learning algorithms working on independent and identically distributed stochastic processes. We introduce a novel non-parametric approach to represent random variables which splits apart dependency and distribution without losing any information. We also propound an associated metric leveraging this representation and its statistical estimate. Besides experiments on synthetic datasets, the benefits of our contribution is illustrated through the example of clustering financial time series, for instance prices from the credit default swaps market. Results are available on the website www.datagrapple.com and an IPython Notebook tutorial is available at www.datagrapple.com/Tech for reproducible research.

Motivation & Objective

• To develop a machine learning-compatible pre-processing and distance metric for i.i.d. stochastic processes that jointly model dependence and distribution.
• To address the instability of clustering results in financial time series, particularly under resampling or perturbations.
• To provide a mathematically grounded, non-parametric alternative to existing distributional or dependence-only distances.
• To demonstrate improved clustering performance on synthetic data and real-world financial time series, such as credit default swap prices.
• To enable reproducible research through open-sourced code and experimental results on www.datagrapple.com

Proposed method

• Proposes a non-parametric representation of random variables that splits dependence and marginal distribution without information loss, inspired by Sklar’s theorem in copula theory.
• Introduces a generalized non-parametric representation (GNPR) distance that combines distributional and dependence information via a parameter θ ∈ [0,1], interpolating between pure distribution (θ=0) and pure dependence (θ=1).
• Employs empirical estimation of the GNPR distance using rank-based and empirical cumulative distribution function techniques to handle real data.
• Applies the GNPR distance to time series of price increments, assuming i.i.d. increments under the random walk hypothesis, particularly for credit default swaps.
• Uses standard clustering algorithms (e.g., Ward’s method, k-means++) with GNPR distance to evaluate partition stability and homogeneity.
• Validates results via resampling (odd/even trading days) and compares stability using the adjusted Rand index (ARI).

Experimental results

Research questions

• RQ1 Can a unified representation of random variables that separates dependence and distribution improve clustering performance on i.i.d. time series?
• RQ2 How does the GNPR distance, which combines distribution and dependence, compare to standard L2 distance on returns in terms of clustering stability?
• RQ3 Does the proposed method yield more homogeneous and stable clusters on financial time series with heavy-tailed distributions, such as credit default swaps?
• RQ4 To what extent does the parameter θ in GNPR allow tuning between distributional and dependence-based clustering?
• RQ5 Is the GNPR-based clustering robust to small perturbations, such as resampling of trading days?

Key findings

• The GNPR distance with θ=0.5, balancing distribution and dependence, achieved an adjusted Rand index (ARI) of 0.85 on CDS data, significantly outperforming the L2 distance on returns, which achieved only 0.64.
• Clustering using GNPR θ=0.5 produced more stable partitions between odd and even trading days, with minimal cluster reassignment, indicating robustness to perturbations.
• The four clusters identified via GNPR θ=0.0 precisely matched the multi-modal empirical distribution of standard deviations in the CDS dataset, confirming sensitivity to distributional differences.
• The GNPR θ=1.0 distance matrix revealed a hierarchical correlation structure in CDS returns, with groupings by region (US, Europe, Japan), credit quality (investment grade vs. high yield), and industrial sector.
• The method demonstrated superior performance on synthetic data by capturing both distributional and dependence structures, leading to finer and more meaningful partitions.
• The results were reproducible and publicly available via www.datagrapple.com, including full code and experimental details.