[Paper Review] Copula Flows for Synthetic Data Generation

Sanket Kamthe, Samuel Assefa+1 more|arXiv (Cornell University)|Jan 3, 2021

Generative Adversarial Networks and Image SynthesisComputer Science41 references38 citations

TL;DR

This paper introduces Copula Flows, a normalising-flow based copula model to learn mixed-type copulas and generate high-fidelity synthetic data, with strong performance on density estimation and downstream ML tasks. It handles discrete and continuous variables via distributional transform and marginal/copula flows.

ABSTRACT

The ability to generate high-fidelity synthetic data is crucial when available (real) data is limited or where privacy and data protection standards allow only for limited use of the given data, e.g., in medical and financial data-sets. Current state-of-the-art methods for synthetic data generation are based on generative models, such as Generative Adversarial Networks (GANs). Even though GANs have achieved remarkable results in synthetic data generation, they are often challenging to interpret.Furthermore, GAN-based methods can suffer when used with mixed real and categorical variables.Moreover, loss function (discriminator loss) design itself is problem specific, i.e., the generative model may not be useful for tasks it was not explicitly trained for. In this paper, we propose to use a probabilistic model as a synthetic data generator. Learning the probabilistic model for the data is equivalent to estimating the density of the data. Based on the copula theory, we divide the density estimation task into two parts, i.e., estimating univariate marginals and estimating the multivariate copula density over the univariate marginals. We use normalising flows to learn both the copula density and univariate marginals. We benchmark our method on both simulated and real data-sets in terms of density estimation as well as the ability to generate high-fidelity synthetic data

Motivation & Objective

• Motivate synthetic data generation with privacy and data-limited settings.
• Propose an interpretable, flexible probabilistic generator based on copulas and normalising flows.
• Enable mixed data types (continuous and discrete) within a unified framework.
• Demonstrate density estimation accuracy and usefulness of synthetic data for ML tasks.

Proposed method

• Represent joint density as f_X(X) = c_X(F_X1,...,F_Xd) * ∏ f_Xk (Equation 5).
• Train marginal flows F_Xk (univariate) with monotone neural spline flows (NSF).
• Train copula flow C_X as an autoregressive/conditional flow using neural splines (conditional CDFs).
• Use distributional transform to handle discrete/mixed marginals for copula learning (Section 4.2).
• Generate data via inverse transform sampling: U ~ Uniform(0,1) → C_X^{-1}(U) → F_Xk^{-1}(·) to obtain X.
• Maximise total log-likelihood L = L_{C_X} + L_{F} and train marginals before the copula (Section 3).

Experimental results

Research questions

• RQ1 Can a copula-based flow model learn complex, mixed-type joint distributions (including discrete variables) without explicit pair-copula structures?
• RQ2 How well does the proposed Copula Flow perform in density estimation compared to state-of-the-art neural density estimators?
• RQ3 Can Copula Flows generate synthetic data that preserve relationships between variables and be useful for downstream ML tasks?
• RQ4 How can discrete data be effectively incorporated via distributional transforms within a normalising-flow copula framework?

Key findings

• Copula Flows can learn copulas over mixed discrete and continuous marginals using distributional transforms (Theorem 4.2: universal density approximator).
• The model achieves competitive density estimation performance close to state-of-the-art neural estimators on benchmark datasets (Table 1).
• Synthetic data generated by Copula Flows yield ML performance (classification/regression) close to real data and competitive with leading methods (Table 2).
• Discrete marginals are handled via quantised distributions and a stochastic distributional transform, enabling continuous copula learning (Figures and discussion in Section 4).
• Copula Flow can model complex joint structures (e.g., “2 rings”) that standard bivariate copulas struggle to capture (Figure 1).
• Copula Flow supports a fully synthetic data pipeline with potential differential privacy extensions (Conclusion and Broader Impact).