[论文解读] Copula Flows for Synthetic Data Generation

Sanket Kamthe, Samuel Assefa+1 more|arXiv (Cornell University)|Jan 3, 2021

Generative Adversarial Networks and Image Synthesis参考文献 41被引用 38

一句话总结

本文介绍 Copula Flows，一种基于正则化流的 Copula 模型，用于学习混合型 Copula 并生成高保真合成数据，在密度估计和下游 ML 任务上具有强大性能。它通过分布变换和边缘/ Copula 流来处理离散与连续变量。

ABSTRACT

The ability to generate high-fidelity synthetic data is crucial when available (real) data is limited or where privacy and data protection standards allow only for limited use of the given data, e.g., in medical and financial data-sets. Current state-of-the-art methods for synthetic data generation are based on generative models, such as Generative Adversarial Networks (GANs). Even though GANs have achieved remarkable results in synthetic data generation, they are often challenging to interpret.Furthermore, GAN-based methods can suffer when used with mixed real and categorical variables.Moreover, loss function (discriminator loss) design itself is problem specific, i.e., the generative model may not be useful for tasks it was not explicitly trained for. In this paper, we propose to use a probabilistic model as a synthetic data generator. Learning the probabilistic model for the data is equivalent to estimating the density of the data. Based on the copula theory, we divide the density estimation task into two parts, i.e., estimating univariate marginals and estimating the multivariate copula density over the univariate marginals. We use normalising flows to learn both the copula density and univariate marginals. We benchmark our method on both simulated and real data-sets in terms of density estimation as well as the ability to generate high-fidelity synthetic data

研究动机与目标

• Motivate synthetic data generation with privacy and data-limited settings.
• Propose an interpretable, flexible probabilistic generator based on copulas and normalising flows.
• Enable mixed data types (continuous and discrete) within a unified framework.
• Demonstrate density estimation accuracy and usefulness of synthetic data for ML tasks.

提出的方法

• Represent joint density as f_X(X) = c_X(F_X1,...,F_Xd) * ∏ f_Xk (Equation 5).
• Train marginal flows F_Xk (univariate) with monotone neural spline flows (NSF).
• Train copula flow C_X as an autoregressive/conditional flow using neural splines (conditional CDFs).
• Use distributional transform to handle discrete/mixed marginals for copula learning (Section 4.2).
• Generate data via inverse transform sampling: U ~ Uniform(0,1) → C_X^{-1}(U) → F_Xk^{-1}(·) to obtain X.
• Maximise total log-likelihood L = L_{C_X} + L_{F} and train marginals before the copula (Section 3).

实验结果

研究问题

• RQ1 Can a copula-based flow model learn complex, mixed-type joint distributions (including discrete variables) without explicit pair-copula structures?
• RQ2 How well does the proposed Copula Flow perform in density estimation compared to state-of-the-art neural density estimators?
• RQ3 Can Copula Flows generate synthetic data that preserve relationships between variables and be useful for downstream ML tasks?
• RQ4 How can discrete data be effectively incorporated via distributional transforms within a normalising-flow copula framework?

主要发现

模型	Power	Gas	Hepmass	Miniboone
FFJORD	0.46±0.01	8.59±0.12	-14.92±0.08	-10.43±0.04
RQ-NSF (AR) [Durkan et al., 2019]	0.66±0.01	13.09±0.02	-14.01±0.03	-9.22±0.48
MAF [Papamakarios et al., 2017]	0.45±0.01	12.35±0.02	-17.03±0.02	-10.92±0.46
Marginal Flows 𝔽	-0.80±0.02	-6.67±0.02	-26.42±0.05	-53.17±0.06
Copula Flow 𝒞	1.39±0.03	15.6±0.67	5.4±0.10	37.77±0.21
Joint Model 𝒞+𝔽	0.59±0.03	8.05±0.68	-19.6±0.12	-14.83±0.21

• Copula Flows can learn copulas over mixed discrete and continuous marginals using distributional transforms (Theorem 4.2: universal density approximator).
• The model achieves competitive density estimation performance close to state-of-the-art neural estimators on benchmark datasets (Table 1).
• Synthetic data generated by Copula Flows yield ML performance (classification/regression) close to real data and competitive with leading methods (Table 2).
• Discrete marginals are handled via quantised distributions and a stochastic distributional transform, enabling continuous copula learning (Figures and discussion in Section 4).
• Copula Flow can model complex joint structures (e.g., “2 rings”) that standard bivariate copulas struggle to capture (Figure 1).
• Copula Flow supports a fully synthetic data pipeline with potential differential privacy extensions (Conclusion and Broader Impact).