[Paper Review] Generation and Simulation of Synthetic Datasets with Copulas

Régis Houssou, Mihai-Cezar Augustin+3 more|arXiv (Cornell University)|Mar 30, 2022

Neural Networks and Applications4 citations

TL;DR

This paper proposes a copula-based method for generating synthetic datasets that preserve both marginal distributions and complex dependency structures in multivariate data, including mixed numeric and categorical variables. By transforming categorical features into ranks, applying copula modeling to capture dependencies, and reversing the transformation, the approach outperforms SMOTE and autoencoders in maintaining joint distributions and correlation structures, as validated on real-world datasets.

ABSTRACT

This paper proposes a new method to generate synthetic data sets based on copula models. Our goal is to produce surrogate data resembling real data in terms of marginal and joint distributions. We present a complete and reliable algorithm for generating a synthetic data set comprising numeric or categorical variables. Applying our methodology to two datasets shows better performance compared to other methods such as SMOTE and autoencoders.

Motivation & Objective

• To address data scarcity and privacy constraints by generating high-fidelity synthetic datasets that mimic real data distributions.
• To develop a robust, end-to-end algorithm for generating synthetic data with both numeric and categorical variables.
• To preserve complex dependency structures—especially between categorical features—more accurately than existing methods like SMOTE and autoencoders.
• To enable secure, private, and efficient data sharing across government, academia, and industry.
• To provide a statistically principled alternative to deep learning-based generative models for synthetic data, especially in low-data regimes.

Proposed method

• Use of copulas to model the dependence structure between variables by separating marginal distributions from joint dependence.
• Application of the probability integral transform to convert original data into uniform variates on [0,1] for copula modeling.
• Transformation of categorical variables into ranks to enable copula-based modeling, followed by inverse transformation to recover categorical labels.
• Simulation of synthetic data using the inverse probability integral transform on the fitted copula model.
• Use of Kendall’s tau and chi-squared tests to evaluate the preservation of dependence and association in synthetic data.
• Comparison of the copula-based method with SMOTE and autoencoder-based approaches using cross-tabulation and correlation analysis.

Experimental results

Research questions

• RQ1 Can copula-based synthetic data generation preserve the joint distribution and dependency structure of real datasets with mixed numeric and categorical variables?
• RQ2 How does the copula method compare to SMOTE and autoencoders in maintaining the association between categorical features in synthetic datasets?
• RQ3 To what extent does the copula method preserve the correlation structure between continuous variables after transformation of categorical features?
• RQ4 Does the proposed method effectively handle high-dimensional categorical data with sparse associations?
• RQ5 Can the copula framework be reliably applied to real-world datasets with limited sample sizes and complex dependency patterns?

Key findings

• The copula-based method outperformed SMOTE and autoencoders in preserving the cross-tabulation structure between categorical features, especially for highly associated levels.
• SMOTE generated noisy distributions with reduced association between categorical variables, as indicated by a significant p-value in the chi-squared test for independence.
• Autoencoders produced highly skewed distributions, focusing on a single currency level and distorting the original association patterns, with independence not rejected in the chi-squared test.
• Kendall’s tau correlation between continuous features in the synthetic data showed that the copula method closely matched the original data, while autoencoders introduced spurious negative correlations.
• The copula method maintained the marginal distributions and dependence structure more faithfully than both SMOTE and autoencoders, as confirmed by visual and statistical analysis.
• The approach is effective even on small datasets (e.g., ~1,000 records), though scalability to larger, high-dimensional categorical datasets remains an open challenge for future work.