[Paper Review] Faking feature importance: A cautionary tale on the use of differentially-private synthetic data

Oscar Giles, Kasra Hosseini+14 more|arXiv (Cornell University)|Mar 2, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper investigates whether differentially private synthetic data can accurately preserve feature importance rankings—critical for early-stage machine learning exploration—across diverse datasets. Using PrivBayes and another DP method, it finds that synthetic data often fails to replicate true feature importance under moderate to high privacy (ε > 0.1), with low and variable similarity scores, especially in complex real-world settings.

ABSTRACT

Synthetic datasets are often presented as a silver-bullet solution to the problem of privacy-preserving data publishing. However, for many applications, synthetic data has been shown to have limited utility when used to train predictive models. One promising potential application of these data is in the exploratory phase of the machine learning workflow, which involves understanding, engineering and selecting features. This phase often involves considerable time, and depends on the availability of data. There would be substantial value in synthetic data that permitted these steps to be carried out while, for example, data access was being negotiated, or with fewer information governance restrictions. This paper presents an empirical analysis of the agreement between the feature importance obtained from raw and from synthetic data, on a range of artificially generated and real-world datasets (where feature importance represents how useful each feature is when predicting a the outcome). We employ two differentially-private methods to produce synthetic data, and apply various utility measures to quantify the agreement in feature importance as this varies with the level of privacy. Our results indicate that synthetic data can sometimes preserve several representations of the ranking of feature importance in simple settings but their performance is not consistent and depends upon a number of factors. Particular caution should be exercised in more nuanced real-world settings, where synthetic data can lead to differences in ranked feature importance that could alter key modelling decisions. This work has important implications for developing synthetic versions of highly sensitive data sets in fields such as finance and healthcare.

Motivation & Objective

• To assess whether differentially private synthetic data can preserve the relative ranking of feature importance from original datasets.
• To evaluate the utility of synthetic data in early-stage machine learning workflows, particularly for feature engineering and selection.
• To identify dataset characteristics that affect the accuracy of feature importance replication in synthetic data.
• To compare synthetic data performance against naive baselines and assess whether privacy-utility trade-offs undermine reliability.
• To provide practical guidance on when and how synthetic data can be safely used for exploratory analysis in privacy-sensitive domains.

Proposed method

• Empirically evaluated feature importance similarity between original and synthetic datasets using two differentially private synthesis methods: PrivBayes and a second DP method.
• Applied multiple utility metrics, including rank correlation (Spearman’s ρ) and Kendall’s τ, to quantify agreement in feature importance rankings.
• Tested on five artificial datasets and four real-world datasets (Adult, German Credit, Bank Marketing, and a healthcare-related dataset) with varying structures and feature types.
• Varied the privacy budget ε across a range of values to assess the impact of differential privacy on feature importance fidelity.
• Used baseline comparisons, including independent column resampling, to establish a lower bound on synthetic data performance.
• Conducted model training on both original and synthetic data to evaluate predictive performance (AUC) as a proxy for utility.

Experimental results

Research questions

• RQ1 To what extent can differentially private synthetic data preserve the ranking of feature importance compared to the original data?
• RQ2 How does the privacy budget ε affect the similarity of feature importance rankings between original and synthetic datasets?
• RQ3 Are there specific dataset characteristics—such as multicollinearity, feature type (categorical vs. numerical), or hierarchical structure—that reduce the accuracy of feature importance replication?
• RQ4 How does synthetic data performance compare to naive baselines (e.g., independent column resampling) in preserving feature importance?
• RQ5 In what scenarios is synthetic data a reliable proxy for original data in exploratory machine learning tasks?

Key findings

• Synthetic data often fails to preserve accurate feature importance rankings under moderate to high privacy (ε > 0.1), with low and highly variable similarity scores.
• The mean similarity scores for feature importance rankings were consistently low across datasets, especially when ε was small, indicating poor fidelity.
• Performance was worse than naive baselines like independent column resampling in many cases, particularly in real-world datasets with complex structures.
• Datasets with high multicollinearity or nearly uniform feature importances showed the greatest degradation in feature importance fidelity.
• Synthetic data performed better on datasets with fewer features, more rows, and clearer distinctions in feature importance, suggesting structural simplicity aids replication.
• Feature importance was more accurately preserved in datasets with categorical features than in those with numerical features, indicating methodological bias in current DP synthesis techniques.