[Paper Review] $k$-Anonymity in Practice: How Generalisation and Suppression Affect Machine Learning Classifiers

Djordje Slijepčević, Maximilian Henzl+4 more|arXiv (Cornell University)|Feb 9, 2021

Privacy-Preserving Technologies in Data35 references4 citations

TL;DR

This paper evaluates how k-anonymity techniques—specifically generalisation and suppression—affect machine learning classifier performance across real-world datasets. It finds that while classification accuracy degrades with stronger k-anonymity constraints, the Mondrian algorithm preserves performance best, especially under high k values, and suppression offers flexibility without clear performance trade-offs.

ABSTRACT

The protection of private information is a crucial issue in data-driven research and business contexts. Typically, techniques like anonymisation or (selective) deletion are introduced in order to allow data sharing, e. g. in the case of collaborative research endeavours. For use with anonymisation techniques, the $k$-anonymity criterion is one of the most popular, with numerous scientific publications on different algorithms and metrics. Anonymisation techniques often require changing the data and thus necessarily affect the results of machine learning models trained on the underlying data. In this work, we conduct a systematic comparison and detailed investigation into the effects of different $k$-anonymisation algorithms on the results of machine learning models. We investigate a set of popular $k$-anonymisation algorithms with different classifiers and evaluate them on different real-world datasets. Our systematic evaluation shows that with an increasingly strong $k$-anonymity constraint, the classification performance generally degrades, but to varying degrees and strongly depending on the dataset and anonymisation method. Furthermore, Mondrian can be considered as the method with the most appealing properties for subsequent classification.

Motivation & Objective

• To systematically assess how k-anonymity techniques—generalisation and suppression—affect machine learning classifier performance.
• To compare multiple k-anonymisation algorithms (e.g., Mondrian, MDAV, k-Optimize) in terms of their impact on downstream ML utility.
• To identify anonymisation strategies that maintain high classification accuracy while ensuring strong privacy guarantees under k-anonymity.
• To provide open-source code and datasets to support reproducibility and future research in privacy-preserving machine learning.

Proposed method

• Applied a suite of k-anonymisation algorithms—including Mondrian, MDAV, and k-Optimize—on four real-world datasets with varying characteristics.
• Used generalisation and suppression as core data transformation techniques to achieve k-anonymity, where each quasi-identifier (QID) value must be shared by at least k−1 other records.
• Trained multiple classifiers (e.g., Random Forest, SVM, Logistic Regression) on both original and anonymised datasets to compare performance.
• Measured classification performance using standard metrics (e.g., accuracy, F1-score) and evaluated information loss and data distortion effects.
• Conducted ablation studies to isolate the impact of generalisation vs. suppression on model performance.
• Released all code, datasets, and metadata via GitHub to promote open science and reproducibility.

Experimental results

Research questions

• RQ1 How does increasing the k-anonymity constraint affect the performance of machine learning classifiers across different datasets?
• RQ2 Which k-anonymisation algorithm—Mondrian, MDAV, or k-Optimize—best preserves classification performance under strong k-anonymity?
• RQ3 What is the relative impact of generalisation versus suppression on downstream ML model accuracy?
• RQ4 How do dataset-specific characteristics (e.g., feature correlation with target variable) influence the performance degradation caused by anonymisation?
• RQ5 To what extent can suppression be used to reduce fluctuations introduced by generalisation without degrading model performance?

Key findings

• Classification performance generally degrades with increasing k-anonymity, but the extent of degradation varies significantly across datasets and anonymisation methods.
• Mondrian consistently outperformed other k-anonymisation methods, maintaining high classification accuracy even at k=100, particularly on the adult and cmc datasets.
• For the adult dataset, performance loss was minimal (under 7%) with Random Forest and Mondrian, while the mgm and cahousing datasets showed around 13% performance loss under similar conditions.
• Suppression introduced less predictable fluctuations than generalisation, and allowing some suppression improved anonymisation flexibility without clear performance penalties.
• Generalisation had a stronger negative impact when quasi-identifiers were highly correlated with the target variable, indicating feature-specific sensitivity to data distortion.
• The study confirms that k-anonymity can be applied with acceptable utility loss even at high k values (e.g., k=100), especially when using Mondrian, supporting its practical viability for privacy-preserving ML.