[Paper Review] PrivacyProber: Assessment and Detection of Soft-Biometric Privacy-Enhancing Techniques

Peter Rot, Peter Peer+1 more|arXiv (Cornell University)|Nov 16, 2022

Face recognition and analysis4 citations

TL;DR

This paper introduces PrivacyProber, a framework that recovers suppressed soft-biometric attributes (e.g., gender, age, ethnicity) from privacy-enhanced facial images, demonstrating that state-of-the-art privacy techniques are vulnerable to attribute reconstruction. It further proposes APEND, a training-free, generalizable detector that identifies privacy-enhanced images with high accuracy, revealing a critical threat vector in biometric privacy systems.

ABSTRACT

Soft-biometric privacy-enhancing techniques represent machine learning methods that aim to: (i) mitigate privacy concerns associated with face recognition technology by suppressing selected soft-biometric attributes in facial images (e.g., gender, age, ethnicity) and (ii) make unsolicited extraction of sensitive personal information infeasible. Because such techniques are increasingly used in real-world applications, it is imperative to understand to what extent the privacy enhancement can be inverted and how much attribute information can be recovered from privacy-enhanced images. While these aspects are critical, they have not been investigated in the literature. We, therefore, study the robustness of several state-of-the-art soft-biometric privacy-enhancing techniques to attribute recovery attempts. We propose PrivacyProber, a high-level framework for restoring soft-biometric information from privacy-enhanced facial images, and apply it for attribute recovery in comprehensive experiments on three public face datasets, i.e., LFW, MUCT and Adience. Our experiments show that the proposed framework is able to restore a considerable amount of suppressed information, regardless of the privacy-enhancing technique used, but also that there are significant differences between the considered privacy models. These results point to the need for novel mechanisms that can improve the robustness of existing privacy-enhancing techniques and secure them against potential adversaries trying to restore suppressed information.

Motivation & Objective

• To evaluate the robustness of soft-biometric privacy-enhancing techniques against attribute recovery attacks.
• To investigate whether privacy-enhanced facial images can be reverse-engineered to restore suppressed attributes such as gender, age, and ethnicity.
• To develop a general-purpose, training-free detection method for identifying privacy-enhanced images across diverse datasets and techniques.
• To assess the real-world reliability of privacy models under adversarial recovery attempts, moving beyond zero-effort evaluations.

Proposed method

• Proposes PrivacyProber, a framework that uses deep learning-based attribute recovery models to reconstruct soft-biometric attributes from privacy-enhanced facial images.
• Employs multiple attribute classifiers (e.g., gender, age, ethnicity) trained on original images to infer suppressed attributes from privacy-enhanced inputs.
• Introduces APEND, a detection method that compares predictions from the recovered image and the original input to detect privacy enhancement without requiring model retraining.
• Uses evidence aggregation across multiple recovery attempts to improve detection robustness, especially in low-signal or artifact-heavy cases.
• Applies the framework across three public datasets—LFW, MUCT, and Adience—under black-box assumptions to ensure generalization.
• Employs a threshold-based decision rule at equal error rate (EER) to balance false positives and false negatives in detection.

Experimental results

Research questions

• RQ1 To what extent can soft-biometric attributes be recovered from privacy-enhanced facial images using state-of-the-art privacy techniques?
• RQ2 How does the performance of attribute recovery vary across different privacy-enhancing models, such as adversarial perturbations and generative models?
• RQ3 Can a training-free, general-purpose detection method be developed to identify privacy-enhanced images regardless of the underlying privacy technique or data distribution?
• RQ4 What is the impact of image quality and artifacts on the reliability of privacy detection and attribute recovery?
• RQ5 How do different privacy models compare in terms of visual impact and residual information leakage?

Key findings

• PrivacyProber successfully recovers a significant portion of suppressed soft-biometric attributes across all tested privacy techniques, indicating that current methods are not robust to targeted recovery attacks.
• The generative models FlowSAN–3 and FlowSAN–5 demonstrated significantly higher robustness against attribute recovery compared to adversarial methods like k–AAP and FGSM.
• APEND achieved an average AUC of 0.940 across datasets, outperforming PREM by over 23% on the Adience dataset with k–AAP, demonstrating the effectiveness of evidence aggregation.
• APEND detected privacy enhancements with high accuracy even under black-box assumptions, generalizing well across diverse data characteristics and privacy models.
• Failure cases in APEND were primarily due to image artifacts or poor image quality, which hindered accurate attribute recovery and led to misdetections.
• The study reveals a critical threat vector: privacy-enhanced images can be reliably detected and potentially exploited through alternative, less artifact-sensitive processing pipelines.