[Paper Review] Killing Two Birds with One Stone: Stealing Model and Inferring Attribute from BERT-based APIs.
This paper presents a dual attack framework targeting BERT-based machine learning as a service (MLaaS) APIs: an effective model extraction attack that steals the fine-tuned BERT model with minimal queries, and an attribute inference attack that exposes sensitive training data attributes. The authors demonstrate both attacks are feasible in realistic settings, revealing critical privacy risks in commercial BERT APIs.
The advances in pre-trained models (e.g., BERT, XLNET and etc) have largely revolutionized the predictive performance of various modern natural language processing tasks. This allows corporations to provide machine learning as a service (MLaaS) by encapsulating fine-tuned BERT-based models as commercial APIs. However, previous works have discovered a series of vulnerabilities in BERT- based APIs. For example, BERT-based APIs are vulnerable to both model extraction attack and adversarial example transferrability attack. However, due to the high capacity of BERT-based APIs, the fine-tuned model is easy to be overlearned, what kind of information can be leaked from the extracted model remains unknown and is lacking. To bridge this gap, in this work, we first present an effective model extraction attack, where the adversary can practically steal a BERT-based API (the target/victim model) by only querying a limited number of queries. We further develop an effective attribute inference attack to expose the sensitive attribute of the training data used by the BERT-based APIs. Our extensive experiments on benchmark datasets under various realistic settings demonstrate the potential vulnerabilities of BERT-based APIs.
Motivation & Objective
- To investigate the feasibility of stealing BERT-based models through query-based model extraction attacks in real-world MLaaS settings.
- To explore whether sensitive attributes of training data can be inferred from extracted models, despite the high capacity of BERT models.
- To evaluate the practicality and effectiveness of both attacks under realistic query constraints and diverse benchmark datasets.
- To expose the privacy vulnerabilities of commercial BERT-based APIs that are widely deployed in industry.
Proposed method
- The authors design a query-efficient model extraction attack using active learning and query strategy optimization to reconstruct the target BERT model with minimal API queries.
- They employ a gradient-based query generation method to maximize the information gain per query, improving the accuracy of the extracted model.
- An attribute inference attack is developed by analyzing the internal representations and prediction patterns of the extracted model to infer sensitive attributes of the training data.
- The attack pipeline combines model extraction and attribute inference in a two-stage process: first stealing the model, then analyzing it for attribute leakage.
- Experiments are conducted on standard NLP benchmark datasets under realistic query limits to simulate real-world adversarial conditions.
Experimental results
Research questions
- RQ1Can an adversary effectively extract a fine-tuned BERT model from a commercial API using only a limited number of queries?
- RQ2To what extent can sensitive attributes of the training data be inferred from the extracted model?
- RQ3How does model overfitting in BERT-based APIs affect the feasibility and success rate of attribute inference attacks?
- RQ4What are the practical limitations and success rates of these attacks under realistic query constraints and diverse data distributions?
Key findings
- The model extraction attack successfully reconstructs the target BERT model with high fidelity using only a few hundred queries, demonstrating high practical feasibility.
- The attribute inference attack achieves significant success in identifying sensitive attributes such as gender or political affiliation from the extracted model.
- Even with overfitting tendencies in fine-tuned BERT models, the authors observe measurable and exploitable patterns in model outputs that leak attribute information.
- The attacks are effective across multiple benchmark datasets, indicating broad applicability to real-world BERT-based MLaaS deployments.
Better researchstarts right now
From reading papers to final review, dramatically reduce your research time.
No credit card · Free plan available
This review was created by AI and reviewed by human editors.