[Paper Review] Model Extraction Attacks against Recurrent Neural Networks

Tatsuya Takemura, Naoto Yanai+1 more|arXiv (Cornell University)|Feb 1, 2020

Adversarial Robustness in Machine Learning22 references4 citations

TL;DR

This paper presents novel model extraction attacks against recurrent neural networks (RNNs) and long short-term memory (LSTM) networks by exploiting intermediate outputs and custom loss functions. It demonstrates that adversaries can extract substitute models with higher accuracy than the original LSTM using only 20% of the training data and partial query access, particularly by leveraging softmax temperature scaling and sequence-wise output leakage.

ABSTRACT

Model extraction attacks are a kind of attacks in which an adversary obtains a new model, whose performance is equivalent to that of a target model, via query access to the target model efficiently, i.e., fewer datasets and computational resources than those of the target model. Existing works have dealt with only simple deep neural networks (DNNs), e.g., only three layers, as targets of model extraction attacks, and hence are not aware of the effectiveness of recurrent neural networks (RNNs) in dealing with time-series data. In this work, we shed light on the threats of model extraction attacks against RNNs. We discuss whether a model with a higher accuracy can be extracted with a simple RNN from a long short-term memory (LSTM), which is a more complicated and powerful RNN. Specifically, we tackle the following problems. First, in a case of a classification problem, such as image recognition, extraction of an RNN model without final outputs from an LSTM model is presented by utilizing outputs halfway through the sequence. Next, in a case of a regression problem. such as in weather forecasting, a new attack by newly configuring a loss function is presented. We conduct experiments on our model extraction attacks against an RNN and an LSTM trained with publicly available academic datasets. We then show that a model with a higher accuracy can be extracted efficiently, especially through configuring a loss function and a more complex architecture different from the target model.

Motivation & Objective

• To investigate the feasibility and effectiveness of model extraction attacks against RNNs and LSTMs, which are underexplored in existing literature.
• To develop new attack strategies that exploit architectural features of RNNs and LSTMs, such as sequential processing and hidden state feedback.
• To demonstrate that substitute models with higher accuracy than the original model can be extracted using fewer resources.
• To evaluate the impact of model architecture complexity and loss function design on the success of model extraction attacks.
• To explore practical countermeasures and limitations of existing defenses like watermarking and differential privacy.

Proposed method

• For classification tasks, the attack leverages intermediate outputs (not just final outputs) from an LSTM model during sequence processing to train a substitute RNN.
• The attack uses softmax with temperature scaling to enhance the information available in model logits, improving the fidelity of the substitute model.
• For regression tasks, a new loss function based on teacher-bounded regression is introduced to align the substitute model’s predictions with the original model’s outputs.
• The substitute model is trained using query responses from the original model, with labels from the original dataset and predictions used as supervision.
• The attack is evaluated on MNIST (converted to time-series) and Air Quality datasets to measure accuracy and generalization under data and query constraints.
• The architecture of the substitute model is intentionally made more complex than the original to improve performance, exploiting architectural flexibility.

Experimental results

Research questions

• RQ1 Can a substitute RNN model be extracted from an LSTM model with higher accuracy than the original using only partial query access?
• RQ2 How does leveraging intermediate outputs (mid-sequence) improve the quality of extracted models in RNN-based attacks?
• RQ3 To what extent can a custom loss function enhance the performance of a substitute model in regression tasks?
• RQ4 What role does architectural complexity play in the success of model extraction attacks against RNNs and LSTMs?
• RQ5 How effective are existing countermeasures such as watermarking and differential privacy against these advanced model extraction techniques?

Key findings

• A substitute RNN model achieved 97.5% accuracy on MNIST using only 20% of the original training data, slightly outperforming the original LSTM’s 97.3%.
• In the regression task on the Air Quality dataset, a substitute model achieved an R² of 87.2% with only three months of training data, compared to the original model’s 89.9% with ten months.
• The use of softmax with temperature scaling significantly improved the information leakage from the original model, enabling higher-fidelity substitute models.
• The attack succeeded by exploiting the sequential nature of RNNs, where intermediate hidden states contain exploitable predictive signals not present in final outputs alone.
• More complex substitute architectures outperformed simpler ones, indicating that architectural freedom enhances the success of model extraction.
• Existing countermeasures such as watermarking and differential privacy were found to be insufficient, as they do not prevent extraction and may be bypassed by distillation-based attacks.