[Paper Review] Good Artists Copy, Great Artists Steal: Model Extraction Attacks Against Image Translation Generative Adversarial Networks.

Sebastian Szyller, Vasisht Duddu+2 more|arXiv (Cornell University)|Apr 26, 2021

Generative Adversarial Networks and Image Synthesis47 references4 citations

TL;DR

This paper presents the first model extraction attack against real-world image translation GANs, where an adversary queries a victim's inference API using domain-specific data to build a functional surrogate model without prior knowledge of the victim's architecture. The attack achieves high fidelity across three image translation tasks, with human perception of outputs from the victim and surrogate models being indistinguishable within a small effect size (Cohen’s d = 0.3).

ABSTRACT

Machine learning models are typically made available to potential client users via inference APIs. Model extraction attacks occur when a malicious client uses information gleaned from queries to the inference API of a victim model $F_V$ to build a surrogate model $F_A$ that has comparable functionality. Recent research has shown successful model extraction attacks against image classification, and NLP models. In this paper, we show the first model extraction attack against real-world generative adversarial network (GAN) image translation models. We present a framework for conducting model extraction attacks against image translation models, and show that the adversary can successfully extract functional surrogate models. The adversary is not required to know $F_V$'s architecture or any other information about it beyond its intended image translation task, and queries $F_V$'s inference interface using data drawn from the same domain as the training data for $F_V$. We evaluate the effectiveness of our attacks using three different instances of two popular categories of image translation: (1) Selfie-to-Anime and (2) Monet-to-Photo (image style transfer), and (3) Super-Resolution (super resolution). Using standard performance metrics for GANs, we show that our attacks are effective in each of the three cases -- the differences between $F_V$ and $F_A$, compared to the target are in the following ranges: Selfie-to-Anime: FID $13.36-68.66$, Monet-to-Photo: FID $3.57-4.40$, and Super-Resolution: SSIM: $0.06-0.08$ and PSNR: $1.43-4.46$. Furthermore, we conducted a large scale (125 participants) user study on Selfie-to-Anime and Monet-to-Photo to show that human perception of the images produced by the victim and surrogate models can be considered equivalent, within an equivalence bound of Cohen's $d=0.3$.

Motivation & Objective

• To investigate whether model extraction attacks are feasible against real-world image translation generative adversarial networks (GANs).
• To develop a framework that enables an adversary to extract a functional surrogate model using only queries to the victim's inference API.
• To evaluate the fidelity of extracted models against the original victim models using standard GAN metrics and human perception.
• To determine whether the surrogate models are perceptually indistinguishable from the victim models in image translation tasks.

Proposed method

• The adversary queries the victim model's inference API using input data drawn from the same domain as the victim’s training data.
• The adversary collects input-output pairs from the victim model to train a surrogate model that mimics its image translation functionality.
• The surrogate model is trained using standard GAN training procedures, with no access to the victim’s architecture or weights.
• The method relies solely on black-box access to the victim’s inference interface, making it practical for real-world deployment.
• Performance is evaluated using standard GAN metrics: Fréchet Inception Distance (FID), Structural Similarity (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
• A large-scale user study with 125 participants was conducted to assess human perception of image quality and similarity between victim and surrogate model outputs.

Experimental results

Research questions

• RQ1 Can a black-box model extraction attack successfully reconstruct a functional image translation GAN using only API queries?
• RQ2 How closely does the performance of the extracted surrogate model match that of the original victim model in terms of standard GAN metrics?
• RQ3 To what extent are the outputs of the surrogate model perceptually indistinguishable from those of the victim model in human evaluations?
• RQ4 How do the attack results vary across different image translation tasks such as style transfer and super-resolution?

Key findings

• The model extraction attack successfully generated surrogate models that closely replicate the functionality of the victim image translation GANs.
• For Selfie-to-Anime translation, the FID between the victim and surrogate models ranged from 13.36 to 68.66, indicating high similarity.
• In Monet-to-Photo style transfer, the FID was between 3.57 and 4.40, showing strong performance fidelity.
• For super-resolution, the surrogate models achieved SSIM differences of 0.06–0.08 and PSNR differences of 1.43–4.46, indicating high-quality reconstruction.
• The human perception study showed that images generated by the surrogate and victim models were indistinguishable, with Cohen’s d effect size of 0.3 within the equivalence bound.
• The attack was effective across diverse image translation tasks without requiring knowledge of the victim model’s architecture or training data.