[Paper Review] Out-domain examples for generative models.

Dario Pasquini, Marco Mingione+1 more|arXiv (Cornell University)|Mar 7, 2019

Generative Adversarial Networks and Image Synthesis29 references4 citations

TL;DR

This paper demonstrates that adversarial inputs can force pre-trained generative models to produce arbitrary out-domain examples—rare or impossible outputs—by manipulating the model's latent space. The method generates adversarial noise indistinguishable from real inputs, enabling reliable reproduction of low-probability, out-of-distribution samples.

ABSTRACT

Deep generative models are being increasingly used in a wide variety of applications. However, the generative process is not fully predictable and at times, it produces an unexpected output. We will refer to those outputs as out-domain examples. In the present paper we show that an attacker can force a pre-trained generator to reproduce an arbitrary out-domain example if fed by a suitable adversarial input. The main assumption is that those outputs lie in an unexplored region of the generator's codomain and hence they have a very low probability of being naturally generated. Moreover, we show that this adversarial input can be shaped so as to be statistically indistinguishable from the set of genuine inputs. The goal is to look for an efficient way of finding these inputs in the generator's latent space.

Motivation & Objective

• To investigate whether pre-trained generative models can be manipulated into producing arbitrary out-domain examples.
• To identify adversarial inputs in the latent space that trigger such unexpected, low-probability outputs.
• To ensure these adversarial inputs remain statistically indistinguishable from genuine inputs, preserving stealthiness.
• To develop an efficient method for locating these adversarial inputs within the generator's codomain.

Proposed method

• The approach involves searching for adversarial latent vectors that, when fed into a pre-trained generator, produce a target out-domain example.
• The method assumes that out-domain examples lie in low-probability regions of the generator's output space.
• It formulates the problem as an optimization task in the latent space to minimize the distance between the generated output and the target out-domain example.
• The adversarial inputs are constrained to lie within a distribution indistinguishable from real data, ensuring stealth.
• The optimization uses gradient-based techniques to navigate the latent space efficiently.
• The method ensures that the adversarial inputs are not easily detectable by distributional similarity to real inputs.

Experimental results

Research questions

• RQ1 Can adversarial inputs be constructed to force a pre-trained generator to produce a specific, arbitrary out-domain example?
• RQ2 How can such adversarial inputs be made statistically indistinguishable from real data inputs?
• RQ3 What is the feasibility and efficiency of locating these adversarial inputs within the latent space?
• RQ4 To what extent can the generator be manipulated to produce low-probability, out-of-distribution outputs?
• RQ5 How robust is the method across different types of generative models and out-domain targets?

Key findings

• Adversarial inputs can successfully force pre-trained generators to produce arbitrary out-domain examples not naturally generated by the model.
• The generated adversarial inputs are statistically indistinguishable from real inputs, preserving stealth.
• The method efficiently locates adversarial latent vectors in the generator's latent space.
• The approach works effectively even when the target out-domain example lies in a region of very low probability.
• The technique demonstrates that generative models are vulnerable to targeted, undetectable manipulations in their latent space.
• The results show that the generator's output space contains exploitable regions that can be accessed via adversarial perturbations.