[Paper Review] PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators

Nils Lukas, Florian Kerschbaum|arXiv (Cornell University)|Apr 14, 2023

Advanced Steganography and Watermarking Techniques4 citations

TL;DR

PTW introduces a fast, data-free watermarking method for pre-trained image generators using pivotal tuning, enabling watermark embedding in under an hour—three orders of magnitude faster than training from scratch—while supporting larger models and longer codes. Crucially, it demonstrates that white-box attackers can remove watermarks with minimal data, challenging the robustness of watermarking for deepfake detection.

ABSTRACT

Deepfakes refer to content synthesized using deep generators, which, when misused, have the potential to erode trust in digital media. Synthesizing high-quality deepfakes requires access to large and complex generators only a few entities can train and provide. The threat is malicious users that exploit access to the provided model and generate harmful deepfakes without risking detection. Watermarking makes deepfakes detectable by embedding an identifiable code into the generator that is later extractable from its generated images. We propose Pivotal Tuning Watermarking (PTW), a method for watermarking pre-trained generators (i) three orders of magnitude faster than watermarking from scratch and (ii) without the need for any training data. We improve existing watermarking methods and scale to generators $4 imes$ larger than related work. PTW can embed longer codes than existing methods while better preserving the generator's image quality. We propose rigorous, game-based definitions for robustness and undetectability, and our study reveals that watermarking is not robust against an adaptive white-box attacker who controls the generator's parameters. We propose an adaptive attack that can successfully remove any watermarking with access to only 200 non-watermarked images. Our work challenges the trustworthiness of watermarking for deepfake detection when the parameters of a generator are available. The source code to reproduce our experiments is available at https://github.com/nilslukas/gan-watermark.

Motivation & Objective

• To address the inefficiency and scalability limitations of existing watermarking methods that require retraining from scratch for pre-trained image generators.
• To enable watermarking of large-scale generators without access to training data, reducing computational cost and time.
• To improve the capacity-to-quality trade-off in watermarking by embedding longer codes while preserving image fidelity.
• To rigorously define and evaluate watermark robustness and undetectability using game-based security models.
• To investigate the vulnerability of watermarking under white-box adversarial settings where the attacker controls the generator’s parameters.

Proposed method

• PTW leverages pivotal tuning to efficiently embed watermarks into pre-trained generators by optimizing only a small subset of the model’s parameters, avoiding full fine-tuning.
• The method uses a secret key to modulate specific layers in the generator’s feature space, embedding a binary code into the latent space during image synthesis.
• Watermark extraction is performed using a key-dependent decoder that analyzes generated images to recover the embedded code.
• The approach is compatible with any generator that maps latent codes to images, including StyleGAN and latent diffusion models.
• An adaptive attack, Reverse Pivotal Tuning (RPT), is proposed that removes watermarks by optimizing the generator’s parameters using only 200 non-watermarked images.
• Game-based security definitions are introduced to formally model robustness (resistance to removal) and undetectability (lack of statistical distinguishability without the key).

Figure 1: A demonstration of our watermark and the impact of the number of embedded bits on the visual image quality. The top row shows the watermarked, synthetic image and the bottom row shows its difference to the same image without a watermark.

Experimental results

Research questions

• RQ1 Can watermarking be efficiently applied to pre-trained image generators without requiring any training data?
• RQ2 How does the performance of watermarking scale with generator size, and can it support longer embedded codes?
• RQ3 Is watermarking robust against an adaptive white-box attacker with full access to the generator’s parameters?
• RQ4 What is the minimum number of non-watermarked images required for an attacker to successfully remove a watermark?
• RQ5 Can watermarking remain undetectable to an adversary without the secret key, even under strong statistical analysis?

Key findings

• PTW reduces watermark embedding time from over a GPU month to under one hour, achieving a three orders of magnitude speedup over prior methods.
• The method successfully embeds longer watermarks into generators up to four times larger than those supported by prior work, with minimal degradation in image quality.
• Watermarking is undetectable without the secret key, requiring at least 100 labeled non-watermarked and watermarked images for an attacker to detect the presence of a watermark.
• An adaptive white-box attacker can remove any watermark using only 200 non-watermarked images, with less than 0.3% of the original training data, and with negligible impact on image quality.
• The study reveals that existing watermarking schemes are not robust in the white-box threat model, undermining their viability for IP protection when generator parameters are exposed.
• Despite strong performance against black-box adversaries, watermarking fails to deter misuse when the model is deployed as a white-box, challenging its role in proactive deepfake mitigation.

Figure 2: Deepfake detection in a no-box setting by watermarking the generator.