[Paper Review] Alias-Free Generative Adversarial Networks

Tero Karras, Miika Aittala+5 more|arXiv (Cornell University)|Jun 23, 2021

Generative Adversarial Networks and Image Synthesis63 references815 citations

TL;DR

The paper identifies aliasing as the root cause of texture sticking in GAN generators and proposes a continuous-signal, alias-free redesign of StyleGAN2 (StyleGAN3) that achieves translation and rotation equivariance, matching StyleGAN2’s FID with improved internal representations.

ABSTRACT

We observe that despite their hierarchical convolutional nature, the synthesis process of typical generative adversarial networks depends on absolute pixel coordinates in an unhealthy manner. This manifests itself as, e.g., detail appearing to be glued to image coordinates instead of the surfaces of depicted objects. We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hierarchical synthesis process. The resulting networks match the FID of StyleGAN2 but differ dramatically in their internal representations, and they are fully equivariant to translation and rotation even at subpixel scales. Our results pave the way for generative models better suited for video and animation.

Motivation & Objective

• Explain how aliasing and border effects cause texture sticking in GAN generators.
• Develop a continuous-domain, alias-free generator architecture to enforce equivariance to sub-pixel translations and rotations.
• Adapt StyleGAN2 into an alias-free, continuously equivariant generator (StyleGAN3) with minimal FID loss.
• Demonstrate that the alias-free design yields rotation and translation equivariance without sacrificing image quality.

Proposed method

• Reformulate CNN signal processing in a continuous domain using bandlimited signals and Shannon–Nyquist theory.
• Identify and mitigate two aliasing sources: upsampling filters and pointwise nonlinearities, via high-attenuation filters and continuous-domain low-pass filtering.
• Replace sampling/padding with boundary extension and windowed sinc (Kaiser) filters to suppress border-induced coordinate references.
• Implement a multi-step redesign of the StyleGAN2 generator (including Fourier features, filtered nonlinearities, and rotation-equivariant 1x1 convolutions) to achieve continuous equivariance.
• Introduce configurable per-layer filter budgets, stopband handling, and per-image Fourier feature controls to manage aliasing across scales.
• Provide a rotation-equivariant generator by using 1x1 convolutions and radially symmetric downsampling filters (jinc-based).
• Propose and validate a learned affine transform on input Fourier features to permit per-image global transformations (translation/rotation).

Experimental results

Research questions

• RQ1 Can aliasing in GAN generators be eliminated to enforce true sub-pixel translation and rotation equivariance?
• RQ2 What minimal architectural and signal-processing changes are needed to convert StyleGAN2 into an alias-free, continuously equivariant generator?
• RQ3 How does an alias-free generator (StyleGAN3) compare to StyleGAN2 in terms of FID and explicit equivariance metrics?
• RQ4 Does rotation equivariance compromise image quality, and under what configurations is it achieved with acceptable FID?
• RQ5 What practical design guidelines (filters, padding, layer specifications) yield stable, scalable equivariant generation across datasets?

Key findings

• The alias-free redesign yields translation and rotation equivariance without sacrificing StyleGAN2 FID performance.
• StyleGAN3-R achieves strong rotation equivariance (EQ-R) with high EQ-T gains while maintaining competitive FID across datasets.
• Replacing border padding with extended canvases and using high-attenuation, windowed sinc filters significantly improves translation equivariance.
• Filtered nonlinearities with upsampling/downsampling and per-layer stopband budgeting reduce aliasing-induced texture sticking.
• Rotation-equivariant generation can be achieved with 1x1 convolutions and radially symmetric downsampling filters, preserving performance.
• Final configurations demonstrate EQ-T values around 60–70 dB for translation and 40–48 dB for rotation with competitive FID on multiple datasets.