[Paper Review] Global versus Localized Generative Adversarial Nets

Guo-Jun Qi, Liheng Zhang+4 more|arXiv (Cornell University)|Nov 16, 2017

Generative Adversarial Networks and Image Synthesis4 references3 citations

TL;DR

This paper proposes Localized Generative Adversarial Networks (LGAN), which use local coordinate charts to model data manifolds, enabling direct access to local geometry and mitigating mode collapse via orthonormality constraints on tangent vectors. LGAN achieves state-of-the-art performance in image generation and semi-supervised classification, with a 9.77% error rate on CIFAR-10 using 4,000 labeled examples.

ABSTRACT

In this paper, we present a novel localized Generative Adversarial Net (GAN) to learn on the manifold of real data. Compared with the classic GAN that {\em globally} parameterizes a manifold, the Localized GAN (LGAN) uses local coordinate charts to parameterize distinct local geometry of how data points can transform at different locations on the manifold. Specifically, around each point there exists a {\em local} generator that can produce data following diverse patterns of transformations on the manifold. The locality nature of LGAN enables local generators to adapt to and directly access the local geometry without need to invert the generator in a global GAN. Furthermore, it can prevent the manifold from being locally collapsed to a dimensionally deficient tangent subspace by imposing an orthonormality prior between tangents. This provides a geometric approach to alleviating mode collapse at least locally on the manifold by imposing independence between data transformations in different tangent directions. We will also demonstrate the LGAN can be applied to train a robust classifier that prefers locally consistent classification decisions on the manifold, and the resultant regularizer is closely related with the Laplace-Beltrami operator. Our experiments show that the proposed LGANs can not only produce diverse image transformations, but also deliver superior classification performances.

Motivation & Objective

• To address the limitations of global GANs in modeling complex, non-linear data manifolds with diverse local transformations.
• To enable direct access to local geometric properties—such as tangent vectors and curvatures—without inverting the generator network.
• To prevent local manifold collapse by enforcing orthonormality on tangent vectors, preserving full-dimensional variation.
• To develop a locally consistent classifier regularized by a geometric prior related to the Laplace-Beltrami operator, without relying on graph-based approximations.
• To demonstrate superior performance in both image generation and semi-supervised classification tasks.

Proposed method

• LGAN employs local generators $G(\mathbf{x}, \mathbf{z})$ parameterized by local coordinate charts centered at individual data points $\mathbf{x}$, enabling localized modeling of data transformations.
• Each local generator operates within a neighborhood of a point on the data manifold, allowing seamless stitching of generated data across overlapping regions.
• An orthonormality prior is imposed on the Jacobian of the local generator to ensure that tangent vectors span a full-dimensional space, preventing local dimensionality deficiency.
• The method avoids the need to invert the generator by directly associating each data point with its local generator, enabling direct geometric analysis at the point level.
• The local geometric regularizer derived from the orthonormality constraint is shown to be equivalent to a Laplace-Beltrami operator-based regularizer, without requiring graph-based manifold approximations.
• The framework supports training a locally consistent classifier robust to perturbations along manifold tangent directions, with performance validated on CIFAR-10 and CIFAR-100.

Experimental results

Research questions

• RQ1 Can local coordinate charts provide a more effective and geometrically consistent alternative to global parameterization in GANs for modeling complex data manifolds?
• RQ2 How can local geometric properties such as tangent vectors and curvatures be directly accessed without inverting the generator network?
• RQ3 To what extent does enforcing orthonormality on local tangent vectors prevent local manifold collapse and improve mode diversity?
• RQ4 Can the geometric regularizer derived from the orthonormality constraint be used to train a classifier robust to manifold-aligned perturbations?
• RQ5 Does the proposed LGAN framework achieve state-of-the-art performance in both image generation and semi-supervised classification?

Key findings

• LGAN achieves a test error of 9.77% on CIFAR-10 with only 4,000 labeled examples, outperforming prior state-of-the-art methods such as VAT and Pi-Model.
• On CIFAR-100 with 10,000 labeled examples, LGAN achieves a test error of 35.52%, surpassing all compared semi-supervised learning baselines.
• The orthonormality constraint on local tangent vectors effectively prevents local manifold collapse, preserving full-dimensional data variation across different directions.
• The local geometric regularizer in LGAN is mathematically equivalent to a Laplace-Beltrami operator-based regularizer, but derived without relying on graph-based approximations of the manifold.
• The method enables direct access to local geometric features such as tangent vectors and curvatures without requiring inversion of the generator network.
• LGAN generates diverse image transformations that are consistent with the underlying manifold structure, demonstrating improved sample diversity and quality.