[논문 리뷰] Asymmetric regularization mechanism for GAN training with Variational Inequalities

Spyridon C. Giagtzoglou, Mark H. M. Winands외 1명|arXiv (Cornell University)|2026. 01. 20.

Stochastic Gradient Optimization Techniques인용 수 0

한 줄 요약

본 논문은 GAN 학습을 VI 프레임워크 하의 Nash 평형 문제로 공식화하고, discriminator-only asymmetric regularization (Tikhonov 및 zero-centered gradient penalties)을 제안하여 단일 호출 Extrapolation-from-the-Past 방법에 대한 명시적 수렴 보장을 달성한다.

ABSTRACT

We formulate the training of generative adversarial networks (GANs) as a Nash equilibrium seeking problem. To stabilize the training process and find a Nash equilibrium, we propose an asymmetric regularization mechanism based on the classic Tikhonov step and on a novel zero-centered gradient penalty. Under smoothness and a local identifiability condition induced by a Gauss-Newton Gramian, we obtain explicit Lipschitz and (strong)-monotonicity constants for the regularized operator. These constants ensure last-iterate linear convergence of a single-call Extrapolation-from-the-Past (EFTP) method. Empirical simulations on an academic example show that, even when strong monotonicity cannot be achieved, the asymmetric regularization is enough to converge to an equilibrium and stabilize the trajectory.

연구 동기 및 목표

• Model GAN training as a two-player zero-sum game and interpret equilibria via variational inequalities.
• Introduce asymmetric regularization acting only on the discriminator to inject curvature and stabilize training.
• Derive explicit Lipschitz and (strong) monotonicity constants for the regularized operator under smoothness and Gauss–Newton identifiability.
• Show convergence guarantees for Extrapolation-from-the-Past (EFTP) and discuss alternatives like FB and EG.
• Provide empirical validation on an academic example illustrating stabilization and convergence improvements.

제안 방법

• Formulate the GAN training as VI(S,F) with F(ω) built from gradients of the generator and negative gradient of the discriminator.
• Introduce two asymmetric regularizations on the discriminator: Tik (γ/2 ||φ||^2) and SGP (zero-centered input-gradient penalty) that vanish at equilibrium.
• Analyze the regularized operator F_γ to establish Lipschitz continuity and strong monotonicity via assumptions including Gauss–Newton identifiability (J_G(ω) ≽ λ_0 I).
• Derive constants: L = L_0 + γ κ_tot and μ ≥ min{μ_θ, μ_φ + γ λ_0} ensuring local linear convergence for iterative schemes.
• Propose Extrapolation-from-the-Past (EFTP) as a computationally efficient algorithm with convergence guarantees under the regularized VI.
• Provide numerical validation on a bilinear toy example to illustrate stabilization effects of asymmetric regularization.

실험 결과

연구 질문

• RQ1 Can discriminator-only asymmetric regularization stabilize GAN training within a VI framework?
• RQ2 Do explicit Lipschitz and strong-monotonicity constants exist for the regularized operator under smoothness and Gauss–Newton identifiability?
• RQ3 Does EFTP converge linearly to a Nash equilibrium for the regularized GAN VI despite potential lack of strong monotonicity in the unregularized setting?
• RQ4 How do the proposed regularizations compare with standard symmetric Tikhonov Regularization in terms of convergence and stability?
• RQ5 Under what conditions do the regularized dynamics preserve the original saddle point while improving conditioning?

주요 결과

• Discriminator-only asymmetric regularization introduces curvature that improves conditioning without altering the equilibrium point.
• Explicit Lipschitz and strong-monotonicity constants for the regularized operator are derived under smoothness and Gauss–Newton identifiability assumptions.
• The regularized operator F_γ is Lipschitz with L = L_0 + γ κ_tot and strongly monotone with μ ≥ min{μ_θ, μ_φ + γ λ_0} on a neighborhood, enabling convergence guarantees.
• EFTP converges linearly to a Nash equilibrium of the regularized VI, with a log(1/ε) iteration complexity for a given accuracy.
• Numerical experiments on a bilinear toy model show discriminator-only curvature stabilizes FB dynamics and enables convergence for EG and EFTP, with EFTP offering similar performance at lower gradient evaluations.