[Paper Review] Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems

Yang Zeng, Jinlong Wu+1 more|arXiv (Cornell University)|Nov 15, 2019

Model Reduction and Neural Networks30 references15 citations

TL;DR

This paper proposes embedding deterministic, imprecise physical constraints—such as divergence-free velocity fields or geometric shapes—directly into the generator loss of generative adversarial networks (GANs) to improve training stability and sample fidelity. By enforcing these constraints via a differentiable penalty term, the method accelerates convergence and enhances robustness, even when constraints are only approximately satisfied, demonstrating significant improvements over standard GANs in both geometric and differential constraint tasks.

ABSTRACT

Generative adversarial networks (GANs) were initially proposed to generate images by learning from a large number of samples. Recently, GANs have been used to emulate complex physical systems such as turbulent flows. However, a critical question must be answered before GANs can be considered trusted emulators for physical systems: do GANs-generated samples conform to the various physical constraints? These include both deterministic constraints (e.g., conservation laws) and statistical constraints (e.g., energy spectrum of turbulent flows). The latter have been studied in a companion paper (Wu et al., Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems. Journal of Computational Physics. 406, 109209, 2020). In the present work, we enforce deterministic yet imprecise constraints on GANs by incorporating them into the loss function of the generator. We evaluate the performance of physics-constrained GANs on two representative tasks with geometrical constraints (generating points on circles) and differential constraints (generating divergence-free flow velocity fields), respectively. In both cases, the constrained GANs produced samples that conform to the underlying constraints rather accurately, even though the constraints are only enforced up to a specified interval. More importantly, the imposed constraints significantly accelerate the convergence and improve the robustness in the training, indicating that they serve as a physics-based regularization. These improvements are noteworthy, as the convergence and robustness are two well-known obstacles in the training of GANs.

Motivation & Objective

• To address the lack of physical consistency in GAN-generated samples for complex physical systems like turbulent flows.
• To improve the convergence and robustness of GAN training, which are well-known challenges in adversarial training.
• To develop a generalizable framework for incorporating deterministic, imprecise physical constraints into GANs without requiring exact constraint satisfaction.
• To evaluate the effectiveness of physics-based regularization via constraint enforcement in the generator loss function.
• To demonstrate that even approximate physical constraints can significantly enhance GAN performance in emulating physical systems.

Proposed method

• Incorporate deterministic physical constraints—such as divergence-free velocity fields or points on a circle—into the generator's loss function as a differentiable penalty term.
• Use a constraint loss function that activates only when the constraint violation exceeds a tolerance threshold ε, allowing for imprecise enforcement.
• Formulate the constraint penalty as a function of the magnitude of constraint violation, e.g., |∇·v| when the divergence exceeds ε, to maintain differentiability.
• Train the GAN using a standard adversarial framework, but with the modified generator loss that includes both the adversarial loss and the physics constraint penalty.
• Adjust the weight λ of the constraint term and the tolerance ε to study their impact on convergence and constraint satisfaction.
• Evaluate performance on two test cases: generating points on a circle (geometric constraint) and generating divergence-free velocity fields (differential constraint).

Experimental results

Research questions

• RQ1 Can imprecise deterministic constraints be effectively embedded into GANs to improve training convergence and robustness?
• RQ2 How does the level of constraint tolerance ε affect the stability and accuracy of constraint enforcement in GAN training?
• RQ3 To what extent does physics-based regularization via constraint penalties improve the quality of generated samples compared to standard GANs?
• RQ4 Does enforcing constraints that are only approximately satisfied still yield meaningful improvements in GAN performance?
• RQ5 What is the impact of discontinuous constraint loss functions (e.g., based on ε-tolerance) on training stability?

Key findings

• Physics-constrained GANs significantly accelerate training convergence and improve robustness compared to standard GANs, even when constraints are only approximately satisfied.
• For the divergence-free velocity field task, constraint enforcement with ε = 0 and ε = 1 led to mean divergence values close to zero (0.64 in training data), while ε = 2.3 caused instability due to intermittent loss function activation.
• The standard GAN (equivalent to ε → ∞) produced velocity fields with larger mean divergence than constrained GANs with ε = 0 or ε = 1, indicating that constraint enforcement improves physical fidelity.
• The constraint loss function with a tolerance ε introduces discontinuities in the gradient when the constraint violation crosses ε, leading to instability when ε is too large (e.g., ε = 2.3), as observed in fluctuating training histories.
• Even with imprecise constraints (e.g., ε = 1), the GANs generated samples that closely satisfied the physical constraints, demonstrating the effectiveness of the regularization approach.
• The method successfully enforces both geometric (circle) and differential (divergence-free) constraints, showing broad applicability to various physical systems.