[Paper Review] tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow

You Xie, Erik Franz+2 more|arXiv (Cornell University)|Jan 29, 2018

Advanced Image Processing Techniques51 references80 citations

TL;DR

tempoGAN introduces a temporally coherent conditional GAN for instant, four-dimensional super-resolution of fluid flows, using a novel temporal discriminator and physics-aware enhancements.

ABSTRACT

We propose a temporally coherent generative model addressing the super-resolution problem for fluid flows. Our work represents a first approach to synthesize four-dimensional physics fields with neural networks. Based on a conditional generative adversarial network that is designed for the inference of three-dimensional volumetric data, our model generates consistent and detailed results by using a novel temporal discriminator, in addition to the commonly used spatial one. Our experiments show that the generator is able to infer more realistic high-resolution details by using additional physical quantities, such as low-resolution velocities or vorticities. Besides improvements in the training process and in the generated outputs, these inputs offer means for artistic control as well. We additionally employ a physics-aware data augmentation step, which is crucial to avoid overfitting and to reduce memory requirements. In this way, our network learns to generate advected quantities with highly detailed, realistic, and temporally coherent features. Our method works instantaneously, using only a single time-step of low-resolution fluid data. We demonstrate the abilities of our method using a variety of complex inputs and applications in two and three dimensions.

Motivation & Objective

• Address the challenge of generating high-resolution, temporally coherent 4D fluid flow fields from a single low-resolution snapshot.
• Advance GAN-based super-resolution to four-dimensional physics problems without requiring temporal data during inference.
• Incorporate physics awareness and input entanglement to improve realism and user control over outputs.
• Demonstrate effectiveness across 2D and 3D flow scenarios and provide training best practices for physics-based GANs.

Proposed method

• Use a conditional GAN architecture to infer 3D high-resolution flow quantities from a single low-resolution frame.
• Introduce a novel temporal discriminator D_t that evaluates sequences of three frames (with advection) to enforce temporal coherence.
• Incorporate a spatial discriminator D_s that ensures realistic spatial details, operating on input x and output G(x).
• Apply a feature-space loss using activations from the discriminator to guide the generator toward realistic features (including the surprising benefit of negative weights in some settings).
• Include a physics-aware data augmentation step to reduce overfitting and manage memory demands, aiding generalization to transport-based flows.
• Employ an advection layer during training to align multi-frame inputs via approximate velocity fields, enabling proper temporal gradients.

Experimental results

Research questions

• RQ1 Can a generative model synthesize high-resolution, temporally coherent 4D fluid flow from a single low-resolution snapshot?
• RQ2 Does adding a temporal discriminator improve coherence of generated flows over time compared to frame-by-frame GANs?
• RQ3 How do physics-informed inputs (e.g., velocity, vorticity) and data augmentation affect realism and controllability of the generated flows?
• RQ4 What training strategies and loss components best balance spatial detail with temporal consistency in physics-based GANs?

Key findings

• A temporal discriminator enables the generator to produce temporally coherent, detailed flow fields across time.
• Incorporating velocity and vorticity as inputs provides both realism and artistic control over outputs.
• Physics-aware data augmentation is crucial to prevent overfitting and to reduce memory usage during training.
• The combined tempoGAN objective yields highly detailed, temporally smooth 4D flow fields inferred from a single low-resolution frame.
• The approach is demonstrated on two and three-dimensional flow problems with instantaneous inference (one time-step).