[Paper Review] Dynamic Upsampling of Smoke through Dictionary-based Learning

Kai Bai, Wei Li+2 more|arXiv (Cornell University)|Oct 21, 2019

Computer Graphics and Visualization Techniques66 references4 citations

TL;DR

This paper proposes a dictionary-based deep learning method for dynamic upsampling of low-resolution smoke simulations to produce high-resolution, physically plausible animations. By learning sparse representations of local velocity patches from coarse-fine simulation pairs, the approach synthesizes realistic fine-scale vortices and flow details more accurately than procedural noise or standard neural networks, achieving near-realistic results up to 100x faster than direct numerical simulation.

ABSTRACT

Simulating turbulent smoke flows is computationally intensive due to their intrinsic multiscale behavior, thus requiring relatively high resolution grids to fully capture their complexity. For iterative editing or simply faster generation of smoke flows, dynamic upsampling of an input low-resolution numerical simulation is an attractive, yet currently unattainable goal. In this paper, we propose a novel dictionary-based learning approach to the dynamic upsampling of smoke flows. For each frame of an input coarse animation, we seek a sparse representation of small, local velocity patches of the flow based on an over-complete dictionary, and use the resulting sparse coefficients to generate a high-resolution smoke animation sequence. We propose a novel dictionary-based neural network which learns both a fast evaluation of sparse patch encoding and a dictionary of corresponding coarse and fine patches from a sequence of example simulations computed with any numerical solver. Our upsampling network then injects into coarse input sequences physics-driven fine details, unlike most previous approaches that only employed fast procedural models to add high frequency to the input. We present a variety of upsampling results for smoke flows and offer comparisons to their corresponding high-resolution simulations to demonstrate the effectiveness of our approach.

Motivation & Objective

• To address the challenge of efficiently generating high-resolution, realistic smoke animations from low-resolution simulations without expensive full numerical solves.
• To overcome limitations of prior methods that rely on procedural noise or lack physical fidelity in fine-scale detail synthesis.
• To develop a method that generalizes across diverse smoke flow contexts, including synthesis from few training examples and re-simulation of original flows with small perturbations.
• To enable fast, iterative animation design by replacing costly high-res simulations with learned upsampling from coarse inputs.

Proposed method

• The method uses a multiscale, space-time encoded dictionary learning framework to represent local velocity patches from coarse and fine simulation pairs.
• A novel neural network is trained to perform sparse coding of coarse input patches using an over-complete dictionary of corresponding fine-scale residuals.
• The network learns to reconstruct high-resolution velocity fields by combining sparse linear combinations of dictionary-patch residuals.
• The approach leverages phase-space encoding to model temporal evolution of local flow structures, improving consistency across frames.
• Training is performed on coarse-fine simulation pairs generated by any numerical solver, enabling generalization across different flow configurations.
• High-resolution outputs are synthesized patch-by-patch, then blended into a coherent velocity grid, preserving divergence-free characteristics through dictionary design.

Experimental results

Research questions

• RQ1 Can a dictionary-based learning approach effectively predict fine-scale vortical structures in turbulent smoke flows from low-resolution inputs?
• RQ2 How well can such a method generalize across diverse animation contexts, including generalized synthesis, restricted synthesis, and re-simulation?
• RQ3 Does learning from coarse-fine simulation pairs yield more physically accurate results than procedural noise-based upsampling?
• RQ4 What are the limitations of the method when applied to flows with significantly different geometries or high turbulence not present in the training data?

Key findings

• The proposed method produces high-resolution smoke animations that visually match ground-truth fine numerical simulations, even when the input is from a different flow configuration than the training data.
• In re-simulation tasks, the method successfully recaptures the original flow dynamics with high fidelity for small perturbations, outperforming baseline methods in realism.
• The method achieves up to 100x speedup compared to direct high-resolution simulation while maintaining visual quality close to the ground truth.
• The approach generalizes well across different flow types, including single-object, multi-object, and vortex ring collisions, when trained on diverse but limited simulation pairs.
• The method outperforms tempoGAN in super-resolution tasks when the coarse input is a downsampled version of a fine simulation, demonstrating superior predictive accuracy.
• Despite limitations in handling highly turbulent or geometrically dissimilar flows, the method maintains near-incompressibility and avoids major visual artifacts in most practical scenarios.