[Paper Review] Adversarial sampling of unknown and high-dimensional conditional distributions

Malik Hassanaly, Andrew Glaws+2 more|arXiv (Cornell University)|Nov 8, 2021

Probabilistic and Robust Engineering Design78 references20 citations

TL;DR

This paper proposes a GAN-based method for sampling high-dimensional conditional distributions when the underlying PDF is unknown and conditioning variables are continuous. By estimating conditional moments via neural networks or stochastic approximation, the approach enables diverse, high-fidelity sampling of complex systems—demonstrated in atmospheric turbulence deconvolution—while improving sample diversity without sacrificing fidelity compared to state-of-the-art methods.

ABSTRACT

Many engineering problems require the prediction of realization-to-realization variability or a refined description of modeled quantities. In that case, it is necessary to sample elements from unknown high-dimensional spaces with possibly millions of degrees of freedom. While there exist methods able to sample elements from probability density functions (PDF) with known shapes, several approximations need to be made when the distribution is unknown. In this paper the sampling method, as well as the inference of the underlying distribution, are both handled with a data-driven method known as generative adversarial networks (GAN), which trains two competing neural networks to produce a network that can effectively generate samples from the training set distribution. In practice, it is often necessary to draw samples from conditional distributions. When the conditional variables are continuous, only one (if any) data point corresponding to a particular value of a conditioning variable may be available, which is not sufficient to estimate the conditional distribution. This work handles this problem using an a priori estimation of the conditional moments of a PDF. Two approaches, stochastic estimation, and an external neural network are compared here for computing these moments; however, any preferred method can be used. The algorithm is demonstrated in the case of the deconvolution of a filtered turbulent flow field. It is shown that all the versions of the proposed algorithm effectively sample the target conditional distribution with minimal impact on the quality of the samples compared to state-of-the-art methods. Additionally, the procedure can be used as a metric for the diversity of samples generated by a conditional GAN (cGAN) conditioned with continuous variables.

Motivation & Objective

• Address the challenge of sampling unknown, high-dimensional conditional distributions in engineering systems with continuous conditioning variables.
• Overcome the limitation of sparse data for continuous conditioning variables, where no exact match exists in training data.
• Improve the diversity of samples generated by conditional GANs without degrading sample quality.
• Enable accurate deconvolution of filtered turbulent flow fields using conditional generative modeling.
• Provide a metric for evaluating the diversity of conditional GAN-generated samples using estimated conditional moments.

Proposed method

• Estimate conditional moments of the unknown high-dimensional distribution using either a neural network or stochastic estimation (e.g., linear or quadratic approximation).
• Use the estimated moments as a regularization term in the GAN loss function to guide the generator toward producing diverse samples consistent with the conditioning variables.
• Integrate the conditional moment estimation into the training of a conditional GAN (cGAN) via a diversity loss function that penalizes deviation from the estimated moments.
• Enforce the generator to produce samples whose conditional moments match the estimated target moments using a moment-matching constraint.
• Balance the loss function across content, adversarial, and diversity terms to prevent any single component from dominating training.
• Apply the method to atmospheric flow deconvolution, using low-resolution observations (e.g., box-filtered fields) as conditioning variables to reconstruct high-resolution turbulent fields.

Experimental results

Research questions

• RQ1 Can conditional moments be reliably estimated from sparse, high-dimensional data when no exact match exists for a given conditioning variable?
• RQ2 How can conditional GANs be trained to generate diverse samples from unknown, high-dimensional conditional distributions with continuous conditioning variables?
• RQ3 To what extent does incorporating estimated conditional moments into the GAN loss improve sample diversity without degrading sample fidelity?
• RQ4 Can the proposed method serve as a reliable metric for evaluating the diversity of cGAN-generated samples?
• RQ5 How does the choice of moment estimation method (neural network vs. stochastic estimation) affect the quality and diversity of generated samples?

Key findings

• The proposed method successfully generates high-dimensional samples from unknown conditional distributions using only sparse, continuous conditioning variables.
• All versions of the algorithm—using neural network or stochastic moment estimation—produced deconvolved fields consistent with their convolved counterparts.
• GAN-based methods with moment regularization generated samples of significantly higher fidelity than baseline methods, with minimal degradation in quality.
• Sample diversity was consistently higher than in methods that encourage diversity without explicit moment guidance, regardless of the moment estimation technique.
• The method outperformed state-of-the-art approaches in both qualitative and quantitative assessments, particularly in capturing fine-scale turbulent structures.
• Loss balancing across content, adversarial, and diversity terms was critical to prevent dominance of one loss component and maintain sample consistency and diversity.