[Paper Review] Learning Algorithms for Coarsening Uncertainty Space and Applications to Multiscale Simulations

Zecheng Zhang, Eric T. Chung+2 more|arXiv (Cornell University)|Apr 9, 2020

Advanced Mathematical Modeling in Engineering47 references4 citations

TL;DR

This paper proposes a deep learning-based method to coarsen uncertainty space in multiscale stochastic PDEs, using convolutional neural networks with adversarial training and multi-component loss functions to learn clusters of permeability fields. The approach reduces computational cost by enabling fast basis function selection across realizations, achieving near-optimal accuracy with 11 clusters—matching traditional GMsFEM performance while accelerating training.

ABSTRACT

In this paper, we investigate and design multiscale simulations for stochastic multiscale PDEs. As for the space, we consider a coarse grid and a known multiscale method, the Generalized Multiscale Finite Element Method (GMsFEM). In order to obtain a small dimensional representation of the solution in each coarse block, the uncertainty space needs to be partitioned (coarsened). This coarsening collects realizations that provide similar multiscale features as outlined in GMsFEM (or other method of choice). This step is known to be computationally demanding as it requires many local solves and clustering based on them. In this paper, we take a different approach and learn coarsening the uncertainty space. Our methods use deep learning techniques in identifying clusters(coarsening) in the uncertainty space. We use convolutional neural networks combined with some techniques in adversary neural networks. We define appropriate loss functions in the proposed neural networks, where the loss function is composed of several parts that includes terms related to clusters and reconstruction of basis functions. We present numerical results for channelized permeability fields in the examples of flows in porous media.

Motivation & Objective

• Address the high computational cost of generating multiscale basis functions for many stochastic realizations in porous media flows.
• Overcome the limitations of traditional clustering methods that rely on expensive local solves and distance metrics between solutions.
• Develop a data-driven, learnable framework to coarsen uncertainty space by identifying representative clusters of heterogeneous permeability fields.
• Integrate multiscale basis functions as supervision in the learning process to ensure clusters reflect physical solution features.
• Enable fast simulation across realizations by pre-computing basis functions for representative clusters, reducing online computational cost.

Proposed method

• Use a conditional generative model where the input is a permeability field and the output is the corresponding multiscale basis functions.
• Design a multi-component loss function combining reconstruction loss, clustering loss, and adversarial loss to guide the latent space toward physically meaningful clusters.
• Employ a U-Net-like encoder-decoder architecture with convolutional layers to extract features and reconstruct basis functions from the latent space.
• Incorporate an adversarial network to improve the quality of generated basis functions by encouraging realism through a discriminator network.
• Apply transfer learning by initializing the network for each coarse block using parameters from a reference block, significantly reducing training time.
• Train the model end-to-end using stochastic gradient descent, optimizing for both accurate basis reconstruction and effective clustering of input permeability fields.

Experimental results

Research questions

• RQ1 Can deep learning be used to coarsen the uncertainty space in multiscale stochastic PDEs without explicit clustering or local solves?
• RQ2 How does incorporating multiscale basis functions as supervision improve the quality of learned clusters in the uncertainty space?
• RQ3 To what extent does adversarial training enhance the fidelity of the learned multiscale basis functions compared to standard autoencoder training?
• RQ4 How does the number of clusters affect the accuracy of the reduced-order model in GMsFEM simulations?
• RQ5 Can transfer learning across coarse blocks significantly reduce training time while maintaining solution accuracy?

Key findings

• The $L_2$ relative error in the GMsFEM solution decreases as the number of clusters increases, with a notable drop from 5 to 11 clusters.
• With 11 clusters, the cluster-based method achieves $L_2$ relative error comparable to the traditional GMsFEM method using a specific realization, indicating negligible loss in accuracy.
• The inclusion of the adversarial loss reduces the MSE between the learned and real multiscale basis functions from 0.9073400 to 0.748312, improving basis fidelity.
• The $L_2$ relative error improves from 41.120439 to 36.760918 when adversarial training is applied, demonstrating the effectiveness of the generative loss.
• The use of transfer learning enables faster training across coarse blocks by initializing networks with parameters from a reference block, reducing computational overhead.
• The proposed method enables fast simulation across many realizations by pre-computing basis functions for representative clusters, significantly reducing online computational cost.