[Paper Review] Multilevel CNNs for Parametric PDEs

Cosmas Heiß, Ingo Gühring+1 more|arXiv (Cornell University)|Apr 1, 2023

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes Multilevel Convolutional Neural Networks (ML-Nets) that integrate multigrid solvers with deep learning to efficiently solve high-dimensional parametric PDEs. By approximating multigrid V-cycles with CNNs, the method achieves approximation error bounds independent of parameter dimension, with network size scaling logarithmically in mesh resolution, significantly outperforming state-of-the-art deep learning solvers on benchmark problems including high-dimensional random conductivity and the random cookie problem.

ABSTRACT

We combine concepts from multilevel solvers for partial differential equations (PDEs) with neural network based deep learning and propose a new methodology for the efficient numerical solution of high-dimensional parametric PDEs. An in-depth theoretical analysis shows that the proposed architecture is able to approximate multigrid V-cycles to arbitrary precision with the number of weights only depending logarithmically on the resolution of the finest mesh. As a consequence, approximation bounds for the solution of parametric PDEs by neural networks that are independent on the (stochastic) parameter dimension can be derived. The performance of the proposed method is illustrated on high-dimensional parametric linear elliptic PDEs that are common benchmark problems in uncertainty quantification. We find substantial improvements over state-of-the-art deep learning-based solvers. As particularly challenging examples, random conductivity with high-dimensional non-affine Gaussian fields in 100 parameter dimensions and a random cookie problem are examined. Due to the multilevel structure of our method, the amount of training samples can be reduced on finer levels, hence significantly lowering the generation time for training data and the training time of our method.

Motivation & Objective

• To address the computational intractability of high-dimensional parametric PDEs in uncertainty quantification, especially when parameter dimensions are large.
• To overcome limitations of traditional reduced-order and stochastic collocation methods that degrade with increasing parameter dimensionality.
• To develop a deep learning architecture that inherits the efficiency and error control of multigrid methods while maintaining the expressivity of neural networks.
• To derive approximation bounds for neural network solutions that are independent of the stochastic parameter dimension.
• To reduce training data and time requirements by leveraging a hierarchical multilevel structure with level-dependent sampling.

Proposed method

• The method constructs a multilevel architecture based on a hierarchy of finite element spaces, where each level corresponds to a refined mesh discretization.
• It models multigrid V-cycles as a sequence of downsampling, smoothing, restriction, prolongation, and correction operations, all approximated by convolutional neural networks.
• The network architecture, termed ML-Net, concatenates CNNs that approximate each multigrid component: diffusion coefficient downsampling, residual correction, and prolongation operators.
• Theoretical analysis shows that any multigrid V-cycle can be approximated by a CNN with error bounded by ε‖f‖*, where ε controls accuracy and f is the source term.
• The number of network weights scales logarithmically with the inverse of the desired accuracy and quadratically with the number of levels, ensuring efficiency in high dimensions.
• The method exploits the fact that correction terms on finer levels contain less information, allowing reduced sampling on higher levels to lower data generation and training costs.

Experimental results

Research questions

• RQ1 Can a deep neural network architecture be constructed to approximate multigrid V-cycles with arbitrary precision using a number of parameters that scales favorably with resolution?
• RQ2 Can such a network achieve approximation bounds for parametric PDE solutions that are independent of the parameter dimension?
• RQ3 Does the multilevel structure enable significant reductions in training data and computation time for high-dimensional problems?
• RQ4 How does the expressivity of the proposed ML-Net compare to state-of-the-art deep learning solvers for parametric PDEs?
• RQ5 Can the network effectively handle non-affine, high-dimensional random fields such as 100-dimensional Gaussian conductivity fields?

Key findings

• The proposed ML-Net architecture can approximate multigrid V-cycles to within arbitrary precision ε‖f‖*, with the number of network weights scaling as O(L log(1/ε) + L²), where L is the number of levels.
• The method achieves approximation error bounds for parametric PDE solutions that are independent of the parameter dimension, enabling efficient solution of high-dimensional problems.
• On a 100-dimensional random conductivity problem with non-affine Gaussian fields, the method significantly outperforms existing deep learning solvers in accuracy and generalization.
• For the challenging random cookie problem, the method maintains high accuracy while reducing training data and time due to level-dependent sampling on finer grids.
• Theoretical analysis confirms that the network’s expressivity is sufficient to represent the solution map of parametric PDEs with optimal convergence rates.
• Empirical results show that the multilevel structure allows for a substantial reduction in training data generation time and training duration, especially on fine levels.