[Paper Review] Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems

Derick Nganyu Tanyu, Jianfeng Ning+5 more|arXiv (Cornell University)|Dec 6, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper evaluates deep learning methods for solving partial differential equations (PDEs) and identifying related parameters, comparing architectures including FNO, MWT, PINO, and DeepONet. It demonstrates that MWT and FNO achieve the lowest errors across various noise levels, with MWT showing superior robustness and generalization in parameter identification tasks.

ABSTRACT

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than standard feed-forward neural networks, recurrent neural networks, or convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering. In this work, we review such methods as well as their extensions for parametric studies and for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

Motivation & Objective

• To compare the performance of deep learning models in solving PDEs and identifying unknown parameters under varying noise levels.
• To assess the robustness and generalization of neural network architectures in solving inverse problems governed by PDEs.
• To evaluate the impact of noise (0%, 0.1%, 1%, 5%, 10%) on model accuracy and stability.
• To identify which deep learning architectures offer the most accurate and reliable solutions for PDE-constrained parameter identification.
• To provide empirical benchmarks for emerging PDE-aware neural networks in realistic noisy settings.

Proposed method

• Employs a suite of deep learning models: 3PCANN, PCALin, FNO, U-FNO, MWT, DeepONet, PINO, and PI-DeepONet for solving PDEs and parameter identification.
• Uses a standardized benchmark setup with synthetic PDE problems and controlled noise injection (0%, 0.1%, 1%, 5%, 10%) to evaluate model performance.
• Measures generalization and robustness via mean squared error (MSE) across multiple noise levels and repeated trials.
• Applies MWT (Multi-Weight Transformer) and FNO (Fourier Neural Operator) as state-of-the-art architectures for learning operators on function spaces.
• Uses PINO (Physics-Informed Neural Operator) and PI-DeepONet for physics-informed learning, embedding PDE constraints directly into the model.
• Compares models across both forward and inverse PDE problems, focusing on parameter recovery accuracy.

Experimental results

Research questions

• RQ1 Which deep learning architecture achieves the lowest error in solving PDEs under increasing noise levels (0% to 10%)?
• RQ2 How do physics-informed models like PINO and PI-DeepONet compare to purely data-driven models in parameter identification tasks?
• RQ3 Does the MWT architecture outperform other neural operators in terms of robustness to noise in PDE solutions?
• RQ4 How does the performance of FNO scale with increasing noise in the training and inference data?
• RQ5 What is the relative stability and generalization capability of each model across different noise regimes?

Key findings

• MWT achieves the lowest error across all noise levels, with 0.0129 MSE at 0% noise, improving to 0.0276 at 10% noise.
• FNO demonstrates strong performance, achieving 0.0210 MSE at 0% noise and 0.0421 at 10% noise, outperforming most models.
• PCALin shows consistent performance across noise levels, with MSE values ranging from 0.0923 to 0.0929, indicating high stability but higher error than MWT and FNO.
• PINO achieves low error (0.0167 at 0% noise), improving to 0.0349 at 10% noise, showing strong robustness.
• PI-DeepONet exhibits the highest error, with 0.0586 at 0% noise and 0.0677 at 10% noise, indicating poor generalization under noise.
• MWT and FNO consistently outperform other models, with MWT showing the best overall performance and noise resilience.