[Paper Review] Multiwavelet-based Operator Learning for Differential Equations

Gaurav Gupta, Xiongye Xiao+1 more|arXiv (Cornell University)|Sep 28, 2021

Image and Signal Denoising Methods61 references67 citations

TL;DR

Introduces a multiwavelet-based neural operator that compresses the kernel of PDE operators into fixed multiwavelet bases, achieving high accuracy and resolution-independence across KdV, Burgers, Darcy flow, and Navier–Stokes equations.

ABSTRACT

The solution of a partial differential equation can be obtained by computing the inverse operator map between the input and the solution space. Towards this end, we introduce a extit{multiwavelet-based neural operator learning scheme} that compresses the associated operator's kernel using fine-grained wavelets. By explicitly embedding the inverse multiwavelet filters, we learn the projection of the kernel onto fixed multiwavelet polynomial bases. The projected kernel is trained at multiple scales derived from using repeated computation of multiwavelet transform. This allows learning the complex dependencies at various scales and results in a resolution-independent scheme. Compare to the prior works, we exploit the fundamental properties of the operator's kernel which enable numerically efficient representation. We perform experiments on the Korteweg-de Vries (KdV) equation, Burgers' equation, Darcy Flow, and Navier-Stokes equation. Compared with the existing neural operator approaches, our model shows significantly higher accuracy and achieves state-of-the-art in a range of datasets. For the time-varying equations, the proposed method exhibits a ($2X-10X$) improvement ($0.0018$ ($0.0033$) relative $L2$ error for Burgers' (KdV) equation). By learning the mappings between function spaces, the proposed method has the ability to find the solution of a high-resolution input after learning from lower-resolution data.

Motivation & Objective

• Motivate data-efficient learning of PDE operators by exploiting kernel properties.
• Develop a multiwavelet-based representation to sparsify and project operator kernels onto fixed bases.
• Create a scalable, resolution-independent neural operator architecture leveraging multiwavelet transforms.
• Demonstrate robustness and generalization across 1D and 2D PDEs and across varying input resolutions.

Proposed method

• Model kernels via multiwavelet transforms with non-uniform measures to handle arbitrary bases.
• Use a non-standard multiwavelet representation to decouple scales and reduce model complexity.
• Implement a two-part MWT architecture with Decomposition (dec) and Reconstruction (rec) modules.
• Train four neural networks A, B, C, and T (barT) to approximate the non-standard kernel terms across scales.
• Exploit fixed multiwavelet filters H and G to perform forward/backward transforms between scales.
• Demonstrate resolution-independence by reusing learned kernels across different input sizes.

Experimental results

Research questions

• RQ1 Can multiwavelet-based kernel representations provide data-efficient, compression-friendly operator learning for PDEs?
• RQ2 Do fixed multiwavelet filters enable scale-wise learning that generalizes across resolutions?
• RQ3 How does the MWT model perform compared with state-of-the-art neural operators on standard PDE benchmarks?
• RQ4 Is the approach robust to input fluctuations and applicable to higher-dimensional problems?
• RQ5 Can learning at lower resolutions generalize to high-resolution outputs?

Key findings

• MWT Leg and MWT Chb achieve the lowest relative L2 errors across KdV, Burgers, Darcy, and Navier–Stokes benchmarks, outperforming FNO, MGNO, LNO, and GNO.
• For time-varying equations, the method shows a 2X–10X improvement in relative L2 error (e.g., 0.0018 vs 0.0033 for Burgers and KdV).
• The model remains robust to input fluctuation strength and works with lower-resolution training data while generalizing to higher resolutions.
• Using fixed multiwavelet filters is crucial; random filters substantially degrade performance (MWT Rnd performs poorly).
• The approach validates theoretical properties of pseudo-differential operators and leverages kernel sparsity in the multiwavelet domain.
• The method extends to 2D Darcy flow and can learn at coarse resolutions while predicting fine-scale outputs; shows potential for high-resolution prediction without retraining.