[Paper Review] Multipole Graph Neural Operator for Parametric Partial Differential Equations

Zongyi Li, Nikola Kovachki+5 more|arXiv (Cornell University)|Jun 16, 2020

Model Reduction and Neural Networks47 references159 citations

TL;DR

MGKN introduces a multi-level graph neural operator inspired by fast multipole methods to learn discretization-invariant solution operators for parametric PDEs with linear time complexity.

ABSTRACT

One of the main challenges in using deep learning-based methods for simulating physical systems and solving partial differential equations (PDEs) is formulating physics-based data in the desired structure for neural networks. Graph neural networks (GNNs) have gained popularity in this area since graphs offer a natural way of modeling particle interactions and provide a clear way of discretizing the continuum models. However, the graphs constructed for approximating such tasks usually ignore long-range interactions due to unfavorable scaling of the computational complexity with respect to the number of nodes. The errors due to these approximations scale with the discretization of the system, thereby not allowing for generalization under mesh-refinement. Inspired by the classical multipole methods, we propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Our multi-level formulation is equivalent to recursively adding inducing points to the kernel matrix, unifying GNNs with multi-resolution matrix factorization of the kernel. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.

Motivation & Objective

• Motivate fast, data-driven learning of solution operators for parametric PDEs beyond fixed discretizations.
• Overcome long-range interaction limitations in standard GNNs by introducing a multi-scale, linear-time framework.
• Unify graph neural networks with multi-resolution matrix factorization to enable discretization-invariant operator learning.

Proposed method

• Model the PDE solution operator as a kernel-based graph operator learned with a kernel network.
• Introduce inducing points to capture long-range interactions and lift to multi-level graphs.
• Decompose the kernel into ranges using a fast multipole-inspired hierarchy and apply a V-cycle to compute the multi-resolution factorization.
• Use Nyström approximation to achieve linear or near-linear complexity.
• Train multiple kernel networks for intra-level and inter-level transitions across L graph levels.
• Demonstrate discretization invariance and linear-time evaluation on Darcy flow and Burgers equations.

Experimental results

Research questions

• RQ1 Can MGKN learn discretization-invariant (mesh-invariant) solution operators for parametric PDEs?
• RQ2 Does the multi-level, multipole-inspired graph architecture yield linear computational complexity with respect to the number of nodes?
• RQ3 How does MGKN perform on linear and nonlinear PDEs with long-range correlations compared to baselines?
• RQ4 What is the impact of the number of levels L on accuracy and efficiency?
• RQ5 To what extent can Nyström-induced approximations preserve operator accuracy across resolutions?

Key findings

• MGKN achieves linear time complexity in the number of nodes, outperforming quadratic-complexity baselines.
• Adding more graph levels reduces test error on Darcy flow, improving accuracy without substantial time-cost increases.
• MGKN trained on a coarse grid can generalize to finer grids, demonstrating discretization invariance (super-resolution capability).
• MGKN provides competitive or superior performance to benchmarks on Burgers’ equation, especially when linear spaces are insufficient.
• Orthogonal kernel decompositions tend to yield better performance in the tested Darcy flow setting.