[Paper Review] FiniteNet: A Fully Convolutional LSTM Network Architecture for Time-Dependent Partial Differential Equations

Ben Stevens, Tim Colonius|arXiv (Cornell University)|Feb 7, 2020

Model Reduction and Neural Networks38 references31 citations

TL;DR

FiniteNet uses a fully convolutional LSTM to augment finite-difference/finite-volume PDE solvers, achieving 2–3x error reduction across linear advection, inviscid Burgers’, and Kuramoto–Sivashinsky equations.

ABSTRACT

In this work, we present a machine learning approach for reducing the error when numerically solving time-dependent partial differential equations (PDE). We use a fully convolutional LSTM network to exploit the spatiotemporal dynamics of PDEs. The neural network serves to enhance finite-difference and finite-volume methods (FDM/FVM) that are commonly used to solve PDEs, allowing us to maintain guarantees on the order of convergence of our method. We train the network on simulation data, and show that our network can reduce error by a factor of 2 to 3 compared to the baseline algorithms. We demonstrate our method on three PDEs that each feature qualitatively different dynamics. We look at the linear advection equation, which propagates its initial conditions at a constant speed, the inviscid Burgers' equation, which develops shockwaves, and the Kuramoto-Sivashinsky (KS) equation, which is chaotic.

Motivation & Objective

• Motivate reducing numerical error in time-dependent PDE solvers.
• Propose a neural architecture that jointly leverages spatial discretization and temporal dynamics.
• Maintain convergence guarantees while learning from simulation data.
• Demonstrate error reduction across multiple qualitatively different PDEs (linear advection, Burgers’, KS).

Proposed method

• Adopt a fully convolutional LSTM that mirrors finite-difference/finite-volume stencils to compute spatial derivatives.
• Use an LSTM at each grid location to propagate information across time steps.
• Learn perturbations to maximum-order discretization coefficients (ĉ -> ĉ̂ + Δĉ) with L2 regularization, followed by an affine transform to enforce order accuracy.
• Train end-to-end by simulating forward in time and minimizing long-horizon simulation error against exact or high-fidelity solutions.
• Ensure numerical stability and convergence guarantees by constraining learned coefficients to satisfy prescribed accuracy conditions (via a closed-form Δĉ computation).

Experimental results

Research questions

• RQ1 Can FiniteNet reduce discretization error relative to baseline FDM/FVM methods across varied PDE dynamics?
• RQ2 Does integrating LSTM-based temporal memory with PDE-inspired spatial discretization preserve known convergence rates?
• RQ3 How does FiniteNet perform on problems with discontinuities (shockwaves) and chaotic dynamics (KS equation) compared to standard solvers?

Key findings

• FiniteNet reduces error by factors of 2–3 compared with baseline methods across three PDEs.
• For linear advection and inviscid Burgers’, FiniteNet sharper resolves discontinuities, with occasional small oscillations but overall lower error.
• For Kuramoto–Sivashinsky, FiniteNet improves tracking of chaotic trajectories and exhibits lower mean and variance in error than FDM.
• Across test cases, FiniteNet demonstrates empirical stability and provides more reliable performance on chaotic dynamics than traditional methods.
• Hyperparameters initializations from the linear advection task generalized with modest tuning to Burgers’ and KS equations.