[Paper Review] Online Weak-form Sparse Identification of Partial Differential Equations

Daniel A. Messenger, Emiliano Dall’Anese+1 more|PubMed|Mar 8, 2022

Model Reduction and Neural Networks13 references4 citations

TL;DR

This paper proposes an online weak-form sparse identification algorithm for partial differential equations (PDEs) that processes data snapshots sequentially using weak-form discretization and online proximal gradient descent with direct $oldsymbol{ ext{ℓ}_0}$-norm thresholding. The method enables robust, memory-efficient PDE discovery and tracking of time-varying coefficients in noisy, high-dimensional settings, successfully identifying systems like the Kuramoto-Sivashinsky and wave equations across 1D to 3D spatial domains with high accuracy even at 10% noise.

ABSTRACT

This paper presents an online algorithm for identification of partial differential equations (PDEs) based on the weak-form sparse identification of nonlinear dynamics algorithm (WSINDy). The algorithm is online in the sense that if performs the identification task by processing solution snapshots that arrive sequentially. The core of the method combines a weak-form discretization of candidate PDEs with an online proximal gradient descent approach to the sparse regression problem. In particular, we do not regularize the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>0</mml:mn></mml:msub></mml:mrow></mml:math>-pseudo-norm, instead finding that directly applying its proximal operator (which corresponds to a hard thresholding) leads to efficient online system identification from noisy data. We demonstrate the success of the method on the Kuramoto-Sivashinsky equation, the nonlinear wave equation with time-varying wavespeed, and the linear wave equation, in one, two, and three spatial dimensions, respectively. In particular, our examples show that the method is capable of identifying and tracking systems with coefficients that vary abruptly in time, and offers a streaming alternative to problems in higher dimensions.

Motivation & Objective

• Address the challenge of identifying PDEs from streaming, noisy spatiotemporal data with limited memory and computational resources.
• Enable real-time system identification and tracking of time-varying PDE coefficients in an online, incremental fashion.
• Overcome the limitations of batch methods in high-dimensional PDEs by reformulating the problem as a streaming optimization task.
• Develop a robust, memory-efficient alternative to batch WSINDy for online PDE discovery under noise and dynamic coefficient changes.
• Demonstrate the efficacy of directly applying the proximal operator of the ℓ₀-pseudo-norm (hard thresholding) in online sparse regression for PDEs.

Proposed method

• Formulates the PDE identification problem using weak-form discretization, replacing numerical differentiation with integration against test functions to improve noise robustness.
• Applies online proximal gradient descent to the least-squares cost function regularized with the ℓ₀-pseudo-norm, avoiding common ℓ₁-regularization heuristics.
• Uses the proximal operator of the ℓ₀-norm (hard thresholding) directly, which is shown to be effective for online sparse regression in noisy environments.
• Processes data snapshots sequentially at each timestep, maintaining only a fixed memory window $K_{ ext{mem}}$ of past snapshots to reduce storage and computational load.
• Adapts the regularization parameter $oldsymbol{ ext{λ}}_t$ dynamically during the optimization process to balance sparsity and accuracy.
• Employs convolutional operations to efficiently compute weak-form integrals, enabling scalability to higher-dimensional PDEs.

Experimental results

Research questions

• RQ1 Can online PDE identification be achieved with high accuracy using only a limited memory window of past data snapshots?
• RQ2 Does direct application of the ℓ₀-proximal operator (hard thresholding) outperform ℓ₁-regularization in online sparse regression for PDE discovery?
• RQ3 How well does the method perform in identifying PDEs with abruptly changing coefficients under high noise levels?
• RQ4 Can the method scale to higher-dimensional PDEs (e.g., 3D wave equation) with streaming data and limited memory?
• RQ5 To what extent does the memory window size $K_{ ext{mem}}$ affect the accuracy and stability of online PDE identification?

Key findings

• The method successfully identifies the Kuramoto-Sivashinsky equation with less than 1% average coefficient error across all noise levels up to 10% NR, even with $K_{ ext{mem}} = 13$.
• For the nonlinear wave equation with time-varying wavespeed, the algorithm accurately tracks coefficient changes and recovers the correct PDE structure with minimal error, even under 10% noise.
• In the 3D linear wave equation case, the method achieves coefficient accuracy to more than two significant digits for $K_{ ext{mem}} oldsymbol{ ext{≥q}} 17$ at 10% noise, with only 5% of trials showing spurious terms.
• The algorithm demonstrates robustness to noise: for $K_{ ext{mem}} oldsymbol{ ext{≥q}} 17$, coefficient recovery remains accurate at 10% noise across all test cases.
• The use of direct ℓ₀-proximal thresholding (hard thresholding) leads to superior performance compared to ℓ₁-regularization in online settings, particularly in handling abrupt coefficient changes.
• The method enables memory-efficient, streaming-based PDE identification, making it a viable alternative to batch processing for high-dimensional problems such as 3D PDEs.