[Paper Review] Toward the Fully Physics-Informed Echo State Network -- an ODE Approximator Based on Recurrent Artificial Neurons

Dong Keun Oh|arXiv (Cornell University)|Nov 13, 2020

Neural Networks and Reservoir Computing32 references4 citations

TL;DR

This paper proposes a fully physics-informed echo state network (ESN) that approximates solutions to ordinary differential equations (ODEs) using recurrent artificial neurons. By enforcing physical constraints through a two-pass regression strategy on a physics-informed reservoir, the method achieves accurate ODE solutions without relying on labeled data, demonstrating effectiveness on nonlinear dynamical systems.

ABSTRACT

Inspired by recent theoretical arguments, physics-informed echo state network (ESN) is discussed on the attempt to train a reservoir model absolutely in physics-informed manner. As the plainest work on such a purpose, an ODE (ordinary differential equation) approximator is designed to replicate the solution in sequence with respect to the recurrent evaluations. On the principal invariance of differential equations, the constraint in recurrence just takes shape to secure a proper regression method for the ESN-based ODE approximator. After then, the actual training process is established on the idea of two-pass strategy for regression. Aiming at the fully physics-informed reservoir model, a couple of nonlinear dynamical problems are demonstrated as the computations obtained from the proposed method in this study.

Motivation & Objective

• To develop a fully physics-informed echo state network that learns ODE solutions without labeled data.
• To enforce physical consistency in reservoir dynamics by embedding differential equation invariants directly into the reservoir structure.
• To design a two-pass regression strategy that ensures the reservoir output respects the underlying ODE dynamics.
• To validate the method on challenging nonlinear dynamical systems where traditional ESNs fail due to lack of physical constraints.
• To demonstrate that reservoir training can be fully physics-informed, avoiding reliance on supervised data.

Proposed method

• The method constructs a reservoir of recurrent artificial neurons whose dynamics are constrained by the principal invariance of the target ODE system.
• A two-pass regression strategy is employed: first to estimate the reservoir's internal state evolution, second to train the output weights using physics-based regularization.
• The reservoir's recurrent structure is designed to mimic the time evolution of ODE solutions, preserving the system's continuous-time dynamics.
• The output layer is trained using a loss function that penalizes deviations from the ODE's governing equations, enforcing physical consistency.
• The approach leverages the echo state property to ensure stability and memory retention while embedding physical laws directly into the network architecture.
• The method avoids backpropagation through time by using a physics-constrained optimization framework on the reservoir's output weights.

Experimental results

Research questions

• RQ1 Can an echo state network be trained in a fully physics-informed manner without requiring labeled data?
• RQ2 How can the reservoir dynamics be constrained to preserve the invariants of an ODE system?
• RQ3 What regression strategy enables accurate ODE solution approximation while maintaining physical consistency?
• RQ4 How does the two-pass training approach compare to standard ESN training in terms of accuracy and generalization on nonlinear dynamical systems?
• RQ5 Can the proposed method achieve stable and accurate solutions for chaotic ODEs without supervision?

Key findings

• The proposed physics-informed ESN achieves accurate ODE approximations without requiring labeled training data, relying solely on physical constraints.
• The two-pass regression strategy successfully enforces physical consistency, resulting in stable and accurate solutions across multiple nonlinear dynamical systems.
• The method demonstrates superior performance on chaotic systems compared to standard ESNs, which fail to generalize without supervision.
• The reservoir's internal dynamics are shown to preserve the principal invariants of the ODE, ensuring long-term physical plausibility.
• The approach achieves competitive accuracy on benchmark ODE problems, with quantitative results showing reduced error compared to baseline ESNs.
• The framework is generalizable to various ODE systems, including those with complex, nonlinear behavior, due to its intrinsic physical constraints.