[Paper Review] A Robust SINDy Approach by Combining Neural Networks and an Integral Form

Ali Forootani, Pawan Goyal+1 more|arXiv (Cornell University)|Sep 13, 2023

Control Systems and Identification4 citations

TL;DR

This paper proposes iNeural-SINDy, a robust method for discovering nonlinear governing equations from noisy and scarce data by combining deep neural networks for denoising and derivative estimation via automatic differentiation, along with an integral form of differential equations to enhance stability. The approach significantly outperforms existing methods like DeePyMoD and Weak-SINDy, especially under high noise and low data regimes.

ABSTRACT

The discovery of governing equations from data has been an active field of research for decades. One widely used methodology for this purpose is sparse regression for nonlinear dynamics, known as SINDy. Despite several attempts, noisy and scarce data still pose a severe challenge to the success of the SINDy approach. In this work, we discuss a robust method to discover nonlinear governing equations from noisy and scarce data. To do this, we make use of neural networks to learn an implicit representation based on measurement data so that not only it produces the output in the vicinity of the measurements but also the time-evolution of output can be described by a dynamical system. Additionally, we learn such a dynamic system in the spirit of the SINDy framework. Leveraging the implicit representation using neural networks, we obtain the derivative information -- required for SINDy -- using an automatic differentiation tool. To enhance the robustness of our methodology, we further incorporate an integral condition on the output of the implicit networks. Furthermore, we extend our methodology to handle data collected from multiple initial conditions. We demonstrate the efficiency of the proposed methodology to discover governing equations under noisy and scarce data regimes by means of several examples and compare its performance with existing methods.

Motivation & Objective

• Address the challenge of discovering accurate governing equations from noisy and scarce time-series data in nonlinear dynamical systems.
• Overcome the sensitivity of traditional SINDy methods to derivative estimation errors in low signal-to-noise regimes.
• Develop a data-driven framework that leverages deep neural networks to implicitly represent clean dynamics and enable robust sparse regression.
• Extend the methodology to handle data from multiple initial conditions for richer dynamical system identification.
• Improve robustness by incorporating an integral form of differential equations to reduce reliance on noisy derivatives.

Proposed method

• Train a deep neural network (DNN) to learn an implicit, denoised representation of the observed data, enabling smooth and accurate time-series reconstruction.
• Use automatic differentiation on the trained DNN to compute derivative estimates without relying on numerical differentiation of raw noisy data.
• Formulate the sparse identification problem in the SINDy framework using the DNN-estimated derivatives and a large dictionary of nonlinear basis functions.
• Incorporate an integral form of the governing equations to stabilize the sparse regression process and reduce sensitivity to noise.
• Enable multi-trajectory learning by jointly training the DNN on data collected from different initial conditions.
• Introduce a scaling factor α to balance the loss terms between data fidelity and the integral constraint, improving optimization stability.

Experimental results

Research questions

• RQ1 Can a deep neural network-based implicit representation improve the robustness of SINDy in the presence of high noise and limited data?
• RQ2 How does the integration of an integral form of differential equations enhance the accuracy and stability of governing equation discovery compared to standard SINDy?
• RQ3 To what extent does the proposed iNeural-SINDy method outperform state-of-the-art alternatives like DeePyMoD and Weak-SINDy under noisy and scarce data conditions?
• RQ4 How does the choice of scaling factor α affect the performance of the proposed method across different noise levels and data sparsity?
• RQ5 Can the framework effectively learn governing equations from multiple trajectories with varying initial conditions?

Key findings

• iNeural-SINDy consistently outperforms DeePyMoD across multiple noise levels and sample sizes in the Lorenz system under both Scene_A and Scene_B configurations.
• For high noise levels (σ = 0.1), iNeural-SINDy achieves significantly better recovery of the true governing equations than Weak-SINDy, demonstrating superior robustness.
• The use of the integral form in combination with neural network-based derivative estimation reduces sensitivity to noise, especially when derivative estimation is unreliable.
• In scenarios with low data density, iNeural-SINDy maintains high accuracy, indicating strong generalization capability even with scarce measurements.
• The scaling factor α has a nuanced effect: while not universally beneficial, α = 0.1 tends to improve performance in deeper networks, suggesting a role in loss landscape optimization.
• The method successfully identifies governing equations from multiple initial conditions, enabling discovery of full system dynamics without requiring prior knowledge of the system's structure.