[论文解读] PDE-Net: Learning PDEs from Data

Zichao Long, Yiping Lu+2 more|arXiv (Cornell University)|Oct 26, 2017

Model Reduction and Neural Networks参考文献 5被引用 219

一句话总结

PDE-Net 通过受约束的可学习滤波器及神经网络来学习微分算子，以预测动力学并从数据中揭示潜在的PDE模型。

ABSTRACT

In this paper, we present an initial attempt to learn evolution PDEs from data. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models. The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses. Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses. A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network. These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory). We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet). Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

研究动机与目标

• 在有限先验知识的情况下，推动对复杂系统控制PDE的基于数据的发现。
• 开发一个能够同时预测动力学并揭示隐藏PDE形式的深度前馈网络。
• 通过可训练且受约束的卷积核以及非线性响应函数F，学习微分算子。

提出的方法

• 将PDEs表示为离散的、可学习的形式，其中 u_t = F(x,y,u, u_x, u_y, u_xx, u_xy, u_yy, ...)。
• 使用卷积核学习微分算子的离散化；用逐点神经网络学习F。
• 通过与和规则相关的矩矩阵对滤波器进行约束，以将学习的算子联系到微分算子。
• 叠加多个 delta-t 块以强制实现更长时间的稳定性并实现长期预测。
• 在层之间共享参数以减少内存并强化一致性。
• 初始化滤波器以对应已知的微分算子，并在训练过程中逐步放宽约束。

实验结果

研究问题

• RQ1 PDE-Net 能否在带有噪声的数据上，长期准确预测复杂动力学？
• RQ2 PDE-Net 能否从观测到的动态中揭示潜在的PDE结构（系数和算子）？
• RQ3 滤波器尺寸和网络深度如何影响预测稳定性和PDE识别？
• RQ4 通过矩矩阵对滤波器进行约束是否有助于可辨识控制PDE？

主要发现

• PDE-Net 实现了长期预测，超过 Frozen-PDE-Net，尤其是在使用更大的 7x7 滤波器时。
• 增加 delta-t 块的数量可提升长期预测的稳定性和准确性。
• 所学系数在线性问题上与真实PDE系数近似匹配，但因噪声存在一些振荡。
• 较大滤波器（7x7）相比较小滤波器（5x5）拓展了可可靠预测的时间窗。
• PDE-Net 能在线性测试中通过识别非出现项的近零系数来揭示隐藏的PDE。
• 分层训练、参数共享和滤波器约束提升学习效率与可辨识性。