[论文解读] SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability

Simone Venturi, Tiernan Casey|arXiv (Cornell University)|Apr 27, 2022

Machine Learning in Materials Science被引用 5

一句话总结

该论文提出了SVD-DeepONet和flexDeepONet，通过整合奇异值分解（SVD）和本征正交分解（POD）技术，提升了DeepONet的灵活性与可解释性。通过使用预变换网络分离刚体运动（如旋转、平移），flexDeepONet将可训练参数减少高达98.8%，并提升了泛化能力，在具有对称性的动力学问题中，以远少于原始DeepONet的参数量实现了高精度。

ABSTRACT

Deep operator networks (DeepONets) are powerful architectures for fast and accurate emulation of complex dynamics. As their remarkable generalization capabilities are primarily enabled by their projection-based attribute, we investigate connections with low-rank techniques derived from the singular value decomposition (SVD). We demonstrate that some of the concepts behind proper orthogonal decomposition (POD)-neural networks can improve DeepONet's design and training phases. These ideas lead us to a methodology extension that we name SVD-DeepONet. Moreover, through multiple SVD analyses, we find that DeepONet inherits from its projection-based attribute strong inefficiencies in representing dynamics characterized by symmetries. Inspired by the work on shifted-POD, we develop flexDeepONet, an architecture enhancement that relies on a pre-transformation network for generating a moving reference frame and isolating the rigid components of the dynamics. In this way, the physics can be represented on a latent space free from rotations, translations, and stretches, and an accurate projection can be performed to a low-dimensional basis. In addition to flexibility and interpretability, the proposed perspectives increase DeepONet's generalization capabilities and computational efficiencies. For instance, we show flexDeepONet can accurately surrogate the dynamics of 19 variables in a combustion chemistry application by relying on 95% less trainable parameters than the ones of the vanilla architecture. We argue that DeepONet and SVD-based methods can reciprocally benefit from each other. In particular, the flexibility of the former in leveraging multiple data sources and multifidelity knowledge in the form of both unstructured data and physics-informed constraints has the potential to greatly extend the applicability of methodologies such as POD and PCA.

研究动机与目标

• 通过将DeepONet与基于SVD的方法（如POD和PCA）关联，提升其可解释性与泛化能力。
• 解决DeepONet在表示具有旋转、平移和拉伸等对称性动力学时的低效问题。
• 开发一个预变换网络，用于生成移动参考系，以分离动力学中的刚体分量。
• 实现在无刚体运动干扰的潜在空间中，对物理规律进行精确的低维投影。
• 证明DeepONet可将POD和SVD-based方法的应用范围扩展至非固定网格与已知数据结构之外。

提出的方法

• 通过SVD-DeepONet将SVD与POD集成到DeepONet中，使用POD模态作为主干网络，分支网络学习系数。
• 对时间与场景聚合的快照矩阵应用SVD，以识别DeepONet在表示对称动力学时的低效性。
• 设计flexDeepONet，其包含一个预变换网络，用于学习移动参考系，以解耦刚体运动与形变。
• 在包含旋转、平移和拉伸的动力学上训练该架构，以在无对称性的潜在空间中隔离物理规律。
• 利用移动参考系实现在低维基底上的精确投影，从而提升效率与泛化能力。
• 在ODE和2D刚体动力学问题上验证该方法，与原始DeepONet对比参数量与精度。

实验结果

研究问题

• RQ1 SVD与POD技术如何提升DeepONet在表示复杂动力系统时的可解释性与泛化能力？
• RQ2 为何DeepONet在表示如旋转和平移等对称性动力学时表现不佳？
• RQ3 一个学习移动参考系的预变换网络是否能增强DeepONet对对称性动力学的建模能力？
• RQ4 在非网格对齐与非时间均匀的数据中，POD等基于SVD的方法通过DeepONet可被扩展到何种程度？
• RQ5 在对称系统中，flexDeepONet架构相较于原始DeepONet在参数效率与预测精度方面表现如何？

主要发现

• 在2D刚体动力学问题中，flexDeepONet相比原始DeepONet将可训练参数减少了98.8%，仅使用1,921个参数。
• 该方法即使在空间训练窗口之外也能实现高精度的动力学重建，表现出显著的泛化能力。
• 在燃烧化学应用中，flexDeepONet相比原始DeepONet仅需95%的参数量，同时在19个热力学变量上保持了高精度。
• SVD分析表明，DeepONet基于投影的设计在表示对称动力学时效率低下，这促使了flexDeepONet的开发。
• 预变换网络所生成的移动参考系，使得物理规律能够被准确地表示为无刚体分量的低维投影。
• 该方法通过事后分析移动参考系的坐标，增强了可解释性。