[论文解读] NOMAD: Nonlinear Manifold Decoders for Operator Learning

Jacob H. Seidman, Georgios Kissas+2 more|arXiv (Cornell University)|Jun 7, 2022

Model Reduction and Neural Networks被引用 29

一句话总结

NOMAD 引入一个用于算子学习的非线性解码器，以捕捉函数空间中的低维非线性流形，在较小的潜在维度和训练成本下达到可比或更好的准确度。

ABSTRACT

Supervised learning in function spaces is an emerging area of machine learning research with applications to the prediction of complex physical systems such as fluid flows, solid mechanics, and climate modeling. By directly learning maps (operators) between infinite dimensional function spaces, these models are able to learn discretization invariant representations of target functions. A common approach is to represent such target functions as linear combinations of basis elements learned from data. However, there are simple scenarios where, even though the target functions form a low dimensional submanifold, a very large number of basis elements is needed for an accurate linear representation. Here we present NOMAD, a novel operator learning framework with a nonlinear decoder map capable of learning finite dimensional representations of nonlinear submanifolds in function spaces. We show this method is able to accurately learn low dimensional representations of solution manifolds to partial differential equations while outperforming linear models of larger size. Additionally, we compare to state-of-the-art operator learning methods on a complex fluid dynamics benchmark and achieve competitive performance with a significantly smaller model size and training cost.

研究动机与目标

• Motivate and address limitations of linear decoders in operator learning when target functions lie on nonlinear, low-dimensional manifolds.
• Propose a fully nonlinear decoder (NOMAD) to learn nonlinear embeddings in function spaces.
• Demonstrate that NOMAD can achieve state-of-the-art accuracy with significantly smaller latent dimensions and training cost across PDE-related benchmarks.

提出的方法

• Frame operator learning as learning a map G from input functions to output functions via a three-map architecture F = D ∘ A ∘ E.
• Use an encoder E to map input functions to finite-dimensional features, an approximator A to act on these features, and a nonlinear decoder D to map to output functions.
• Replace the linear decoder with a nonlinear decoder D(β, y) = f(β, y), where β ∈ R^n are latent coordinates and y ∈ Y is a query point, enabling nonlinear embedding of the output manifold.
• Ground the approach in the Operator Learning Manifold Hypothesis: the output manifold G(U) is well-approximated by an n-dimensional nonlinear manifold.
• Provide theoretical motivation showing linear decoders incur lower bounds tied to eigenvalue decay and Kolmogorov n-width, which NOMAD can overcome by nonlinear decoding.

实验结果

研究问题

• RQ1 Can a nonlinear decoder learn low-dimensional representations of nonlinear solution manifolds in function spaces more efficiently than linear decoders?
• RQ2 Does NOMAD reduce latent dimension, parameter count, and training cost while maintaining or improving predictive accuracy on operator learning tasks?
• RQ3 How does NOMAD perform on benchmark PDE/operator-learning tasks (antiderivative, advection, shallow-water) compared to linear decoders and state-of-the-art methods?
• RQ4 What are the trade-offs and limitations when replacing linear decoders with nonlinear decoders in operator learning architectures?

主要发现

方法	rho	v1	v2	最坏情况	d_theta	n	成本
LOCA	0.040±0.015	2.7±0.3	2.9±0.4	(0.1,3.5,4.2)	O(10^6)	480	12.1
DON	0.100±0.030	5.5±1.2	5.9±1.4	(0.6,11,11)	O(10^6)	480	15.4
FNO	0.140±0.060	3.4±1.2	3.5±1.2	(0.4,8.9,8.7)	O(10^6)	N/A	14.0
NOMAD	0.048±0.017	2.0±0.4	2.6±0.3	(0.1,5.8,4.9)	O(10^5)	20	5.5

• NOMAD consistently outperforms linear decoders on the antiderivative example, achieving about an order of magnitude better error and even 10% relative error with a single latent dimension.
• On a parametric advection PDE, NOMAD rapidly captures the low-dimensional nonlinear solution manifold where linear decoders struggle with small latent dimensions.
• For the shallow-water benchmark, NOMAD matches or closely approaches state-of-the-art methods while using markedly fewer trainable parameters, a smaller latent dimension, and shorter training time.
• Across benchmarks, NOMAD delivers competitive accuracy with substantially reduced model size and training cost compared to LOCA, DON, and FNO configurations.
• The results demonstrate that learning nonlinear coordinates with NOMAD enables efficient representation of nonlinear manifolds in output function spaces.