[论文解读] PINNACLE: PINN Adaptive ColLocation and Experimental points selection
PINNACLE 联合优化所有训练点类型的选择,使用基于 NTK 的收敛度,提升前向、逆向与迁移学习任务中的训练效率与泛化。
Physics-Informed Neural Networks (PINNs), which incorporate PDEs as soft constraints, train with a composite loss function that contains multiple training point types: different types of collocation points chosen during training to enforce each PDE and initial/boundary conditions, and experimental points which are usually costly to obtain via experiments or simulations. Training PINNs using this loss function is challenging as it typically requires selecting large numbers of points of different types, each with different training dynamics. Unlike past works that focused on the selection of either collocation or experimental points, this work introduces PINN Adaptive ColLocation and Experimental points selection (PINNACLE), the first algorithm that jointly optimizes the selection of all training point types, while automatically adjusting the proportion of collocation point types as training progresses. PINNACLE uses information on the interaction among training point types, which had not been considered before, based on an analysis of PINN training dynamics via the Neural Tangent Kernel (NTK). We theoretically show that the criterion used by PINNACLE is related to the PINN generalization error, and empirically demonstrate that PINNACLE is able to outperform existing point selection methods for forward, inverse, and transfer learning problems.
研究动机与目标
- Motivate the need for efficient training of PINNs with multiple point types (exp, collocation PDE, and IC/BC) under budget constraints.
- Propose a unified framework to jointly optimize all training point types rather than treating them separately.
- Leverage Neural Tangent Kernel analysis in an augmented input space to guide point selection.
- Define a convergence degree criterion that links point choice to training convergence and generalization bound.
提出的方法
- Introduce an augmented input space Z that jointly represents Exp, PDE collocation, and IC/BC collocation points.
- Model PINN training dynamics with a combined NTK that includes cross-type blocks (e.g., Theta_sp) to capture interactions between point types.
- Define convergence degree alpha(Z) as the RKHS norm of the expected residual update, relating point selection to generalization bound.
- Approximate the NTK eigenfunctions/eigenvalues via Nyström method to compute a practical convergence criterion.
- Propose two point-selection variants (Sampling and K-Means++) to maximize the approximate convergence degree under a pool of candidate points.
- Iteratively alternate between point selection and PINN training, re-evaluating the NTK as training progresses.
实验结果
研究问题
- RQ1Can joint optimization of Exp, PDE collocation, and IC/BC points improve PINN training efficiency under a fixed budget?
- RQ2Does maximizing the convergence degree alpha(Z) correlate with reduced generalization error for PINNs?
- RQ3How does cross-information between point types (captured in the augmented NTK) influence point selection and training dynamics?
- RQ4Can Nyström-based approximations provide reliable estimates for point selection in practice?
- RQ5Do PINNACLE variants outperform existing adaptive or non-adaptive point selection methods across forward, inverse, and transfer learning tasks?
主要发现
- PINNACLE outperforms benchmarks in forward problems (1D Advection and 1D Burgers) by achieving lower predictive loss and better qualitative fits.
- The method automatically adjusts the mix of PDE collocation and IC/BC points during training, aligning with intuitive heuristics.
- Cross-type NTK terms enable leveraging information across point types for more effective training.
- Nyström-based approximations provide a practical way to estimate the convergence degree and drive point selection.
- PINNACLE also demonstrates advantages in inverse problems (2D Navier–Stokes) and transfer learning with perturbed ICs, reducing training time.
- Theoretical connections show that maximizing the convergence degree can lead to lower generalization error bounds.
阅读本论文
在 Nubint 阅读器中打开原文,边读边就任意段落提问。
在 Nubint 中阅读本论文本解读由 AI 根据论文原文生成,引用前请核对原文。