[论文解读] Neural Relational Inference with Fast Modular Meta-learning

Ferran Alet, Erica Weng+2 more|arXiv (Cornell University)|Oct 10, 2023

Domain Adaptation and Few-Shot Learning参考文献 19被引用 25

一句话总结

论文将神经关系推断重新表述为对GNN的模块化元学习，学习节点和边模组库，并使用快速的提案辅助搜索（BounceGrad）来推断关系结构，实现在未观测或潜在节点下数据高效、基于模型的推理与可扩展的计划。

ABSTRACT

extit{Graph neural networks} (GNNs) are effective models for many dynamical systems consisting of entities and relations. Although most GNN applications assume a single type of entity and relation, many situations involve multiple types of interactions. extit{Relational inference} is the problem of inferring these interactions and learning the dynamics from observational data. We frame relational inference as a extit{modular meta-learning} problem, where neural modules are trained to be composed in different ways to solve many tasks. This meta-learning framework allows us to implicitly encode time invariance and infer relations in context of one another rather than independently, which increases inference capacity. Framing inference as the inner-loop optimization of meta-learning leads to a model-based approach that is more data-efficient and capable of estimating the state of entities that we do not observe directly, but whose existence can be inferred from their effect on observed entities. To address the large search space of graph neural network compositions, we meta-learn a extit{proposal function} that speeds up the inner-loop simulated annealing search within the modular meta-learning algorithm, providing two orders of magnitude increase in the size of problems that can be addressed.

研究动机与目标

• 推断多关系系统的关系结构与动态，基于模块化元学习从观测中获得。
• 学习一个节点和边的神经模组库，可组合以建模各种交互类型。
• 通过将关系推断视为基于模型的方法的元学习，提高数据效率并实现对潜在/未观测实体的推断。
• 通过学习的提案函数来加速GNN结构的组合搜索，以便处理更大规模的问题。

提出的方法

• 将关系推断框架化为具有节点模组 m_i 与边模组 h_j 的模块化元学习。
• 使用 BounceGrad：在模块化结构的模拟退火与梯度下降之间交替，以优化模组权重。
• 通过在大量任务上学习模组权重进行元训练，在新任务上进行元测试以推断最佳结构 S。
• 元学习一个提案函数以加速内部循环的结构搜索，利用训练批次上的 SA 结果进行训练。
• 通过创建多任务的超图进行批量训练，以利用并行性并保持常数参数成本。
• 利用基于模型的推理，在内部循环中优化未观测实体的潜在状态以预测。

Figure 1: Modular meta-learning with graph networks; adapted from Alet et al. ( 2018 ) . The system meta-learns a library of node and edge modules, represented as small neural networks; at performance (meta-test) time, it is only necessary to infer the combination of modules that best predict the ob

实验结果

研究问题

• RQ1 模块化元学习结合模组库是否能够在从轨迹数据推断关系结构方面达到或超过变分NRI？
• RQ2 在边类型的结构级联合优化是否比独立边预测更有利于关系推断？
• RQ3 是否可以通过学习的提案函数显著加速内部循环搜索，以扩展到更大规模的任务分布和模组组合？
• RQ4 是否可以通过将潜在/未观测实体整合进内部循环优化来推断它们及其关系？

主要发现

• 具有联合推断边模组的模块化元学习，在 Springs 和 Charged-particles 数据集上达到与基线相当或更优的轨迹预测性能。
• 边类型预测精度接近黄金性能，模块化元学习在两个领域都达到或超过了前人方法。
• 基于模型的方法在数据效率方面比变分方法更高，尤在小数据情境下更明显。
• 学习的提案函数将内部循环搜索速度提升数量级，使更大模组空间和元数据集（高达 50,000 个任务）成为可能。
• 该框架通过在内部循环中优化潜在轨迹来推断潜在/未观测实体，从而提升预测准确性。
• 跨任务批处理并将计算映射到超图，显著提升训练和评估的速度。

Figure 2: Task setup, taken from Kipf et al. ( 2018 ) : we want to predict the evolution of a dynamical system by inferring the set of relations between the entities, such as attraction and repulsion between charged particles.