[论文解读] Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

Michael M. Bronstein, Joan Bruna+2 more|arXiv (Cornell University)|Apr 27, 2021

Graph Theory and Algorithms参考文献 255被引用 551

一句话总结

一份综述，通过对称性、表示和不变性在图、网格、群、流形和丛等领域框架中来界定神经网络，从而统一几何深度学习。

ABSTRACT

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

研究动机与目标

• 通过利用对称性与不变性，激发并形式化对深度学习的几何视角。
• 将几何先验作为归纳偏置引入，以对抗高维数据中的维度灾难。
• 为跨越多样几何领域（图、网格、群、流形、丛）的体系结构提供统一的蓝图。
• 解释表示、自同构和形变如何影响模型设计与学习。
• 指导从对称性第一原理推导未来几何模型的体系结构。

提出的方法

• 提出一个基于群作用与表示的、受 Erlangen 程序启发的几何深度学习正式框架。
• 刻画相对于对称群的函数的不变性与等变性。
• 将尺度分离与形变稳定性作为塑造架构的几何先验进行讨论。
• 综述几何领域（图、网格、群、流形）及其相应的神经网络架构。
• 以统一的方式描述卷积、池化与等变消息传递等模型构件。
• 说明规范/丛理论概念如何在几何情境中指导架构设计。

实验结果

研究问题

• RQ1 对称性、群表示和不变性的原理如何在多样几何域中统一并引导神经网络的设计？
• RQ2 哪些几何先验（对称性、尺度分离、形变稳定性）能够在高维结构化数据中提升学习？
• RQ3 如卷积神经网络（CNNs）、图神经网络（GNNs）和变换器（Transformers）如何从几何和对称性的第一原理推导？
• RQ4 流形、测地线、规范和丛在塑造几何深度学习模型中扮演何种角色？
• RQ5 如何系统地在网格、图、群和流形之间转移先验以构建有原理的模型？

主要发现

• 通过对称性和表示理论提供一个统一框架，将 CNNs、GNNs、Transformers 及相关模型联系起来。
• 阐明相对于数据域上群作用的函数的不变性与等变性属性。
• 强调几何先验如尺度分离和形变稳定性在结构化域学习中的重要性。
• 解释不同几何领域（图、网格、群、流形）如何从第一原理构建相应的神经架构。
• 描述规范和丛概念如何在流形及更复杂领域中的网络设计中起作用。