[论文解读] Deep Learning and Quantum Physics : A Fundamental Bridge

Yoav Levine, David Yakira+2 more|arXiv (Cornell University)|Apr 5, 2017

Quantum Computing Algorithms and Architecture被引用 1

一句话总结

该论文通过其共享的张量网络结构，建立了深度卷积算术电路（ConvACs）与量子多体波函数之间的基本等价性，使量子纠缠度量能够量化网络的表达能力。研究证明，每一层的通道数通过图论中的最小割分析直接控制归纳偏置，为深度网络提供了原则性的设计框架。

ABSTRACT

Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

研究动机与目标

• 揭示深度卷积网络中归纳偏置的理论基础。
• 通过张量网络表示，建立深度学习与量子多体物理之间的正式联系。
• 证明卷积层中的通道数通过图论最小割分析直接控制归纳偏置。
• 提供一种基于表达能力与相关性建模的、原则性且可量化的深度网络设计方法。

提出的方法

• 将深度ConvACs形式化为张量网络，以利用其与量子波函数共享的数学结构。
• 使用量子纠缠度量（如纠缠熵）作为网络建模复杂输入相关性的能力的量化指标。
• 将网络架构表示为图，并通过其最小割结构分析，将通道数与归纳偏置联系起来。
• 应用图论工具，证明通道数可直接控制网络的表达能力。
• 通过实证分析，在具有ReLU和最大池化的标准ConvNets上验证理论发现。
• 利用张量网络框架，统一深度学习架构设计与量子启发的形式化方法。

实验结果

研究问题

• RQ1 深度卷积网络每一层的通道数在多大程度上影响其归纳偏置？
• RQ2 深度ConvACs与量子多体波函数之间存在何种正式的数学等价性？
• RQ3 量子纠缠度量能否作为深度网络表达能力的有意义量化指标？
• RQ4 深度网络的图结构如何与其建模复杂输入相关性的能力相关联？
• RQ5 对网络张量网络图的最小割分析在多大程度上可预测归纳偏置？

主要发现

• 由于共享的张量网络结构，深度ConvAC在数学上等价于量子多体波函数。
• 量子纠缠度量（如纠缠熵）可为网络建模输入相关性的表达能力提供明确的量化指标。
• 每一层的通道数通过其与底层图结构中最小割的关系，直接控制网络的归纳偏置。
• 张量网络表示使图论分析成为可能，从而将架构设计与表达能力联系起来。
• 实证验证在具有ReLU和最大池化的标准ConvNets上确认了理论预测，与所提出的框架保持一致。
• 深度学习与量子物理之间的正式联系为理解与设计深度网络提供了新的跨学科视角。