[论文解读] Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design

Yoav Levine, David Yakira+2 more|arXiv (Cornell University)|Apr 5, 2017

Quantum Computing Algorithms and Architecture参考文献 32被引用 35

一句话总结

该论文通过其共享的张量网络结构，建立了深度卷积算术电路（ConvAC）与量子多体波函数之间的基本等价性。通过利用量子纠缠测度作为相关性建模的量化指标，作者表明，每一层的通道数通过底层图中的最小割（min-cut）控制网络的归纳偏置，从而为设计具有表现力的深层网络提供了原则性框架。

ABSTRACT

Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding, but for the most part there are still significant mysteries to unravel. The inductive bias, which reflects prior knowledge embedded in the network architecture, is one of them. In this work, we establish a fundamental connection between the fields of quantum physics and deep learning. We use this connection for asserting novel theoretical observations regarding the role that the number of channels in each layer of the convolutional network fulfills in the overall inductive bias. Specifically, we show an equivalence between the function realized by a deep convolutional arithmetic circuit (ConvAC) and a quantum many-body wave function, which relies on their common underlying tensorial structure. This facilitates the use of quantum entanglement measures as well-defined quantifiers of a deep network's expressive ability to model intricate correlation structures of its inputs. Most importantly, the construction of a deep ConvAC in terms of a Tensor Network is made available. This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph. This result is relevant to any practitioner designing a network for a specific task. We theoretically analyze ConvACs, and empirically validate our findings on more common ConvNets which involve ReLU activations and max pooling. Beyond the results described above, the description of a deep convolutional network in well-defined graph-theoretic tools and the formal connection to quantum entanglement, are two interdisciplinary bridges that are brought forth by this work.

研究动机与目标

• 通过分析架构选择如何塑造相关性建模，理解深度卷积网络的归纳偏置。
• 利用张量网络表示，建立ConvAC与量子多体波函数之间的正式结构等价性。
• 使用量子纠缠测度作为定量工具，评估网络对复杂输入相关性的表达能力。
• 证明每一层的通道数通过图论中的最小割分析，直接控制网络的表达能力。
• 为特定任务设计具有所需归纳偏置的深层网络，提供一种原则性、理论基础坚实的构建方法。

提出的方法

• 通过共享的张量结构，将ConvAC的函数映射为量子多体波函数。
• 将网络的权重张量表示为分层张量分解（HT），等价于张量网络（TN）。
• 使用量子纠缠测度——如纠缠熵和互信息——作为深度网络中相关性建模的正式量化指标。
• 通过底层张量网络图的图论最小割分析，评估网络的表达能力。
• 建立每一层通道数与张量网络图中最小割值之间的直接对应关系，该值决定了网络建模复杂相关性的能力。
• 在标准的ReLU基于和最大池化卷积网络上，通过实证验证理论发现，结果与ConvAC预测一致。

实验结果

研究问题

• RQ1 深度卷积网络中通道数如何影响其对输入数据中复杂相关性的建模能力？
• RQ2 量子纠缠测度能否作为深度网络表达能力的正式、定量指标？
• RQ3 深度卷积算术电路与量子多体波函数之间存在何种结构等价性？
• RQ4 深度网络张量网络表示的图结构如何与其归纳偏置和表达能力相关联？
• RQ5 张量网络图的最小割分析能否预测网络对特定输入划分的建模能力？

主要发现

• 深度ConvAC实现的函数在结构上等价于量子多体波函数，使得量子纠缠测度可作为分析网络表达能力的正式工具。
• 量子纠缠测度——如纠缠熵——为网络对输入数据中复杂相关性结构的建模能力提供了明确定义的定量度量。
• 每一层的通道数直接控制网络的归纳偏置，因为其决定了底层张量网络图中的最小割值。
• 具有分层池化（如2×2窗口）的深度ConvAC相较于浅层网络（如CP分解）在表达能力上具有指数级优势，该结论通过最小割分析得到验证。
• ConvAC的张量网络表示支持图论分析，可直接将架构设计（通道数）与表达能力及相关性建模能力关联起来。
• 在标准ReLU和最大池化网络上的实证验证表明，基于ConvAC和张量网络框架推导出的理论预测在实践中成立。