[论文解读] Entanglement and Tensor Networks for Supervised Image Classification

John R. C. Martyn, Guifré Vidal+2 more|arXiv (Cornell University)|Jul 12, 2020

Computational Physics and Python Applications被引用 9

一句话总结

本文研究了在使用MNIST数字进行监督图像分类的张量网络模型中的量子纠缠特性。研究发现，尽管长程纠缠并非必需——仅具有局部纠缠（n×n区块）的块乘积态在n=2时即可实现99.97%的训练准确率——但这些模型存在过拟合问题，表明当前方法在优化方面仍需改进才能实现良好泛化能力。

ABSTRACT

Tensor networks, originally designed to address computational problems in quantum many-body physics, have recently been applied to machine learning tasks. However, compared to quantum physics, where the reasons for the success of tensor network approaches over the last 30 years is well understood, very little is yet known about why these techniques work for machine learning. The goal of this paper is to investigate entanglement properties of tensor network models in a current machine learning application, in order to uncover general principles that may guide future developments. We revisit the use of tensor networks for supervised image classification using the MNIST data set of handwritten digits, as pioneered by Stoudenmire and Schwab [Adv. in Neur. Inform. Proc. Sys. 29, 4799 (2016)]. Firstly we hypothesize about which state the tensor network might be learning during training. For that purpose, we propose a plausible candidate state $|Σ_{\ell} angle$ (built as a superposition of product states corresponding to images in the training set) and investigate its entanglement properties. We conclude that $|Σ_{\ell} angle$ is so robustly entangled that it cannot be approximated by the tensor network used in that work, which must therefore be representing a very different state. Secondly, we use tensor networks with a block product structure, in which entanglement is restricted within small blocks of $n imes n$ pixels/qubits. We find that these states are extremely expressive (e.g. training accuracy of $99.97 \%$ already for $n=2$), suggesting that long-range entanglement may not be essential for image classification. However, in our current implementation, optimization leads to over-fitting, resulting in test accuracies that are not competitive with other current approaches.

研究动机与目标

• 通过分析其纠缠特性，理解张量网络为何在机器学习中取得成功，尤其是在图像分类任务中。
• 探究张量网络是否在学习训练图像的叠加态，这将需要极大的纠缠度。
• 测试是否将纠缠限制在小块（n×n像素）内，仍能保持高性能并改善泛化能力。
• 识别结构与优化方面的改进，以减少张量网络模型在图像分类任务中的过拟合现象。

提出的方法

• 提出一个叠加态 |Σℓ⟩ 作为张量网络所学状态的候选，定义为所有属于类别 ℓ 的训练图像的叠加态。
• 分析 |Σℓ⟩ 的纠缠熵，表明其需要约 χ ≈ 6,000 的键维数才能近似，远超典型MPS的键维数（χ ≤ 120）。
• 引入块乘积态（BPS），将纠缠限制在 n×n 像素块内部，从而实现高效计算与可控纠缠。
• 采用矩阵乘积态（MPS）架构，并使用分块张量表示BPS，允许通过损失函数的梯度下降进行优化。
• 采用MPS张量的冗余参数化方式（例如边界处键维数 >2），以改善泛化性能，并观察到测试准确率的提升。
• 通过比较不同块大小（n=2,3,4）与键维数（χ）下的训练与测试准确率，评估过拟合与表达能力。

实验结果

研究问题

• RQ1 先前工作（Stoudenmire 和 Schwab, 2016）中的张量网络模型是否在近似所有类别 ℓ 的训练图像的叠加态 |Σℓ⟩？
• RQ2 张量网络在图像分类任务中实际学习到的状态具有怎样的纠缠结构？
• RQ3 将纠缠限制在局部 n×n 像素块内，是否仍能实现高分类准确率？
• RQ4 为何当前用于图像分类的张量网络模型存在过拟合？如何改进优化方法？

主要发现

• 作为所学状态的合理候选，叠加态 |Σℓ⟩ 具有极高的纠缠度，无法用键维数 χ ≤ 120 的MPS近似，表明模型学习的是另一种状态。
• 当 n=2（即 2×2 像素块）时，块乘积态可实现 99.97% 的训练准确率，证明长程纠缠并非实现高表达能力的必要条件。
• 尽管训练准确率很高，测试准确率仍较低，表明当前优化方案存在显著过拟合。
• 采用冗余参数化（例如MPS边界处键维数 >2）可提升测试准确率，表明存在优化空间以实现更好泛化。
• 最大测试准确率随块大小 n 单调递增，表明更大的块能提升泛化潜力。
• 尽管表达能力强，当前实现的块乘积态在测试准确率上仍无法与最先进模型相媲美，凸显了对更优优化策略的迫切需求。