[Paper Review] Toward Trainability of Quantum Neural Networks

Kaining Zhang, Min-Hsiu Hsieh+2 more|arXiv (Cornell University)|Nov 12, 2020

Quantum Computing Algorithms and Architecture33 references43 citations

TL;DR

The paper proves trainability guarantees for tree tensor (TT) and step-controlled (SC) quantum neural networks by deriving lower bounds on gradient norms, and demonstrates superior training and accuracy over random QNNs on binary classification tasks. It avoids the unitary 2-design assumption and validates results via simulations on MNIST-derived data.

ABSTRACT

Quantum Neural Networks (QNNs) have been recently proposed as generalizations of classical neural networks to achieve the quantum speed-up. Despite the potential to outperform classical models, serious bottlenecks exist for training QNNs; namely, QNNs with random structures have poor trainability due to the vanishing gradient with rate exponential to the input qubit number. The vanishing gradient could seriously influence the applications of large-size QNNs. In this work, we provide a viable solution with theoretical guarantees. Specifically, we prove that QNNs with tree tensor and step controlled architectures have gradients that vanish at most polynomially with the qubit number. We numerically demonstrate QNNs with tree tensor and step controlled structures for the application of binary classification. Simulations show faster convergent rates and better accuracy compared to QNNs with random structures.

Motivation & Objective

• Address the barren plateaus issue in quantum neural networks (QNNs).
• Introduce TT-QNN and SC-QNN architectures with provable trainability.
• Provide gradient norm lower bounds that do not rely on 2-design assumptions.
• Show through simulations that TT-QNN and SC-QNN outperform random QNNs in training and accuracy.
• Offer an input encoding framework and theoretical guarantees applicable to near-term quantum devices.

Proposed method

• Define TT-QNN and SC-QNN architectures and their parameterized circuits.
• Prove lower bounds on the gradient norm: E||∇θ fTT||^2 ≥ ~Ω(1/n) and E||∇θ fSC||^2 ≥ ~Ω(2−nc).
• Use the parameter-shifting rule to compute gradients for single-qubit phase-encoded gates.
• Introduce an encoding circuit to prepare input states and bound α(ρin) via Theorem 3.2 (Eβα(ρin) ≥ 2−2L).
• Derive results without relying on the unitary 2-design assumption.
• Conduct binary classification experiments on MNIST-derived data to compare TT-QNN, SC-QNN, and Random-QNN.

Experimental results

Research questions

• RQ1 Do TT-QNNs and SC-QNNs avoid barren plateaus and provide polynomially-bounded gradient norms?
• RQ2 What are the gradient norm lower bounds for TT-QNNs and SC-QNNs, and how do they depend on n and nc?
• RQ3 How does the input-state encoding affect trainability and gradient behavior?
• RQ4 Do TT-QNN and SC-QNN architectures yield better training dynamics and accuracy than random-structure QNNs in binary classification?

Key findings

• The TT-QNN gradient norm has a lower bound ~Ω(1/n).
• The SC-QNN gradient norm has a lower bound ~Ω(2−nc).
• These bounds hold without assuming unitary 2-designs and apply to linear-depth circuits.
• Encoding circuit analysis shows Eβα(ρin) ≥ 2−2L, stabilizing gradient estimates against input state variations.
• Experiments on MNIST-based binary classification show TT-QNN and SC-QNN outperform Random-QNN in training loss and test accuracy across several qubit counts (n=8,10,12).
• Gradient norms observed in simulations align with the theoretical bounds.