[Paper Review] Exponentially Many Local Minima in Quantum Neural Networks

Xuchen You, Xiaodi Wu|arXiv (Cornell University)|Oct 6, 2021

Quantum Computing Algorithms and Architecture4 citations

TL;DR

This paper demonstrates that under-parameterized quantum neural networks (QNNs) can exhibit exponentially many spurious local minima in their loss landscapes due to quantum interference, not non-linear activations as in classical networks. Using a constructive approach, the authors prove the existence of QNN instances with loss functions containing 2^Ω(p) local minima for p parameters, and empirically validate that standard optimizers struggle to escape them, highlighting a fundamental training challenge in near-term quantum machine learning.

ABSTRACT

Quantum Neural Networks (QNNs), or the so-called variational quantum circuits, are important quantum applications both because of their similar promises as classical neural networks and because of the feasibility of their implementation on near-term intermediate-size noisy quantum machines (NISQ). However, the training task of QNNs is challenging and much less understood. We conduct a quantitative investigation on the landscape of loss functions of QNNs and identify a class of simple yet extremely hard QNN instances for training. Specifically, we show for typical under-parameterized QNNs, there exists a dataset that induces a loss function with the number of spurious local minima depending exponentially on the number of parameters. Moreover, we show the optimality of our construction by providing an almost matching upper bound on such dependence. While local minima in classical neural networks are due to non-linear activations, in quantum neural networks local minima appear as a result of the quantum interference phenomenon. Finally, we empirically confirm that our constructions can indeed be hard instances in practice with typical gradient-based optimizers, which demonstrates the practical value of our findings.

Motivation & Objective

• To investigate the loss landscape of quantum neural networks (QNNs) and understand the nature and number of local minima in their training.
• To determine whether local minima in QNNs are benign or spurious, especially in under-parameterized settings.
• To identify the root cause of local minima in QNNs, distinguishing them from classical neural networks.
• To construct explicit QNN instances with exponentially many local minima and validate their hardness empirically.
• To provide theoretical bounds on the maximum number of local minima possible in such QNNs.

Proposed method

• Construct a family of simple yet hard QNN instances using a two-layer XY-encoder and parameterized unitary gates.
• Design a dataset that induces a loss function with exponentially many local minima via quantum interference effects.
• Prove that for p parameters, the number of local minima is at least 2^Ω(p), using quantum circuit symmetries and interference.
• Establish an almost-matching upper bound on the number of local minima, showing the construction is nearly optimal.
• Use gradient-based optimizers (e.g., RMSProp) on randomly initialized parameters to empirically evaluate convergence to global vs. local minima.
• Perturb the labels with Gaussian noise and measure robustness of local minima and success rates in finding global optima.

Experimental results

Research questions

• RQ1 Can under-parameterized QNNs have exponentially many local minima in their loss landscape?
• RQ2 What is the origin of local minima in QNNs—quantum interference or non-linearities?
• RQ3 Are these local minima robust to perturbations such as label noise or measurement errors?
• RQ4 Can standard gradient-based optimizers reliably find the global minimum in such hard instances?
• RQ5 How does the number of local minima scale with the number of parameters in QNNs?

Key findings

• The number of spurious local minima in the loss function of typical under-parameterized QNNs can grow exponentially with the number of parameters, specifically at least 2^Ω(p) for p parameters.
• Local minima in QNNs arise due to quantum interference, not non-linear activation functions as in classical neural networks.
• The constructed QNN instances are nearly optimal, as the upper bound on the number of local minima matches the lower bound up to a sub-exponential factor.
• Empirical results show that standard gradient-based optimizers like RMSProp fail to find the global minimum with exponentially decaying success rate as the number of parameters increases.
• The local minima remain robust under Gaussian label noise, quantum noise, and finite measurement errors, as long as the perturbation in the loss function is bounded in ℓ∞-norm.
• Even on natural, linearly separable datasets, the number of local minima increases significantly with parameter count, and the success rate of finding the global minimum drops sharply.