[Paper Review] Encoding-dependent generalization bounds for parametrized quantum circuits

C. Matthias, Elies Gil-Fuster+3 more|arXiv (Cornell University)|Jan 1, 2021

Quantum Computing Algorithms and Architecture62 references94 citations

TL;DR

This paper derives generalization bounds for parametrized quantum circuits (PQCs) that explicitly depend on data-encoding strategies, using trigonometric polynomial representations and statistical learning theory. It shows that generalization performance is directly influenced by the frequency spectrum of the encoding, enabling data-encoding strategy selection via structural risk minimization with polynomial scaling of accessible frequencies.

ABSTRACT

A large body of recent work has begun to explore the potential of parametrized quantum circuits (PQCs) as machine learning models, within the framework of hybrid quantum-classical optimization. In particular, theoretical guarantees on the out-of-sample performance of such models, in terms of generalization bounds, have emerged. However, none of these generalization bounds depend explicitly on how the classical input data is encoded into the PQC. We derive generalization bounds for PQC-based models that depend explicitly on the strategy used for data-encoding. These imply bounds on the performance of trained PQC-based models on unseen data. Moreover, our results facilitate the selection of optimal data-encoding strategies via structural risk minimization, a mathematically rigorous framework for model selection. We obtain our generalization bounds by bounding the complexity of PQC-based models as measured by the Rademacher complexity and the metric entropy, two complexity measures from statistical learning theory. To achieve this, we rely on a representation of PQC-based models via trigonometric functions. Our generalization bounds emphasize the importance of well-considered data-encoding strategies for PQC-based models.

Motivation & Objective

• To address the lack of explicit dependence on data-encoding strategies in existing generalization bounds for PQC-based models.
• To enable principled selection of data-encoding strategies via structural risk minimization using rigorous complexity measures.
• To derive generalization bounds that depend on architectural hyper-parameters of data encoding, particularly in circuits with data re-uploading.
• To connect the expressive power of PQCs to their generalization performance through frequency spectrum analysis of trigonometric polynomial representations.

Proposed method

• Representing PQC-based models as generalized trigonometric polynomials (GTPs) to link data-encoding strategies to frequency spectrum complexity.
• Bounding the Rademacher complexity and metric entropy of GTPs using the number of accessible frequencies as a complexity measure.
• Deriving generalization bounds that explicitly depend on encoding-specific hyper-parameters such as the number of data-encoding gates.
• Applying Dudley’s theorem and Talagrand’s lemma to bound empirical Rademacher complexity via metric entropy.
• Using structural risk minimization to combine empirical risk with encoding-dependent generalization bounds for model selection.
• Analyzing data-encoding strategies where the number of accessible frequencies scales polynomially with the number of encoding gates.

Experimental results

Research questions

• RQ1 How does the choice of data-encoding strategy affect the generalization performance of a PQC-based model?
• RQ2 Can generalization bounds be derived that explicitly depend on encoding-specific architectural hyper-parameters?
• RQ3 What is the relationship between the frequency spectrum of a PQC's GTP representation and its generalization capability?
• RQ4 How can encoding-dependent generalization bounds be used to guide model selection in practice?
• RQ5 In what regime are the derived generalization bounds meaningful, and how do they compare to bounds that depend only on circuit depth or gate count?

Key findings

• Generalization bounds for PQC-based models are derived that explicitly depend on the data-encoding strategy, particularly through the number of accessible frequencies in the GTP representation.
• The bounds scale with the square root of the number of accessible frequencies, providing a direct link between encoding design and generalization performance.
• For various natural data-encoding strategies, the number of accessible frequencies scales polynomially with the number of data-encoding gates, leading to polynomially scaling generalization bounds.
• The derived bounds enable encoding-dependent model selection via structural risk minimization, combining empirical risk with generalization risk.
• The results are applicable to PQC architectures with data re-uploading, which are known to enhance expressive power, and extend beyond 'encoding-first' models.
• The bounds are expected to be most meaningful in the moderate-complexity regime, where the number of parameters is less than the number of training samples.