[Paper Review] Contextuality and inductive bias in quantum machine learning

Joseph E. Bowles, Victoria J Wright+3 more|arXiv (Cornell University)|Feb 2, 2023

Quantum Computing Algorithms and Architecture9 citations

TL;DR

This paper develops a general framework linking quantum contextuality to inductive bias in machine learning, defines contextual multi-task models, and shows that contextuality can enhance expressivity and, in a toy setup, quantum models can outperform classical surrogates by encoding a linearly conserved label bias.

ABSTRACT

Generalisation in machine learning often relies on the ability to encode structures present in data into an inductive bias of the model class. To understand the power of quantum machine learning, it is therefore crucial to identify the types of data structures that lend themselves naturally to quantum models. In this work we look to quantum contextuality -- a form of nonclassicality with links to computational advantage -- for answers to this question. We introduce a framework for studying contextuality in machine learning, which leads us to a definition of what it means for a learning model to be contextual. From this, we connect a central concept of contextuality, called operational equivalence, to the ability of a model to encode a linearly conserved quantity in its label space. A consequence of this connection is that contextuality is tied to expressivity: contextual model classes that encode the inductive bias are generally more expressive than their noncontextual counterparts. To demonstrate this, we construct an explicit toy learning problem -- based on learning the payoff behaviour of a zero-sum game -- for which this is the case. By leveraging tools from geometric quantum machine learning, we then describe how to construct quantum learning models with the associated inductive bias, and show through our toy problem that they outperform their corresponding classical surrogate models. This suggests that understanding learning problems of this form may lead to useful insights about the power of quantum machine learning.

Motivation & Objective

• Motivate the search for data structures that naturally leverage quantum inductive biases.
• Define a framework for generalized contextuality in machine learning and a notion of contextual learning models.
• Connect contextuality, operational equivalence, and inductive bias to expressivity and learning performance.
• Identify data configurations where contextuality constrains noncontextual models and may necessitate quantum approaches.
• Demonstrate via a toy problem how quantum models encoding the bias can outperform classical surrogates.

Proposed method

• Introduce a framework of generalized contextuality (operational statistics, preparations, effects) for learning models.
• Define operational scenarios for multi-task learning and formalize noncontextuality via ontological models.
• Establish a main result linking a linear conservation bias in label space to constraints on noncontextual learnable distributions.
• Describe two quantum-ansatz schemes to encode the inductive bias: state-based and measurement-based approaches.
• Construct a toy rock–paper–scissors learning problem to prove expressivity limits of noncontextual models.
• Use geometric quantum machine learning tools to design quantum models aligned with the bias and compare to classical surrogates.

Figure 1: A. An example of the type of learning problem we consider in this work. Labels are generated for input training data $\boldsymbol{x}_{i}$ via a conditional process $P(\boldsymbol{y}_{i}|\boldsymbol{x}_{i})$ . Here, the labels take the form $\boldsymbol{y}_{i}=(y_{i}^{(1)},y_{i}^{(2)},y_{i}

Experimental results

Research questions

• RQ1 What is a suitable notion of contextuality for learning models in machine learning?
• RQ2 How does an inductive bias that encodes a linear conservation law on label space affect the expressivity of noncontextual models?
• RQ3 Can contextuality-based inductive biases in quantum models yield superior generalisation compared to classical surrogates on structured tasks?
• RQ4 In which learning scenarios do contextuality-inspired biases emerge as a necessary resource for accurate learning?

Key findings

• Contextuality is connected to expressivity: contextual model classes that encode the inductive bias tend to be more expressive than noncontextual ones.
• A linearly conserved quantity in the label space imposes operational equivalences that constrain noncontextual models, potentially limiting generalisation.
• A toy rock–paper–scissors problem demonstrates precise bounds on noncontextual model expressivity for learning payoff behaviour.
• Quantum learning models that encode the bias via state structure or measurement design can outperform corresponding classical surrogate models on the toy task.
• Numerical evidence shows a quantum model with the contextuality-inspired bias achieving lower generalisation error than classical surrogates under regularisation.

Figure 2: A. (top) A prepare-and-measure scenario. A preparation S is a procedure (a list of actions) that a user carries out. In this example, the procedure is to input the data $\boldsymbol{x}$ into the machine learning model. (bottom) An effect E is another procedure that additionally has an obse