[Paper Review] Beyond Classical Attention: Quantum Attention for Scalable Computation

Yeqi Gao, Zhao Song+2 more|arXiv (Cornell University)|Jul 16, 2023

Quantum Computing Algorithms and Architecture4 citations

TL;DR

This paper proposes a quantum attention mechanism that leverages Grover's search algorithm to achieve polynomial speedup in computing sparse attention matrices for large language models (LLMs). By exploiting sparsity in attention scores and using quantum amplitude amplification, the method reduces time complexity from O(n²d) to Õ(n¹·⁵k⁰·⁵d + nkd), with provable error bounds and a low-rank structure beneficial for training.

ABSTRACT

As large language models (LLMs) demonstrate outstanding performance across various tasks, attention-driven models have profoundly transformed the field of machine learning. Since attention computations account for the primary computational overhead in both model inference and training, efficiently computing attention matrices has become one of the core challenges in accelerating large language models. It is well-known that quantum machines possess computational advantages over classical machines, and the role of quantum computing in LLMs remains largely unexplored. In this work, we focus on leveraging the Grover search algorithm to efficiently compute a sparse attention matrix. Through comparisons with classical algorithms, we demonstrate that our method achieves quantum acceleration in polynomial time. Additionally, we observe that the generated quantum attention matrices naturally exhibit low-rank structures, providing further theoretical support for efficient modeling. Moreover, within the specific context of attention matrix computation, we conduct a systematic and detailed analysis of the error and time complexity of the proposed algorithm.

Motivation & Objective

• To address the high computational cost of classical self-attention in large language models (LLMs), which scales as O(n²d).
• To explore whether quantum computing can accelerate attention computation in LLMs, particularly through sparsity exploitation.
• To design a quantum algorithm that achieves a provable speedup over classical methods while maintaining accuracy.
• To analyze the time complexity and error bounds of the proposed quantum attention mechanism.

Proposed method

• The method uses Grover’s search algorithm to identify the k largest attention scores per query token, exploiting sparsity in the QKᵀ matrix.
• It constructs a sparse attention matrix B by marking indices j where (QKᵀ)_{i,j} ≥ τ using a quantum oracle.
• The algorithm applies amplitude amplification to achieve a quadratic speedup in finding high-scoring attention entries.
• The resulting sparse matrix B is used to compute the attention output via D(B)⁻¹BV, with time complexity O(nkd) for sparse parts.
• The method ensures low-rank structure in the attention matrix, which may accelerate subsequent training phases.
• Error analysis bounds the difference between classical and quantum attention outputs by O(η²), where η is the bound on the infinity norm of the value matrix.

Experimental results

Research questions

• RQ1 Can quantum algorithms provide a scalable speedup for attention computation in large language models?
• RQ2 Does exploiting sparsity in attention scores enable a practical quantum advantage in attention matrix computation?
• RQ3 What is the time complexity and error tolerance of a quantum attention mechanism compared to classical counterparts?
• RQ4 Can the quantum-constructed attention matrix preserve the low-rank structure useful for efficient fine-tuning and training?

Key findings

• The proposed quantum algorithm achieves a time complexity of Õ(n¹·⁵k⁰·⁵d + nkd), offering a polynomial speedup over the classical O(n²d) method.
• The quantum attention matrix B exhibits a low-rank structure, which may enable faster training of LLMs.
• The error between the classical attention output D(A)⁻¹AV and the quantum output D(B)⁻¹BV is bounded by O(η²), where η = ||V||∞.
• The method relies on Grover’s search to identify k high-scoring attention entries per query, reducing the effective computation to sparse operations.
• The algorithm is robust under bounded perturbations, with error scaling quadratically with the norm of the value matrix.
• The theoretical framework establishes a foundation for quantum acceleration in attention mechanisms, particularly in sparse regimes common in LLMs.