[Paper Review] Quantum Computer Benchmarking via Quantum Algorithms

Κωνσταντίνος Γεωργόπουλος, Clive Emary+1 more|arXiv (Cornell University)|Dec 17, 2021

Quantum Computing Algorithms and Architecture4 citations

TL;DR

This paper introduces a quantum algorithm-based benchmarking framework that evaluates IBM quantum computers using scalable, real-world-inspired algorithms—such as discrete- and continuous-time quantum walks, Grover’s, and phase estimation—combined with noise modeling and ideal simulation. The key contribution is a detailed, architecture-specific performance analysis that reveals differences in noise behavior, circuit efficiency, and implementation trade-offs, including the first benchmark of a continuous-time quantum algorithm and its Pauli decomposition on a digital quantum processor, showing the decomposition is inefficient for small systems.

ABSTRACT

We present a framework that utilizes quantum algorithms, an architecture aware quantum noise model and an ideal simulator to benchmark quantum computers. The benchmark metrics highlight the difference between the quantum computer evolution and the simulated noisy and ideal quantum evolutions. We utilize our framework for benchmarking three IBMQ systems. The use of multiple algorithms, including continuous-time ones, as benchmarks stresses the computers in different ways highlighting their behaviour for a diverse set of circuits. The complexity of each quantum circuit affects the efficiency of each quantum computer, with increasing circuit size resulting in more noisy behaviour. Furthermore, the use of both a continuous-time quantum algorithm and the decomposition of its Hamiltonian also allows extracting valuable comparisons regarding the efficiency of the two methods on quantum systems. The results show that our benchmarks provide sufficient and well-rounded information regarding the performance of each quantum computer.

Motivation & Objective

• Address the lack of comprehensive, scalable, and architecture-aware benchmarks for Noisy Intermediate-Scale Quantum (NISQ) devices.
• Overcome limitations of generic metrics like quantum volume by providing algorithm-specific performance insights relevant to real-world applications.
• Evaluate how different quantum algorithms stress quantum hardware in distinct ways, revealing hardware-specific weaknesses and strengths.
• Investigate the efficiency trade-offs between native continuous-time quantum walks and their digital decompositions into Pauli gate sequences.
• Provide a detailed, multi-metric comparison between quantum computer execution, noisy simulations, and ideal simulations to isolate hardware-specific noise effects.

Proposed method

• Design a benchmarking framework using five scalable quantum algorithms: discrete-time quantum walks, continuous-time quantum walks (CTQW), CTQW Hamiltonian decomposition into Pauli gates, Grover’s algorithm, and quantum phase estimation.
• Implement a noise model that reflects the actual hardware characteristics of IBMQ devices (e.g., gate errors, decoherence) to simulate realistic quantum evolution.
• Use an ideal quantum simulator to generate noise-free reference outcomes for comparison with actual quantum computer results.
• Define three benchmark metrics: (1) fidelity between quantum computer output and ideal simulation, (2) fidelity between quantum computer and noisy simulation, and (3) fidelity between noisy and ideal simulations, forming a triangle inequality structure.
• Execute all circuits on three IBMQ superconducting processors (5-qubit Bogota, Santiago; 7-qubit Casablanca) and compare results across machines.
• Analyze the results using visualizations of outcome distributions and comparative metrics to assess algorithmic performance, noise sensitivity, and circuit efficiency.

Experimental results

Research questions

• RQ1 How do different quantum algorithms stress quantum hardware in distinct ways, revealing unique performance characteristics of NISQ devices?
• RQ2 To what extent does the decomposition of a continuous-time quantum walk’s Hamiltonian into Pauli gates affect circuit efficiency and fidelity on digital quantum processors?
• RQ3 Can benchmarking with real-world-inspired algorithms provide more informative insights into quantum computer performance than architecture-neutral metrics like quantum volume?
• RQ4 How do the three benchmark metrics—fidelity to ideal, fidelity to noisy simulation, and fidelity between noisy and ideal—reveal structural weaknesses in quantum hardware?
• RQ5 Does the use of continuous-time quantum algorithms as benchmarks expose differences in noise behavior and gate calibration that are not captured by standard benchmarks?

Key findings

• The benchmarking framework successfully captures performance differences between IBMQ devices with identical quantum volume (32), demonstrating that quantum volume alone is insufficient for characterizing hardware behavior.
• The continuous-time quantum walk (CTQW) and its Pauli decomposition produce distinct fidelity outcomes, with the decomposition showing significantly lower efficiency on small systems due to circuit depth and gate count.
• Fidelity to the ideal simulation drops with increasing circuit size, indicating that noise accumulates more severely in larger, more complex circuits.
• The framework reveals that calibrated noise parameters may over- or underestimate actual noise levels, as shown by discrepancies between noisy simulations and real hardware outputs.
• The use of multiple algorithms highlights diverse hardware behaviors: for example, the CTQW is more sensitive to noise than discrete-time walks, exposing underlying connectivity and gate error patterns.
• This is the first work to benchmark a digital quantum computer using a native continuous-time quantum algorithm and its digital decomposition, establishing a new benchmarking paradigm for future NISQ systems.