[Paper Review] Impact of unreliable devices on stability of quantum computations

Samudra Dasgupta, Travis S. Humble|arXiv (Cornell University)|Jul 13, 2023

Quantum Computing Algorithms and Architecture4 citations

TL;DR

This paper proposes a framework to quantify the reliability of noisy intermediate-scale quantum (NISQ) devices by analyzing time-varying noise statistics using similarity metrics from device characterization data. It derives bounds on observable stability using joint probability distributions of gate fidelity, dephasing time, SPAM, and cross-talk errors, validating the approach via simulations of the Bernstein-Vazirani circuit on the IBM Washington superconducting device, which shows significant monthly fluctuations in performance, indicating instability for reproducible results beyond a 2.2% normalized similarity threshold.

ABSTRACT

Noisy intermediate-scale quantum (NISQ) devices are valuable platforms for testing the tenets of quantum computing, but these devices are susceptible to errors arising from de-coherence, leakage, cross-talk and other sources of noise. This raises concerns regarding the stability of results when using NISQ devices since strategies for mitigating errors generally require well-characterized and stationary error models. Here, we quantify the reliability of NISQ devices by assessing the necessary conditions for generating stable results within a given tolerance. We use similarity metrics derived from device characterization data to derive and validate bounds on the stability of a 5-qubit implementation of the Bernstein-Vazirani algorithm. Simulation experiments conducted with noise data from IBM Washington, spanning January 2022 to April 2023, revealed that the reliability metric fluctuated between 41% and 92%. This variation significantly surpasses the maximum allowable threshold of 2.2% needed for stable outcomes. Consequently, the device proved unreliable for consistently reproducing the statistical mean in the context of the Bernstein-Vazirani circuit.

Motivation & Objective

• To address the lack of methods for assessing long-term stability in NISQ devices due to non-stationary noise processes.
• To develop a rigorous framework for evaluating device reliability across temporal and spatial scales.
• To quantify how fluctuations in key error metrics affect computational stability and reproducibility.
• To validate the reliability bounds using real characterization data from a superconducting transmon device.
• To support future benchmarking protocols by enabling application-specific reliability thresholds.

Proposed method

• Uses similarity metrics—specifically normalized Hellinger distance—between time-resolved probability distributions of device observables to assess reliability.
• Derives theoretical bounds on observable stability using joint probability distributions of noise parameters: gate fidelity, dephasing time, SPAM, and cross-talk.
• Applies Hölder’s inequality and integral approximations to bound the variation in quantum circuit outcomes across time.
• Employs monthly characterization data from the IBM Washington device to simulate the Bernstein-Vazirani algorithm and evaluate stability over 1.5 years.
• Models non-stationary noise by comparing distributions at different time points and assessing their similarity to infer reliability.
• Validates bounds using numerical simulations and analyzes the impact of approximation errors on tightness of the bounds.

Figure 1. Daily estimates of the state preparation and measurement fidelity for two register elements in a superconducting transmon device collected from 1-Jan-2022 to 30-Apr-2023 ( aleksandrowicz2019qiskit, ) . Day of the year is indexed starting with 1 and index 365 corresponds to Dec 31.

Experimental results

Research questions

• RQ1 How do time-varying noise statistics in NISQ devices affect the stability of quantum circuit outcomes?
• RQ2 What is the relationship between device characterization metrics and the reproducibility of quantum computation results?
• RQ3 To what extent can theoretical bounds on observable stability be derived from joint probability distributions of noise parameters?
• RQ4 How does the normalized Hellinger distance serve as a reliable metric for quantifying device reliability across time?
• RQ5 What are the practical implications of these bounds for error mitigation and benchmarking in real-world NISQ devices?

Key findings

• The normalized Hellinger distance between device states at different times varied between 0.41 and 0.92 over a 1.5-year period, indicating significant instability in the IBM Washington device.
• A threshold of 2.2% normalized similarity is required to ensure the mean output of the Bernstein-Vazirani circuit remains within ±0.25 of its original value, which was not consistently met.
• The theoretical bound on observable stability is loose due to three factors: use of Hölder’s inequality, approximation using maximum observable values, and non-monotonic noise behavior.
• The bound becomes tighter when noise distributions at later times are stochastically dominated by earlier ones, which occurs between calibrations.
• The study reveals that current benchmarking methods fail to account for non-stationary noise, undermining reproducibility and uncertainty quantification.
• The framework enables application-specific reliability thresholds and supports improved calibration metadata and protocol design for stable quantum computing.

Figure 2. A quantum circuit implementation of the Bernstein-Vazirani algorithm that employs 5 qubits, denoted $q_{0}$ to $q_{4}$ . The first four qubits are used to compute the 4-bit secret string, while the fifth qubit serves as an ancilla and initially resides in the $\ket{-}$ superposition state.