[Paper Review] Mitigating Noise in Quantum Software Testing Using Machine Learning

Asmar Muqeet, Tao Yue+2 more|arXiv (Cornell University)|Jun 29, 2023

Quantum Computing Algorithms and ArchitectureComputer Science3 citations

TL;DR

This paper proposes QOIN, a machine learning-based approach to mitigate noise in quantum software testing by learning and filtering noise patterns from NISQ quantum computers. Using transfer learning with fully connected neural networks, QOIN reduces noise effects by over 80% on most backends, enabling accurate test case assessment via filtered outputs, achieving 99% precision, 75% recall, and 86% F1-score with a published test oracle.

ABSTRACT

Quantum Computing (QC) promises computational speedup over classic computing for solving complex problems. However, noise exists in current and near-term quantum computers. Quantum software testing (for gaining confidence in quantum software's correctness) is inevitably impacted by noise, to the extent that it is impossible to know if a test case failed due to noise or real faults. Existing testing techniques test quantum programs without considering noise, i.e., by executing tests on ideal quantum computer simulators. Consequently, they are not directly applicable to testing quantum software on real quantum computers or noisy simulators. To this end, we propose a noise-aware approach (named QOIN) to alleviate the noise effect on test results of quantum programs. QOIN employs machine learning techniques (e.g., transfer learning) to learn the noise effect of a quantum computer and filter it from a quantum program's outputs. Such filtered outputs are then used as the input to perform test case assessments (determining the passing or failing of a test case execution against a test oracle). We evaluated QOIN on IBM's 23 noise models, Google's two available noise models, and Rigetti's Quantum Virtual Machine (QVM), with nine real-world quantum programs and 1000 artificial quantum programs. Results show that QOIN can reduce the noise effect by more than $80\%$ on the majority of noise models. For quantum software testing, we used an existing test oracle and showed that QOIN attained scores of $99\%$, $75\%$, and $86\%$ for precision, recall, and F1-score, respectively, for the test oracle across six real-world programs. For artificial programs, QOIN achieved scores of $93\%$, $79\%$, and $86\%$ for precision, recall, and F1-score. This highlights QOIN's effectiveness in learning noise patterns for noise-aware quantum software testing.

Motivation & Objective

• Address the challenge of noise in current and near-term quantum computers, which undermines the reliability of quantum software testing.
• Overcome the limitation of existing testing techniques that rely on ideal, noise-free simulators and thus fail on real noisy hardware.
• Enable accurate test case assessment on real NISQ devices by distinguishing between failures due to real faults and those due to noise.
• Develop a scalable, backend-specific noise filtering mechanism using machine learning that adapts to both general and circuit-specific noise patterns.
• Validate the effectiveness of the approach across diverse quantum programs and multiple real-world and simulated noisy backends.

Proposed method

• Train a fully connected neural network (MLP) on noisy outputs from baseline quantum circuits to learn general noise patterns of a specific NISQ backend.
• Apply transfer learning to adapt the general noise model to circuit-specific noise by fine-tuning on a target program’s noisy outputs.
• Use the trained circuit-specific model as a noise filter to predict clean outputs from noisy program executions.
• Feed the filtered outputs into an existing test oracle to assess test case pass/fail status, improving accuracy over unfiltered results.
• Use Qiskit to generate 1000 diverse artificial quantum circuits to evaluate generalizability across different circuit structures and depths.
• Integrate a Baseline Tuner module that periodically re-trains the noise model when backend noise characteristics change, with a cost of ~14 minutes per circuit-backend pair.

Experimental results

Research questions

• RQ1 Can a machine learning model effectively learn and filter out noise from quantum program outputs on real and simulated NISQ backends?
• RQ2 To what extent does QOIN reduce noise effects compared to unfiltered test results across diverse quantum programs and backends?
• RQ3 How effective is QOIN in distinguishing between test failures caused by real software faults versus noise-induced errors?
• RQ4 What is the computational overhead of integrating QOIN into the quantum software testing pipeline?
• RQ5 How well does QOIN generalize across different circuit structures, depths, and diversity criteria?

Key findings

• QOIN reduced noise effects by more than 80% on the majority of the 26 evaluated backends, including IBM’s 23 noise models, Google’s two models, and Rigetti’s QVM.
• With a published test oracle, QOIN achieved 99% precision, 75% recall, and 86% F1-score on six real-world quantum programs, demonstrating high accuracy in fault detection.
• For 1000 artificially generated quantum programs, QOIN achieved 93% precision, 79% recall, and 86% F1-score, confirming robustness across diverse circuit types.
• The inference time for the noise filter model on a GPU was only a few seconds, indicating minimal runtime overhead for test assessment.
• The Baseline Tuner module required approximately 14 minutes per circuit-backend pair on average, with 12 minutes spent on data generation, but this cost would be significantly lower on actual NISQ hardware.
• The one-time training cost for the baseline model on an Nvidia 3080 GPU averaged about one minute, making the approach feasible despite initial setup.