[Paper Review] Testing Quantum Programs with Multiple Subroutines

Peixun Long, Jianjun Zhao|arXiv (Cornell University)|Aug 19, 2022

Quantum Computing Algorithms and Architecture4 citations

TL;DR

This paper proposes a novel testing framework for multi-subroutine quantum programs in Q#, addressing critical gaps in existing methods. It introduces three new coverage criteria—classical-superposition partition, classical-superposition-mixed partition, and entanglement partition—for equivalence class partitioning of quantum variables, enabling both unit and integration testing. The framework is implemented in QSharpTester, which successfully detects nearly all faulty mutations in benchmark programs with acceptable performance on a personal computer.

ABSTRACT

Errors in quantum programs are challenging to track down due to the uncertainty of quantum programs. Testing is, therefore, an indispensable method for assuring the quality of quantum software. Existing testing methods focus only on testing quantum programs with quantum circuits or single subroutines and, therefore, cannot effectively test quantum programs with multi-subroutines. In this paper, we first discuss several critical issues that must be considered when testing multi-subroutine quantum programs and point out the limitations and problems with existing testing methods. We then present a novel framework for testing multi-subroutine quantum programs that allow for both unit and integration testing. Our framework includes two novel test coverage criteria for the equivalent class partition of quantum variables to guide our testing tasks and techniques to test quantum programs with several common patterns. We also discuss how to generate test cases based on our framework. To evaluate the effectiveness of our testing framework, we implemented a tool called QSharpTester for testing Q\# programs with multiple subroutines. We used it to conduct experiments on hundreds of mutation programs deriving from seven original Q\# programs. The experimental results show that our testing methods can deal with broader types of quantum programs than existing ones and perform well on almost all faulty mutation programs.

Motivation & Objective

• To address the lack of systematic testing support for multi-subroutine quantum programs, which are common in practice but underexplored in existing testing methods.
• To identify and clarify critical issues in testing multi-subroutine quantum programs that have been overlooked in prior research.
• To develop a comprehensive testing framework that supports both unit and integration testing for quantum programs with multiple subroutines.
• To define new test coverage criteria tailored to the quantum nature of variables, including superposition and entanglement.
• To implement and evaluate a tool, QSharpTester, for testing Q# programs with multi-subroutines using mutation testing.

Proposed method

• Introduces IO marking to explicitly define inputs and outputs of each subroutine, enabling structured testing.
• Proposes three novel test coverage criteria: classical-superposition partition, classical-superposition-mixed partition, and entanglement partition, for equivalence class partitioning of quantum variables.
• Designs test case generation strategies based on input selection principles and output detection techniques, including projection-based and measurement-based output verification.
• Supports testing of two common program patterns in multi-subroutine quantum programs, enhancing practical applicability.
• Implements the framework in QSharpTester, a tool that integrates with the Q# simulator to execute and evaluate test cases.
• Employs mutation testing with four mutation types on seven original Q# programs and 244 mutant versions to evaluate effectiveness.

Experimental results

Research questions

• RQ1 How do the unique properties of multi-subroutine quantum programs—such as quantum state superposition, entanglement, and non-cloning—affect testing strategies?
• RQ2 What novel coverage criteria are needed to effectively test quantum variables across classical, superposition, and entangled states?
• RQ3 How can test case generation be systematically guided to detect quantum-specific bugs while maintaining efficiency?
• RQ4 To what extent can the proposed framework detect faults in multi-subroutine quantum programs compared to existing methods?
• RQ5 Can the framework be efficiently implemented and scaled on standard hardware using the Q# simulator?

Key findings

• The proposed framework successfully detects almost all faulty mutation programs in the benchmark set, demonstrating high fault-detection capability.
• The framework identifies some equivalent mutations, indicating sensitivity to subtle quantum behavior and potential for false positives.
• Testing tasks are completed in an acceptable time on a personal computer using the Q# simulator, confirming practical scalability.
• The three new coverage criteria—classical-superposition partition, classical-superposition-mixed partition, and entanglement partition—enable effective and targeted testing of quantum variables.
• QSharpTester, the implemented tool, supports both unit and integration testing, offering a systematic approach to multi-subroutine quantum program testing.
• The framework outperforms existing methods in handling broader types of quantum programs and detecting quantum-nature-related bugs.