[Paper Review] Four Conceptions of Instruction Sequence Faults

J.A. Bergstra|arXiv (Cornell University)|Apr 9, 2012

Software Testing and Debugging Techniques40 references3 citations

TL;DR

This paper proposes four distinct conceptions of instruction sequence faults—mechanical, product authority, process authority, and incorrectness substitute—each offering a different perspective on fault identification and testing. It argues that these views are complementary, with the mechanical view emphasizing confirmation testing, and advocates for flexible adoption of these conceptions depending on context, contributing to a refined framework for instruction sequence testing grounded in formal semantics and practical validation.

ABSTRACT

The notion of an instruction sequence fault is considered from various perspectives. Four different viewpoints on what constitutes a fault, or how to use the notion of a fault, are formulated. An integration of these views is proposed.

Motivation & Objective

• To address the ambiguity in defining 'program fault' by using instruction sequences as a formalized model for program behavior.
• To reduce contradictions with existing literature on program faults by focusing on instruction sequences, which have clearer definitions than general programs.
• To develop a coherent framework for instruction sequence testing by analyzing fault concepts through multiple, complementary perspectives.
• To highlight the dependence of fault concepts on testing practices, particularly confirmation testing, and to support flexible application of fault views in software engineering contexts.

Proposed method

• Formalizing instruction sequences as a model for programs, leveraging established theories from [11, 15, 12, 9] to ensure definitional clarity.
• Introducing four distinct conceptions of faults: mechanical (based on execution and confirmation testing), product authority (defect-focused), process authority (process-based adequacy), and incorrectness substitute (communicative use of 'fault' as a proxy for 'defect').
• Using 'putting into effect' as a general term encompassing execution, interpretation, compilation, simulation, and hybrid mechanisms, avoiding the limitations of 'execution'.
• Proposing that fault definitions are not absolute but context-dependent, with each view serving different purposes in testing and quality assurance.
• Integrating the views by distinguishing terms like 'mistake', 'fault', 'error', 'failure', 'defect', and 'violation' to clarify conceptual boundaries.
• Emphasizing that confirmation testing is central to the mechanical view and that unification of views requires careful terminology to avoid dominance of one perspective.

Experimental results

Research questions

• RQ1 How can the concept of a fault be meaningfully defined in the context of instruction sequences, given the lack of a standard definition for 'program'?
• RQ2 What are the distinct roles and justifications for the four conceptions of instruction sequence faults: mechanical, product authority, process authority, and incorrectness substitute?
• RQ3 How does the mechanical view of faults, particularly through confirmation testing, relate to the idealized repair criterion and practical limitations like regression testing?
• RQ4 In what ways can the four fault conceptions be unified without privileging one over the others, especially given the central role of mechanical faults?
• RQ5 To what extent can insights from instruction sequence faults be generalized to real-world program faults, and what are the risks of such generalization?

Key findings

• The mechanical view of faults, grounded in confirmation testing, is essential for practical fault detection, even when the idealized repair criterion is unattainable.
• The product authority and process authority views provide valuable, non-mechanical perspectives on faults, particularly in quality engineering and organizational standards.
• The incorrectness substitute view enables clear communication in contexts where formal correctness is unattainable, allowing 'fault' to be used as a proxy for 'defect'.
• Unification of the four views is possible only by distinguishing terms like 'defect' and 'violation' from 'fault', preventing dominance of the mechanical perspective.
• The mechanical view is not inherently superior; its centrality may diminish if future engineering advances eliminate mechanical faults through better specification and verification.
• The paper's main contribution is clarifying the role of confirmation testing in defining mechanical faults, especially when formal correctness cannot be verified.