[Paper Review] Assurance 2.0: A Manifesto

Robin Bloomfield, John Rushby|arXiv (Cornell University)|Apr 22, 2020

Safety Systems Engineering in Autonomy19 references4 citations

TL;DR

Assurance 2.0 proposes a rigorous, dynamic framework for system assurance that enhances innovation by embedding formal reasoning, evidence evaluation, and systematic defeater analysis into assurance cases. It strengthens traditional assurance through deductive reasoning, confirmation theory, and indefeasibility criteria, ensuring defensible, transparent, and auditable claims about system trustworthiness.

ABSTRACT

System assurance is confronted by significant challenges. Some of these are new, for example, autonomous systems with major functions driven by machine learning and AI, and ultra-rapid system development, while others are the familiar, persistent issues of the need for efficient, effective and timely assurance. Traditional assurance is seen as a brake on innovation and often costly and time consuming. We therefore propose a modernized framework, Assurance 2.0, as an enabler that supports innovation and continuous incremental assurance. Perhaps unexpectedly, it does so by making assurance more rigorous, with increased focus on the reasoning and evidence employed, and explicit identification of defeaters and counterevidence.

Motivation & Objective

• To address the growing challenges of assuring modern systems, especially AI-driven and rapidly developed systems, where traditional assurance is seen as slow and inhibiting innovation.
• To transform assurance from a post-hoc validation step into an integrated, constructive design activity that supports continuous incremental assurance.
• To strengthen assurance through formal reasoning, explicit evidence evaluation, and systematic identification of doubts and counterarguments (defeaters).
• To establish a framework that supports certification by ensuring claims are not only plausible but also sound and indefeasible through documented reasoning and rebuttal of all credible objections.

Proposed method

• Adopting a Claims-Arguments-Evidence (CAE) structure with five standardized argument blocks (evidence incorporation, calculation, decomposition, substitution, concretion) to reduce ambiguity and error.
• Requiring argument steps to be as deductive as possible, with rigorous side conditions to ensure completeness and distribution over elements (e.g., components or hazards).
• Applying Confirmation Theory to assess the weight of evidence, ensuring it discriminates between a claim and its negation or counterclaim.
• Implementing Natural Language Deductivism (NLD) to ensure logical validity of the overall argument, with all reasoning steps traceable from evidence to top-level claim.
• Systematically identifying, documenting, and defeating potential defeaters—both undercutting and rebutting—through dialectical reasoning and counterclaims.
• Separating system models and theoretical foundations (e.g., static analysis) from the assurance case itself, allowing focus on claim assembly and evaluation.

Experimental results

Research questions

• RQ1 How can assurance be restructured to support innovation in AI and autonomous systems without compromising rigor?
• RQ2 What mechanisms can ensure that assurance arguments are not only plausible but also sound and indefeasible?
• RQ3 How can confirmation bias in assurance be mitigated through systematic exploration of counterclaims and defeaters?
• RQ4 What role do formalized argument templates and evidence weighting play in improving the reliability and auditability of assurance cases?
• RQ5 In what ways can assurance evolve from a static, post-implementation activity to a dynamic, continuous process integrated into system development?

Key findings

• Assurance 2.0 enables more reliable and transparent assurance by requiring deductive reasoning steps supported by complete and well-justified evidence.
• The framework reduces ambiguity and error in assurance case construction by using a standardized set of five argument blocks with strict side conditions.
• By applying Confirmation Theory, the framework ensures that evidence not only supports a claim but also discriminates it from alternatives, enhancing its epistemic strength.
• The indefeasibility criterion ensures that all known doubts and objections are explicitly addressed, and the absence of unexamined doubts is documented, thereby strengthening justification of claims.
• Systematic documentation of defeaters and their defeat through counterclaims or design adjustments makes the assurance case more resilient and auditable.
• The approach supports future automation in tooling for defeater discovery, argument construction, and dialectical evaluation, paving the way for scalable assurance in complex systems.