[Paper Review] Coverage-Driven Verification - An approach to verify code for robots that directly interact with humans

Dejanira Araiza-Illan, David Western+2 more|arXiv (Cornell University)|Sep 16, 2015

Formal Methods in Verification31 references20 citations

TL;DR

This paper proposes Coverage-Driven Verification (CDV) for robotic code in human-robot interaction (HRI), using a simulation-based testbench with automated test generation, assertion checking, and coverage collection to achieve systematic verification. The approach demonstrates feasibility and effectiveness in verifying safety and liveness requirements for a robot object-handover task using ROS and Gazebo.

ABSTRACT

Collaborative robots could transform several industries, such as manufacturing and healthcare, but they present a significant challenge to verification. The complex nature of their working environment necessitates testing in realistic detail under a broad range of circumstances. We propose the use of Coverage-Driven Verification (CDV) to meet this challenge. By automating the simulation-based testing process as far as possible, CDV provides an efficient route to coverage closure. We discuss the need, practical considerations, and potential benefits of transferring this approach from microelectronic design verification to the field of human-robot interaction. We demonstrate the validity and feasibility of the proposed approach by constructing a custom CDV testbench and applying it to the verification of an object handover task.

Motivation & Objective

• Address the challenge of verifying complex, concurrent robotic control code in human-robot interaction (HRI) systems.
• Overcome limitations of formal methods and manual testing by enabling systematic, scalable verification under resource constraints.
• Develop a practical, automated testbench for HRI that integrates with existing robotics tools like ROS and Gazebo.
• Enable coverage closure for both functional (liveness) and safety requirements in a realistic simulation environment.
• Demonstrate the feasibility and benefits of CDV in a real-world HRI scenario—robot-assisted object handover.

Proposed method

• Design a CDV testbench with four core components: Test Generator, Driver, Checker, and Coverage Collector.
• Implement model-based, constrained pseudorandom, and pseudorandom test generation strategies to explore diverse system behaviors.
• Use assertion monitors in the Checker to validate requirements at multiple abstraction levels, including safety and liveness properties.
• Collect multiple coverage metrics: requirement coverage (via assertions), code statement coverage, and cross-product coverage.
• Integrate the testbench with ROS and Gazebo to simulate the robot’s actual control code in a realistic environment.
• Apply feedback mechanisms to guide test generation toward uncovered requirements and edge cases, though full automation of feedback loops remains challenging.

Experimental results

Research questions

• RQ1 Can Coverage-Driven Verification (CDV) be effectively adapted from microelectronics to human-robot interaction (HRI) robotics?
• RQ2 How can CDV be implemented in a robotics context using existing tools like ROS and Gazebo to achieve systematic test coverage?
• RQ3 What combination of test generation strategies (model-based, pseudorandom, constrained) best balances coverage and discovery of unexpected behaviors?
• RQ4 To what extent can assertion-based checking and multi-level coverage metrics support verification of safety and liveness requirements in HRI?
• RQ5 How scalable and reusable is the proposed CDV testbench for more complex HRI tasks and systems?

Key findings

• The CDV testbench successfully verified key safety and liveness requirements for a robot object-handover task in a ROS-Gazebo simulation.
• Model-based test generation efficiently targeted requirements, while pseudorandom generation uncovered unanticipated system behaviors and edge cases.
• The combination of test generation strategies provided complementary strengths, improving overall coverage and reducing blind spots.
• Assertion monitors enabled effective requirement checking at different abstraction levels, supporting verification across system layers.
• The testbench demonstrated feasibility for real robot code execution in simulation, enabling direct validation of actual control software.
• The approach supports scalability and extensibility, with potential for integration with Hardware-in-the-Loop or Human-in-the-Loop simulation.