[Paper Review] RTAMT: Online Robustness Monitors from STL

Dejan Ničković, Tomoya Yamaguchi|arXiv (Cornell University)|May 24, 2020

Formal Methods in Verification15 references20 citations

TL;DR

This paper presents rtamt, an online monitoring library for Signal Temporal Logic (STL) and its interface-aware variant (IA-STL), enabling real-time robustness monitoring of cyber-physical systems with both discrete- and dense-time semantics. It integrates with ROS via rtamt4ros and demonstrates effectiveness in robotic case studies through online property monitoring, sensitivity analysis, and falsification testing, achieving early violation detection and efficient runtime verification.

ABSTRACT

We present RTAMT, an online monitoring library for Signal Temporal Logic (STL) and its interface-aware variant (IA-STL), providing both discrete- and dense-time interpretation of the logic. We also introduce RTAMT4ROS, a tool that integrates RTAMT with Robotic Operating System (ROS), a common environment for developing robotic applications. We evaluate RTAMT and RTAMT4ROS on two robotic case studies.

Motivation & Objective

• To address the scalability and real-time verification bottleneck in complex robotic cyber-physical systems (CPS).
• To enable online, runtime verification of system properties using quantitative robustness semantics in STL.
• To support both discrete- and dense-time interpretations of STL for accurate monitoring in real-time robotic applications.
• To integrate robustness monitoring into ROS via rtamt4ros for seamless deployment in robotic workloads.
• To enable practical verification tasks such as sensitivity analysis and falsification testing using the monitoring framework.

Proposed method

• rtamt automatically generates online robustness monitors from declarative (IA-)STL specifications using ANTLR4 to parse textual formulas into abstract parse trees (APTs).
• It applies pastification to convert bounded-future STL formulas into equi-satisfiable past STL formulas, enabling online evaluation without clairvoyance.
• The library implements two back-ends: a Python version for prototyping and a C++ version for performance-critical applications, with Boost.Python for interoperability.
• It supports four variants: standard and interface-aware STL with discrete- and dense-time interpretations, using infinity-norm quantitative semantics.
• rtamt4ros integrates the library with ROS, enabling real-time monitoring in robotic systems through custom S-Function blocks in MATLAB Simulink.
• The framework supports online monitoring, sensitivity analysis, and falsification testing by systematically varying input parameters and evaluating robustness over simulation traces.

Experimental results

Research questions

• RQ1 Can online robustness monitoring of STL specifications be efficiently and accurately performed in real-time robotic systems?
• RQ2 How can pastification be effectively applied to enable online monitoring of bounded-future STL formulas without requiring future knowledge?
• RQ3 To what extent can rtamt and rtamt4ros support practical verification tasks such as sensitivity analysis and falsification in robotic CPS?
• RQ4 How does the integration of rtamt with ROS and MATLAB Simulink enable early detection of property violations during simulation?
• RQ5 What is the performance and accuracy trade-off between discrete-time and dense-time monitoring in real-world robotic applications?

Key findings

• rtamt successfully generates online robustness monitors from (IA-)STL specifications, enabling real-time evaluation of how far a system's behavior is from satisfying or violating a property.
• The pastification procedure allows online monitoring of bounded-future STL formulas by transforming them into past-only formulas, eliminating the need for clairvoyant monitoring.
• The integration of rtamt with ROS via rtamt4ros enables deployment of specification-based runtime verification in robotic applications, supporting real-time anomaly detection.
• In MATLAB Simulink, the online monitoring block stops simulations upon property violation, significantly reducing simulation time when violations are detected early.
• Sensitivity analysis and falsification testing using rtamt revealed critical parameter dependencies in the aircraft elevator control system, with heat-maps visualizing robustness across input variations.
• The framework demonstrated effective detection of system failures, such as loss of tracking after 4 seconds in the AECS case study, confirming its utility in safety-critical validation.