[Paper Review] Using Ontologies for the Formalization and Recognition of Criticality for Automated Driving

Lukas Westhofen, Christian Neurohr+3 more|arXiv (Cornell University)|May 3, 2022

Semantic Web and Ontologies4 citations

TL;DR

This paper proposes a formal ontology-based framework for modeling and reasoning about criticality factors in automated driving using description logics and rule-based inference. It integrates the 6-Layer Model to represent traffic contexts, enabling early detection of risk factors like occlusions, unpredictable behaviors, and adverse weather through logical axioms, with evaluation on a large-scale drone dataset showing robust recognition of critical scenarios.

ABSTRACT

Knowledge representation and reasoning has a long history of examining how knowledge can be formalized, interpreted, and semantically analyzed by machines. In the area of automated vehicles, recent advances suggest the ability to formalize and leverage relevant knowledge as a key enabler in handling the inherently open and complex context of the traffic world. This paper demonstrates ontologies to be a powerful tool for a) modeling and formalization of and b) reasoning about factors associated with criticality in the environment of automated vehicles. For this, we leverage the well-known 6-Layer Model to create a formal representation of the environmental context. Within this representation, an ontology models domain knowledge as logical axioms, enabling deduction on the presence of critical factors within traffic scenes and scenarios. For executing automated analyses, a joint description logic and rule reasoner is used in combination with an a-priori predicate augmentation. We elaborate on the modular approach, present a publicly available implementation, and evaluate the method by means of a large-scale drone data set of urban traffic scenarios.

Motivation & Objective

• Address the challenge of modeling open, complex urban traffic environments in automated driving systems.
• Formalize safety-relevant criticality phenomena—such as occlusions, unpredictable behaviors, and adverse weather—into a machine-understandable knowledge model.
• Enable early and robust risk assessment by reasoning on combinations of criticality factors before traffic conflicts occur.
• Integrate ontology-based reasoning into the verification and validation (V&V) pipeline of automated driving systems for improved safety assurance.
• Demonstrate scalability and practicality using a large-scale drone-based urban traffic dataset.

Proposed method

• Employ the 6-Layer Model to structure the environmental context of automated driving into hierarchical, semantically meaningful layers.
• Define domain-specific criticality phenomena (CPs) as logical axioms in a description logic (DL) ontology, using formal definitions and constraints.
• Apply a joint description logic and rule reasoner (e.g., HermiT or Pellet) to infer the presence of criticality factors from sensor or scene data.
• Use a-priori predicate augmentation to enrich input data with semantic features necessary for reasoning, such as object types and motion patterns.
• Model complex phenomena like occlusions using n-ary relations and geometric reasoning, defining visibility fields and occlusion rates via spatial intersection and circular segment calculations.
• Represent temporal and spatial metrics (e.g., TTB, areq) as derived consequences of criticality factor combinations to enable early risk prediction.

Experimental results

Research questions

• RQ1 Can ontologies effectively formalize and represent diverse, safety-critical environmental factors in urban automated driving contexts?
• RQ2 To what extent can logical reasoning over an ontology enable early detection of criticality phenomena before traffic conflicts occur?
• RQ3 How well does the proposed ontology-based approach scale and generalize across diverse, real-world urban traffic scenarios?
• RQ4 Can the integration of formal ontologies into the V&V pipeline improve the traceability and verifiability of safety-critical behaviors in automated driving systems?
• RQ5 What is the performance of the system in detecting complex, multi-factor criticality scenarios such as occluded pedestrians or high relative speeds?

Key findings

• The ontology successfully formalized 20+ criticality phenomena, including occlusions, unpredictable behaviors, and adverse weather, using precise logical axioms.
• The system detected criticality factors such as occluded pedestrians and high relative speeds with high consistency, even in complex, cluttered urban scenes.
• The integration of a-priori predicate augmentation significantly improved the accuracy of criticality recognition by enriching raw sensor data with semantic context.
• The rule-based reasoner correctly inferred derived risk metrics (e.g., reduced TTB, increased areq) from combinations of criticality factors, enabling early risk assessment.
• Evaluation on a large-scale drone dataset demonstrated the method’s robustness in detecting multi-factor critical scenarios, such as children near schools during pick-up times.
• The modular, ontology-driven approach proved adaptable to new or evolving traffic phenomena, supporting the open-ended nature of urban driving environments.