[Paper Review] Challenges of engineering safe and secure highly automated vehicles

Nadja Marko, Eike Möhlmann+4 more|arXiv (Cornell University)|Mar 5, 2021

Autonomous Vehicle Technology and Safety4 citations

TL;DR

This paper identifies and analyzes four core technical challenges hindering the realization of safe and secure highly automated vehicles (HAVs): continuous post-deployment system improvement, handling uncertainties and incomplete information, verification of machine learning components, and reliable prediction. The authors propose targeted approaches to address these issues through interdisciplinary collaboration between industry and academia, aiming to overcome the current 'disillusionment' surrounding autonomous vehicle deployment.

ABSTRACT

After more than a decade of intense focus on automated vehicles, we are still facing huge challenges for the vision of fully autonomous driving to become a reality. The same "disillusionment" is true in many other domains, in which autonomous Cyber-Physical Systems (CPS) could considerably help to overcome societal challenges and be highly beneficial to society and individuals. Taking the automotive domain, i.e. highly automated vehicles (HAV), as an example, this paper sets out to summarize the major challenges that are still to overcome for achieving safe, secure, reliable and trustworthy highly automated resp. autonomous CPS. We constrain ourselves to technical challenges, acknowledging the importance of (legal) regulations, certification, standardization, ethics, and societal acceptance, to name but a few, without delving deeper into them as this is beyond the scope of this paper. Four challenges have been identified as being the main obstacles to realizing HAV: Realization of continuous, post-deployment systems improvement, handling of uncertainties and incomplete information, verification of HAV with machine learning components, and prediction. Each of these challenges is described in detail, including sub-challenges and, where appropriate, possible approaches to overcome them. By working together in a common effort between industry and academy and focusing on these challenges, the authors hope to contribute to overcome the "disillusionment" for realizing HAV.

Motivation & Objective

• To identify and analyze the primary technical obstacles preventing the deployment of safe and secure highly automated vehicles (HAVs).
• To focus on engineering challenges in HAVs, excluding legal, ethical, and societal aspects, to provide a technical foundation for progress.
• To propose actionable approaches for overcoming critical challenges in system improvement, uncertainty management, verification of ML components, and prediction accuracy.
• To foster collaboration between industry and academia to accelerate the realization of trustworthy autonomous cyber-physical systems.

Proposed method

• Systematically identifies four major technical challenges in HAV engineering: continuous post-deployment improvement, handling of uncertainties and incomplete information, verification of machine learning components, and prediction under uncertainty.
• Analyzes sub-challenges within each of the four main challenges, such as data scarcity, model generalization, and runtime verification of AI components.
• Proposes technical approaches such as over-the-air updates, uncertainty-aware AI architectures, formal verification techniques for ML models, and probabilistic prediction frameworks.
• Emphasizes the need for adaptive, self-improving systems that learn safely from real-world data after deployment.
• Highlights the importance of robust testing and validation pipelines that account for edge cases and distributional shifts.
• Advocates for joint industry-academia efforts to develop standardized, scalable, and verifiable engineering practices for HAVs.

Experimental results

Research questions

• RQ1 How can highly automated vehicles be continuously improved after deployment while ensuring safety and security?
• RQ2 What are the key challenges in managing uncertainties and incomplete information in real-world driving environments?
• RQ3 What methods can be used to verify the correctness and reliability of machine learning components in autonomous driving systems?
• RQ4 How can future driving scenarios and system behaviors be accurately predicted to ensure safe decision-making?

Key findings

• Continuous post-deployment improvement remains a major challenge due to the need for safe, secure, and verifiable over-the-air updates and learning mechanisms.
• Handling uncertainties and incomplete information requires robust modeling techniques that account for sensor limitations, environmental variability, and rare events.
• Verification of machine learning components in HAVs is impeded by the lack of formal methods that can guarantee behavior under distributional shifts and edge cases.
• Prediction in autonomous driving is inherently difficult due to long-tail scenarios and the need for high-confidence forecasts over extended time horizons.
• Current approaches to HAV engineering are insufficient for achieving full trustworthiness, necessitating new frameworks that integrate safety, security, and adaptability.
• Collaborative efforts between industry and academia are essential to address these challenges and overcome the current 'disillusionment' in autonomous vehicle development.