[Paper Review] An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software

Rick Salay, Rodrigo Queiroz+1 more|arXiv (Cornell University)|Sep 7, 2017

Adversarial Robustness in Machine Learning20 references106 citations

TL;DR

The paper analyzes how ML-based software affects ISO 26262 and offers recommendations to adapt the standard for safe ML in automotive systems.

ABSTRACT

Machine learning (ML) plays an ever-increasing role in advanced automotive functionality for driver assistance and autonomous operation; however, its adequacy from the perspective of safety certification remains controversial. In this paper, we analyze the impacts that the use of ML as an implementation approach has on ISO 26262 safety lifecycle and ask what could be done to address them. We then provide a set of recommendations on how to adapt the standard to accommodate ML.

Motivation & Objective

• Identify how ML-based software impacts ISO 26262 during hazard analysis and software development phases.
• Highlight ML-specific challenges such as non-transparency, error rates, training data, and stability.
• Propose concrete recommendations to adapt ISO 26262 to accommodate ML technologies.
• Discuss implications for hazard identification, fault analysis, training data requirements, and software techniques.

Proposed method

• Review and analyze ISO 26262 Part 3 and Part 6 with respect to ML components.
• Characterize ML characteristics that affect safety assessment (non-transparency, error rate, training-based development, instability).
• Map ML impacts to ISO 26262 processes and identify five areas of impact.
• Propose targeted recommendations for hazard identification, fault analysis, training data, ML usage scope, and software techniques.

Experimental results

Research questions

• RQ1 What are the ML-specific hazards and safety concerns not fully addressed by ISO 26262?
• RQ2 How does ML usage affect hazard analysis and the software development lifecycle under ISO 26262?
• RQ3 What adaptations to the standard are recommended to safely accommodate ML in automotive software?
• RQ4 What are the implications for training data, fault detection, and architectural decisions when using ML?

Key findings

• ML can create new hazards stemming from human–machine interaction and complex behavioral interactions that are not simply malfunctions.
• ML components have fault types and failure modes that differ from traditional software and require ML-specific fault detection tools.
• Training data and the ML lifecycle violate the assumption of fully specified behavior in the V-model, necessitating different safety requirements based on functionality.
• End-to-end ML approaches challenge ISO 26262 assumptions about modular hierarchical architectures and may be unsuitable under current guidance.
• Many unit-level techniques in Part 6 remain applicable to ML, but a substantial portion are not directly applicable or require adaptation.
• The standard bias toward imperative programming languages creates gaps for ML components, suggesting a shift toward technique-intent-based requirements.