[Paper Review] Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities

Yousef Emami, Lúıs Almeida+3 more|arXiv (Cornell University)|Aug 22, 2024

Autonomous Vehicle Technology and Safety4 citations

TL;DR

This paper proposes Human-In-The-Loop Machine Learning (HITL-ML) for safe and ethical autonomous vehicles by integrating human expertise into curriculum learning, reinforcement learning, and active learning. It enhances model robustness, ethical decision-making, and public trust through human oversight, with key contributions in safety, security, and regulatory frameworks.

ABSTRACT

Rapid advances in Machine Learning (ML) have triggered new trends in Autonomous Vehicles (AVs). ML algorithms play a crucial role in interpreting sensor data, predicting potential hazards, and optimizing navigation strategies. However, achieving full autonomy in cluttered and complex situations, such as intricate intersections, diverse sceneries, varied trajectories, and complex missions, is still challenging, and the cost of data labeling remains a significant bottleneck. The adaptability and robustness of humans in complex scenarios motivate the inclusion of humans in the ML process, leveraging their creativity, ethical power, and emotional intelligence to improve ML effectiveness. The scientific community knows this approach as Human-In-The-Loop Machine Learning (HITL-ML). Towards safe and ethical autonomy, we present a review of HITL-ML for AVs, focusing on Curriculum Learning (CL), Human-In-The-Loop Reinforcement Learning (HITL-RL), Active Learning (AL), and ethical principles. In CL, human experts systematically train ML models by starting with simple tasks and gradually progressing to more difficult ones. HITL-RL significantly enhances the RL process by incorporating human input through techniques like reward shaping, action injection, and interactive learning. AL streamlines the annotation process by targeting specific instances that need to be labeled with human oversight, reducing the overall time and cost associated with training. Ethical principles must be embedded in AVs to align their behavior with societal values and norms. In addition, we provide insights and specify future research directions.

Motivation & Objective

• Address the limitations of fully autonomous ML in complex, real-world driving scenarios where training data may not cover rare or culturally specific behaviors.
• Improve model generalization and safety by incorporating human feedback in curriculum learning, reinforcement learning, and active learning workflows.
• Ensure ethical alignment of AV behavior with societal norms through human-in-the-loop integration of moral reasoning and contextual judgment.
• Develop secure, accountable, and transparent HITL-ML systems that support human oversight while minimizing bias and ensuring regulatory compliance.

Proposed method

• Employ Curriculum Learning (CL) to progressively train AV models on increasingly complex driving scenarios, starting from simple tasks to complex urban intersections.
• Integrate Human-In-The-Loop Reinforcement Learning (HITL-RL) using reward shaping, action injection, and interactive learning to refine policy learning with human feedback.
• Apply Active Learning (AL) to prioritize uncertain or ambiguous data samples for human annotation, reducing labeling costs and improving model efficiency.
• Embed ethical principles into AV decision-making by incorporating human-defined values and norms into reward functions and behavior policies.
• Implement robust security mechanisms including end-to-end encryption, two-way authentication, and post-quantum cryptography (e.g., Ring Learning with Errors) to prevent session hijacking and tampering.
• Establish regulatory and documentation frameworks to define human operator responsibilities, ensure legal accountability, and protect privacy in HITL-ML systems.

Experimental results

Research questions

• RQ1 How can human-in-the-loop mechanisms improve the safety and adaptability of autonomous vehicles in complex, real-world driving environments?
• RQ2 What role does human feedback play in enhancing ethical decision-making during unavoidable accident scenarios in AVs?
• RQ3 How can active learning and curriculum learning reduce data labeling costs while improving model performance in autonomous driving?
• RQ4 What security and privacy mechanisms are necessary to protect human operators and AV systems from cyber threats in HITL-ML pipelines?
• RQ5 How can regulatory frameworks be designed to ensure accountability, transparency, and ethical compliance in HITL-ML-enabled autonomous vehicles?

Key findings

• HITL-RL significantly improves safety and public trust by enabling human-in-the-loop refinement of reward functions, especially in edge cases like unfamiliar traffic behaviors.
• Active Learning reduces labeling costs and improves performance in object detection, anomaly detection, and semantic mapping by prioritizing informative samples for human annotation.
• Curriculum Learning enhances model convergence and performance by structuring training from simple to complex driving tasks, improving generalization in diverse environments.
• Human input, when properly validated, enhances scene understanding and self-recognition in AVs, particularly in regions with non-standard traffic rules or cultural driving norms.
• Robust security protocols, including post-quantum cryptography and mutual authentication, are essential to prevent session hijacking and ensure secure human-machine interaction.
• Regulatory frameworks that define human operator roles, intervention protocols, and data accountability are critical for legal compliance and public acceptance of HITL-ML in AVs.