[Paper Review] AI-in-the-Loop -- The impact of HMI in AI-based Application

Julius Schöning, Clemens Westerkamp|arXiv (Cornell University)|Mar 21, 2023

Age of Information Optimization4 citations

TL;DR

This paper proposes an AI-in-the-loop framework that integrates human-machine interaction (HMI) early in AI development to design lightweight, efficient neural architectures for embedded systems. By using intuitive HMI during AI architecture design, the approach identifies and removes unproductive layers, reducing computational overhead and enabling real-time, explainable AI inference with human oversight when uncertainty arises.

ABSTRACT

Artificial intelligence (AI) and human-machine interaction (HMI) are two keywords that usually do not fit embedded applications. Within the steps needed before applying AI to solve a specific task, HMI is usually missing during the AI architecture design and the training of an AI model. The human-in-the-loop concept is prevalent in all other steps of developing AI, from data analysis via data selection and cleaning to performance evaluation. During AI architecture design, HMI can immediately highlight unproductive layers of the architecture so that lightweight network architecture for embedded applications can be created easily. We show that by using this HMI, users can instantly distinguish which AI architecture should be trained and evaluated first since a high accuracy on the task could be expected. This approach reduces the resources needed for AI development by avoiding training and evaluating AI architectures with unproductive layers and leads to lightweight AI architectures. These resulting lightweight AI architectures will enable HMI while running the AI on an edge device. By enabling HMI during an AI uses inference, we will introduce the AI-in-the-loop concept that combines AI's and humans' strengths. In our AI-in-the-loop approach, the AI remains the working horse and primarily solves the task. If the AI is unsure whether its inference solves the task correctly, it asks the user to use an appropriate HMI. Consequently, AI will become available in many applications soon since HMI will make AI more reliable and explainable.

Motivation & Objective

• Address the lack of human-in-the-loop (HMI) in early AI development stages, particularly during architecture design for embedded applications.
• Reduce computational overhead in AI model development by identifying and eliminating unproductive layers in convolutional neural networks (CNNs) through interactive HMI.
• Enable reliable, safe, and explainable AI in embedded systems by integrating minimal HMI during inference, allowing human experts to intervene when the AI is uncertain.
• Accelerate the deployment of AI in safety- and security-critical embedded applications by shifting from full automation to cooperative AI-in-the-loop systems.
• Bridge the gap between high-complexity AI models and resource-constrained embedded hardware through lightweight, HMI-optimized architectures.

Proposed method

• Introduce a user interface (UI) that visualizes and evaluates individual CNN layers for productivity during architecture design, enabling immediate identification of redundant or unproductive components.
• Apply the Human-Machine Interaction (HMI) concept during AI architecture design to guide developers in selecting only essential layers, leading to lightweight, efficient neural networks.
• Use interactive visualization techniques to translate high-dimensional AI representations into human-understandable problem descriptions, supporting informed design decisions.
• Implement an AI-in-the-loop inference mechanism where the AI proactively requests human input when confidence in prediction drops below a threshold, ensuring safety and reliability.
• Integrate minimal HMI into embedded AI systems to support domain experts in validating or correcting AI decisions in real time, especially in critical or ambiguous situations.
• Leverage existing explainability techniques (e.g., saliency maps, attention visualization) to inform HMI design and improve transparency during inference.

Experimental results

Research questions

• RQ1 How can HMI be effectively integrated during the AI architecture design phase to reduce computational complexity in embedded AI systems?
• RQ2 What role does interactive HMI play in identifying unproductive layers in CNN architectures before training?
• RQ3 To what extent can lightweight AI architectures be developed through HMI-guided design, and how do they impact inference efficiency on edge devices?
• RQ4 How does the AI-in-the-loop concept improve reliability and explainability in safety-critical embedded AI applications?
• RQ5 What are the practical benefits of introducing minimal HMI during AI inference, particularly in reducing the risk of undetected errors?

Key findings

• The integration of HMI during AI architecture design enables immediate identification of unproductive layers, significantly reducing model complexity and computational requirements.
• By removing redundant layers early, the resulting lightweight AI architectures are better suited for deployment on resource-constrained embedded systems.
• The AI-in-the-loop approach reduces the need for extensive training and evaluation of suboptimal architectures, cutting development time and resource usage.
• Interactive HMI during inference allows domain experts to intervene when the AI is uncertain, enhancing system reliability and safety in critical applications.
• The proposed method supports the development of explainable AI by translating complex model representations into human-understandable feedback through UI design.
• The study demonstrates that minimal HMI integration during AI inference is a feasible and effective step toward deploying safe, secure, and reliable AI in embedded systems.