[Paper Review] Are We On The Same Page? Hierarchical Explanation Generation for Planning Tasks in Human-Robot Teaming using Reinforcement Learning

Mehrdad Zakershahrak, Samira Ghodratnama|arXiv (Cornell University)|Dec 21, 2020

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes a hierarchical reinforcement learning (H-RL) framework for generating explanations in human-robot teaming that adapt to human cognitive load by abstracting detailed plans into multi-level explanations. By modeling explanation generation as a goal-based Markov Decision Process, the method learns human preferences for information hierarchy, achieving 94.4% accuracy in matching human responses while reducing mental workload and explanation time compared to baseline methods.

ABSTRACT

Providing explanations is considered an imperative ability for an AI agent in a human-robot teaming framework. The right explanation provides the rationale behind an AI agent's decision-making. However, to maintain the human teammate's cognitive demand to comprehend the provided explanations, prior works have focused on providing explanations in a specific order or intertwining the explanation generation with plan execution. Moreover, these approaches do not consider the degree of details required to share throughout the provided explanations. In this work, we argue that the agent-generated explanations, especially the complex ones, should be abstracted to be aligned with the level of details the human teammate desires to maintain the recipient's cognitive load. Therefore, learning a hierarchical explanations model is a challenging task. Moreover, the agent needs to follow a consistent high-level policy to transfer the learned teammate preferences to a new scenario while lower-level detailed plans are different. Our evaluation confirmed the process of understanding an explanation, especially a complex and detailed explanation, is hierarchical. The human preference that reflected this aspect corresponded exactly to creating and employing abstraction for knowledge assimilation hidden deeper in our cognitive process. We showed that hierarchical explanations achieved better task performance and behavior interpretability while reduced cognitive load. These results shed light on designing explainable agents utilizing reinforcement learning and planning across various domains.

Motivation & Objective

• To address the challenge of maintaining human cognitive load during complex planning tasks in human-robot teaming.
• To model human preferences for information hierarchy in explanations, recognizing that not all details are equally relevant at once.
• To develop a generalizable framework for hierarchical explanation generation that balances detail abstraction with interpretability.
• To reduce the mental and temporal workload of human teammates by learning optimal explanation sequences via reinforcement learning.
• To improve task performance and behavior interpretability by aligning explanation structure with human cognitive processing.

Proposed method

• The framework models explanation generation as a goal-based Markov Decision Process (MDP), where states represent abstraction levels of plan details.
• Reinforcement learning is used to learn a policy that selects the next explanation step based on human feedback traces.
• The method employs hierarchical abstraction, where high-level explanations are generated first, followed by progressively detailed sub-explanations.
• The approach integrates explanation generation with plan execution, reducing information overload through progressive disclosure.
• A deterministic policy is used to reduce search space and improve inference efficiency in explanation generation.
• The model is trained on human-robot interaction traces to learn preferences for explanation order and detail level.

Experimental results

Research questions

• RQ1 Do humans exhibit consistent preferences for the hierarchical ordering of explanation details in complex planning tasks?
• RQ2 Can reinforcement learning effectively model human preferences for explanation abstraction levels in human-robot teaming?
• RQ3 Does hierarchical explanation generation reduce human cognitive load compared to flat or progressive non-hierarchical methods?
• RQ4 Can the learned explanation policy generalize across new scenarios while maintaining high task performance and interpretability?
• RQ5 How does the hierarchical explanation framework compare to existing methods in terms of explanation accuracy, time, and workload?

Key findings

• The H-RL framework achieved 94.4% accuracy in matching human responses across 10 scavenger-hunt scenarios, significantly outperforming OEG (86.6%) and PEG (77.5%).
• H-RL reduced the average number of explanation actions to 3.17 per scenario, substantially lower than OEG (9.3) and PEG (9.0), indicating reduced cognitive demand.
• Subjective workload measures (TLX) showed H-RL had the lowest mental and temporal demand, with statistically significant differences (p < 0.05) compared to OEG and PEG.
• Explanation generation time for H-RL averaged 25.37 seconds per scenario, significantly lower than OEG (43.3s) and PEG (35.9s).
• The results confirm that abstraction in explanations is more effective than full or progressive detail disclosure, as PEG performed worse than H-RL despite progressive delivery.
• The study validates that explanation generation is a hierarchical cognitive process, and that modeling this hierarchy improves both interpretability and task performance.