[Paper Review] How Human-Centered Explainable AI Interface Are Designed and Evaluated: A Systematic Survey

Thu Nguyen, Alessandro Canossa+1 more|arXiv (Cornell University)|Mar 21, 2024

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This systematic survey analyzes 53 studies on human-centered explainable AI (XAI) interfaces (EIs), identifying design and evaluation practices for improving usability, interpretability, and user efficacy. It reveals key trends in user involvement, interface design (e.g., sequential information structure, instructing interaction), and evaluation methods, while proposing a generative framework to guide future EI development through underexplored design combinations.

ABSTRACT

Despite its technological breakthroughs, eXplainable Artificial Intelligence (XAI) research has limited success in producing the {\em effective explanations} needed by users. In order to improve XAI systems' usability, practical interpretability, and efficacy for real users, the emerging area of {\em Explainable Interfaces} (EIs) focuses on the user interface and user experience design aspects of XAI. This paper presents a systematic survey of 53 publications to identify current trends in human-XAI interaction and promising directions for EI design and development. This is among the first systematic survey of EI research.

Motivation & Objective

• Address the gap in XAI research where technical explainability outpaces user usability and practical interpretability.
• Investigate how human-centered design principles are applied in XAI user interfaces (EIs) to improve real-world usability.
• Identify current practices and promising directions in the design and evaluation of EIs through a systematic review of empirical studies.
• Provide a structured framework to guide future EI development by analyzing underexplored design combinations and user-centered evaluation methods.

Proposed method

• Conducted a systematic literature review using PRISMA guidelines to identify and analyze 53 publications on explainable interfaces (EIs).
• Categorized studies based on user involvement (e.g., user studies, personas, task modeling), interface design (e.g., information structure, interaction type, interactivity), and evaluation methods.
• Applied cluster analysis to group studies by design and evaluation patterns, revealing dominant and underexplored combinations of interface properties.
• Used a framework of eight design and evaluation properties (e.g., information structure, interaction type, interactivity) to systematically map current EI research.
• Reviewed reported user participation practices, including qualitative methods like personas and task modeling, and evaluated how these informed design decisions.
• Proposed a generative framework by identifying non-represented property combinations (e.g., static interactivity with hierarchical structure) as opportunities for future EI innovation.

Experimental results

Research questions

• RQ1 RQ1: How do researchers involve human participants in the design and development of XAI applications?
• RQ2 RQ2: How do XAI researchers design EIs in terms of structure, format, and interaction?
• RQ3 RQ3: How do XAI researchers evaluate EIs in practice?
• RQ4 RQ4: What are the dominant and underexplored combinations of EI design properties?
• RQ5 RQ5: How can the findings inform future design of more effective and user-centered EIs?

Key findings

• A majority of EI studies (over 60%) use sequential information structure, while hierarchical structures are rare, indicating a design imbalance.
• Instructing interaction types are dominant (majority of studies), whereas conversational or exploratory interaction modes are underutilized.
• Only a small fraction of studies (under 10%) implement static interactivity with hierarchical information architecture, suggesting a significant design gap.
• User involvement is often limited to informal feedback or post-hoc evaluation; few studies use structured methods like personas or task modeling in early design stages.
• Evaluation methods are predominantly quantitative and focused on trust or accuracy, with limited use of qualitative or longitudinal assessment of interpretability.
• The analysis reveals that no studies combine static interactivity with hierarchical information architecture, indicating a potential opportunity for future EI innovation.