[Paper Review] Mapping the Design Space of Human-AI Interaction in Text Summarization

Ruijia Cheng, Alison Smith+3 more|arXiv (Cornell University)|Jun 29, 2022

Topic Modeling4 citations

TL;DR

This paper maps the design space of human-AI interaction in text summarization by analyzing 70 papers to develop a taxonomy of five interaction types and associated design dimensions. Through prototyping and user interviews with 16 participants, it identifies key design considerations for efficiency, control, and trust in AI-assisted summarization, offering actionable insights for human-centered AI system design in text generation tasks.

ABSTRACT

Automatic text summarization systems commonly involve humans for preparing data or evaluating model performance, yet, there lacks a systematic understanding of humans' roles, experience, and needs when interacting with or being assisted by AI. From a human-centered perspective, we map the design opportunities and considerations for human-AI interaction in text summarization and broader text generation tasks. We first conducted a systematic literature review of 70 papers, developing a taxonomy of five interactions in AI-assisted text generation and relevant design dimensions. We designed text summarization prototypes for each interaction. We then interviewed 16 users, aided by the prototypes, to understand their expectations, experience, and needs regarding efficiency, control, and trust with AI in text summarization and propose design considerations accordingly.

Motivation & Objective

• To systematically understand the roles, experiences, and needs of humans interacting with or being assisted by AI in text summarization.
• To identify and categorize recurring interaction patterns between humans and AI in text generation tasks.
• To develop design dimensions that support effective human-AI collaboration in summarization workflows.
• To evaluate user expectations, experiences, and needs regarding efficiency, control, and trust in AI-assisted summarization.
• To propose concrete design considerations grounded in empirical user feedback for future AI system development.

Proposed method

• Conducted a systematic literature review of 70 papers on human-AI interaction in text summarization and text generation.
• Developed a taxonomy of five distinct interaction types in AI-assisted text generation, each with associated design dimensions.
• Designed and implemented interactive prototypes for each of the five interaction types to simulate real-world use cases.
• Recruited 16 users to conduct interviews using the prototypes, focusing on their experiences with AI in summarization tasks.
• Analyzed interview data to extract themes related to efficiency, control, and trust in human-AI collaboration.
• Synthesized findings into a set of actionable design considerations for building human-centered AI systems in text summarization.

Experimental results

Research questions

• RQ1 What are the primary interaction types between humans and AI in text summarization tasks?
• RQ2 What design dimensions are most relevant to supporting effective human-AI collaboration in text generation?
• RQ3 How do users perceive efficiency, control, and trust when interacting with AI in summarization workflows?
• RQ4 What are the key user expectations and pain points in AI-assisted summarization?
• RQ5 How can design principles be derived from user experiences to guide future system development?

Key findings

• The study identified five distinct interaction types in AI-assisted text summarization: input preparation, model refinement, result selection, feedback provision, and outcome validation.
• Users emphasized the need for granular control over AI-generated summaries, particularly in editing and adjusting output.
• Trust in AI output was significantly influenced by transparency in the model’s reasoning and the ability to trace changes.
• Efficiency was most improved when users could quickly identify and apply relevant edits, suggesting the value of intelligent suggestion interfaces.
• Users expressed concern about over-reliance on AI, highlighting the importance of maintaining human oversight and decision-making authority.
• The design dimensions—such as editability, traceability, and feedback mechanisms—were consistently cited as critical for effective collaboration.