[Paper Review] Evaluating 5W3H Structured Prompting for Intent Alignment in Human-AI Interaction

Peng Gang|arXiv (Cornell University)|Mar 19, 2026

Human-Automation Interaction and Safety0 citations

TL;DR

The paper evaluates the 5W3H-based PSP (PPS) structured prompting framework for better user-intent alignment in human-AI interactions, comparing rendered PPS with simple prompts and raw JSON across multiple models and domains.

ABSTRACT

Natural language prompts often suffer from intent transmission loss: the gap between what users actually need and what they communicate to AI systems. We evaluate PPS (Prompt Protocol Specification), a 5W3H-based framework for structured intent representation in human-AI interaction. In a controlled three-condition study across 60 tasks in three domains (business, technical, and travel), three large language models (DeepSeek-V3, Qwen-Max, and Kimi), and three prompt conditions - (A) simple prompts, (B) raw PPS JSON, and (C) natural-language-rendered PPS - we collect 540 AI-generated outputs evaluated by an LLM judge. We introduce goal_alignment, a user-intent-centered evaluation dimension, and find that rendered PPS outperforms both simple prompts and raw JSON on this metric. PPS gains are task-dependent: gains are large in high-ambiguity business analysis tasks but reverse in low-ambiguity travel planning. We also identify a measurement asymmetry in standard LLM evaluation, where unconstrained prompts can inflate constraint adherence scores and mask the practical value of structured prompting. A preliminary retrospective survey (N = 20) further suggests a 66.1% reduction in follow-up prompts required, from 3.33 to 1.13 rounds. These findings suggest that structured intent representations can improve alignment and usability in human-AI interaction, especially in tasks where user intent is inherently ambiguous.

Motivation & Objective

• Address how structured intent representation affects alignment between user goals and AI outputs.
• Quantify performance of 5W3H-based PPS prompts versus simple prompts and raw JSON across tasks and domains.
• Investigate task-dependency of gains and potential measurement biases in LLM evaluation.
• Provide a preliminary user survey on how PPS impacts interaction efficiency.

Proposed method

• Use PPS, a 5W3H-based framework, to represent intents in structured prompts.
• Conduct a controlled three-condition study with 60 tasks across three domains.
• Evaluate 540 AI outputs from three LLMs under three prompt conditions: simple prompts, raw PPS JSON, and natural-language-rendered PPS.
• Develop a goal_alignment metric to assess user-intent-centered alignment.
• Use an LLM judge to evaluate outputs on alignment and usefulness.
• Include a preliminary retrospective survey (N=20) on follow-up prompts.

Experimental results

Research questions

• RQ1 How does rendered PPS compare to simple prompts and raw JSON in achieving goal alignment?
• RQ2 Are the benefits of PPS consistent across task types and domains (business, technical, travel)?
• RQ3 Do structured prompts introduce or reveal biases in standard LLMevaluation measures?
• RQ4 What is the impact of PPS on user-effort, measured by follow-up prompts?

Key findings

• Rendered PPS outperforms both simple prompts and raw JSON on the goal_alignment metric.
• PPS gains are task-dependent: large in high-ambiguity business analysis tasks but reverse in low-ambiguity travel planning.
• There is a measurement asymmetry in standard LLM evaluation, where unconstrained prompts can inflate constraint adherence scores.
• Preliminary survey (N=20) suggests a 66.1% reduction in follow-up prompts, from 3.33 to 1.13 rounds.