[Paper Review] ExperienceWeaver: Optimizing Small-sample Experience Learning for LLM-based Clinical Text Improvement

Ziyan Xiao, Yinghao Zhu+2 more|arXiv (Cornell University)|Jan 31, 2026

Artificial Intelligence in Healthcare and Education0 citations

TL;DR

ExperienceWeaver introduces a hierarchical experience distillation framework that converts multi-dimensional clinical feedback into error-specific tips and high-level strategies, enabling small-sample LLM-based clinical text improvements and outperforming several SOTA models in experiments.

ABSTRACT

Clinical text improvement is vital for healthcare efficiency but remains difficult due to limited high-quality data and the complex constraints of medical documentation. While Large Language Models (LLMs) show promise, current approaches struggle in small-sample settings: supervised fine-tuning is data-intensive and costly, while retrieval-augmented generation often provides superficial corrections without capturing the reasoning behind revisions. To address these limitations, we propose ExperienceWeaver, a hierarchical framework that shifts the focus from data retrieval to experience learning. Instead of simply recalling past examples, ExperienceWeaver distills noisy, multi-dimensional feedback into structured, actionable knowledge. Specifically, error-specific Tips and high-level Strategies. By injecting this distilled experience into an agentic pipeline, the model learns "how to revise" rather than just "what to revise". Extensive evaluations across four clinical datasets demonstrate that ExperienceWeaver consistently improves performance, surpassing state-of-the-art models such as Gemini-3 Pro in small-sample settings.

Motivation & Objective

• Motivate the need for improved clinical text improvement under data scarcity and heterogeneous clinician expectations.
• Propose a paradigm shift from retrieval-based methods to distillation-based experience learning.
• Develop a two-stage experience weaving process that produces actionable Tips and high-level Strategies.
• Integrate distilled experience into an agentic, multi-agent revision pipeline for error detection, revision, and self-critique.
• Demonstrate effectiveness across multiple clinical datasets and compare against strong baselines in small-sample settings.

Proposed method

• Two-stage Experience Weaver: Stage 1 distills raw feedback into structured experiences via Experience Abstraction and Experience Combination guided by LLMs.
• Stage 2 re-weaves distilled experiences into two layers: error-specific Tips with supporting cases and functional Strategies for each phase (Detection, Revision, Self-Critique).
• An Experience Retriever injects layered experiences into the agentic pipeline with a maximum tip limit to balance informativeness and conciseness.
• An agentic framework follows a multi-agent ReAct-style setup with Error Detection, Revision, and Self-Critique agents coordinating via shared memory, RAG, and the Experience Retriever.
• Feedback is multi-dimensional (Correctness, Formatting, Meaningfulness, Readability) and informs both evaluation and guidance prompts.
• Hyperparameters (group size Ng, error-frequency threshold τe, maximum retrieved tips τt) control the granularity and efficiency of weaving and retrieval.

Experimental results

Research questions

• RQ1 How reliable is an LLM-as-a-Judge framework for evaluating clinical text quality and which base models are most stable?
• RQ2 Can ExperienceWeaver transform multi-dimensional, small-sample feedback into actionable experience that improves clinical text revision quality?
• RQ3 How effective is ExperienceWeaver in small-sample clinical text processing across diverse data sources and languages?
• RQ4 What factors influence ExperienceWeaver performance, such as feedback volume and weaving granularity?

Key findings

• ExperienceWeaver consistently improves text quality compared with strong baselines including Gemini-3 Pro and GPT-5.1 across four clinical datasets.
• Average improvements across datasets on a 5-point Likert scale were 0.794 (English Chest X-ray), 0.456 (Chinese Abdominal CT), 0.412 (Discharge), and 0.137 (Clinical Free Text).
• In real-world validation, incorporating woven experience enhanced error detection metrics (accuracy, precision, recall) by up to 20.4%, 16.3%, and 28.6%, respectively.
• Higher stability was observed for a moderate tip length (τ = 3) in small-sample validation tasks.
• The approach outperformed several single-shot LLM baselines and RAG-based methods, demonstrating strong performance in small-sample clinical text improvement.