[论文解读] Query Suggestion for Retrieval-Augmented Generation via Dynamic In-Context Learning

Fabian Spaeh, Tianyi Chen+2 more|arXiv (Cornell University)|Jan 13, 2026

Topic Modeling被引用 0

一句话总结

本文研究面向具备工具调用的代理式检索增强生成（agentic RAG）的查询推荐，提出一种带模板化与自学习的动态少量学习方法，以生成可回答、语义相似的后续查询，显著优于基线。

ABSTRACT

Retrieval-augmented generation with tool-calling agents (agentic RAG) has become increasingly powerful in understanding, processing, and responding to user queries. However, the scope of the grounding knowledge is limited and asking questions that exceed this scope may lead to issues like hallucination. While guardrail frameworks aim to block out-of-scope questions (Rodriguez et al., 2024), no research has investigated the question of suggesting answerable queries in order to complete the user interaction. In this paper, we initiate the study of query suggestion for agentic RAG. We consider the setting where user questions are not answerable, and the suggested queries should be similar to aid the user interaction. Such scenarios are frequent for tool-calling LLMs as communicating the restrictions of the tools or the underlying datasets to the user is difficult, and adding query suggestions enhances the interaction with the RAG agent. As opposed to traditional settings for query recommendations such as in search engines, ensuring that the suggested queries are answerable is a major challenge due to the RAG's multi-step workflow that demands a nuanced understanding of the RAG as a whole, which the executing LLM lacks. As such, we introduce robust dynamic few-shot learning which retrieves examples from relevant workflows. We show that our system can be self-learned, for instance on prior user queries, and is therefore easily applicable in practice. We evaluate our approach on three benchmark datasets based on two unlabeled question datasets collected from real-world user queries. Experiments on real-world datasets confirm that our method produces more relevant and answerable suggestions, outperforming few-shot and retrieval-only baselines, and thus enable safer, more effective user interaction with agentic RAG.

研究动机与目标

• 定义在具备工具调用的代理式检索增强生成（RAG）中，无法回答的用户查询问题。
• 将查询可回答性分为可执行类别，并以工作流可行性为基础进行推荐。
• 提出一种动态少量学习框架，检索并使用相关示例来引导查询建议。
• 实现自学习：从真实用户查询中自动标注训练示例，无需人工标注。
• 通过在真实世界数据集上的评估，展示实际部署潜力及相较基线的改进。

提出的方法

• 将查询模板化以映射到可执行工作流，通过屏蔽数值并保留工作流结构。
• 鲁棒的动态少量检索：使用嵌入式相似性和局部多数投票选取少量多样的正负示例来判断可回答性。
• 生成可回答的查询模板，然后进行数值填充以产生具体、可回答的后续查询。
• 自学习：RAG 助手在执行后为查询标注可回答性，从而实现对历史用户查询的无监督自适应。
• 在真实数据集上对比静态少量学习和仅检索的基线，以评估可回答性和语义相似性。

实验结果

研究问题

• RQ1 当原始查询不可回答时，如何为具备代理能力的 RAG 生成可回答且语义相似的后续查询？
• RQ2 与静态少量学习和仅检索基线相比，动态少量学习结合模板检索是否能提升可回答性与相似性？
• RQ3 是否可以利用 RAG 系统自标注训练数据，从而实现实际可用的无监督学习？
• RQ4 代理式 RAG 中出现哪些可回答性类别，如何检测并将其用于查询建议？

主要发现

• 动态少量学习在多个真实世界数据集上产生了更可回答且更相似的查询建议。
• 仅检索基线在相似性上有所提升，但由于幻觉及工具/数据限制，可能降低可回答性。
• 静态少量学习在相似性与可回答性之间的平衡困难，凸显需要动态检索。
• 自学习使得在未标注的历史查询上实现实际部署成为可能，降低标注需求。
• 在三个带无标签训练数据的基准数据集上，该方法始终优于基线。
• 该方法通过使建议与可执行工作流对齐，提升了更安全且更高效的用户交互。