[論文レビュー] AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents

Yixi Lin, Jiangrong Wu+4 more|arXiv (Cornell University)|Mar 8, 2026

Advanced Malware Detection Techniques被引用数 0

ひとこと要約

AgentRaft は、クロスツール機能呼び出しグラフを構築し、決定論的プロンプトを合成し、マルチLLM投票委員会を使用して GDPR/CCPA/PIPL に基づいてデータフローを監査することで、LLM エージェントにおけるデータ過露出（DOE）を自動検出します。DOE が一般的で、スケールで効率的に検出可能であることを示します。

ABSTRACT

The rapid integration of Large Language Model (LLM) agents into autonomous task execution has introduced significant privacy concerns within cross-tool data flows. In this paper, we systematically investigate and define a novel risk termed Data Over-Exposure (DOE) in LLM Agent, where an Agent inadvertently transmits sensitive data beyond the scope of user intent and functional necessity. We identify that DOE is primarily driven by the broad data paradigms in tool design and the coarse-grained data processing inherent in LLMs. In this paper, we present AgentRaft, the first automated framework for detecting DOE risks in LLM agents. AgentRaft combines program analysis with semantic reasoning through three synergistic modules: (1) it constructs a Cross-Tool Function Call Graph (FCG) to model the interaction landscape of heterogeneous tools; (2) it traverses the FCG to synthesize high-quality testing user prompts that act as deterministic triggers for deep-layer tool execution; and (3) it performs runtime taint tracking and employs a multi-LLM voting committee grounded in global privacy regulations (e.g., GDPR, CCPA, PIPL) to accurately identify privacy violations. We evaluate AgentRaft on a testing environment of 6,675 real-world agent tools. Our findings reveal that DOE is indeed a systemic risk, prevalent in 57.07% of potential tool interaction paths. AgentRaft achieves a high detection accuracy and effectiveness, outperforming baselines by 87.24%. Furthermore, AgentRaft reaches near-total DOE coverage (99%) within only 150 prompts while reducing per-chain verification costs by 88.6%. Our work provides a practical foundation for building auditable and privacy-compliant LLM agent systems.

研究の動機と目的

• Formally define Data Over-Exposure (DOE) in LLM Agents and quantify its risk across cross-tool data flows.
• Develop AgentRaft to automatically detect DOE using a cross-tool function call graph, prompt synthesis, and runtime data-flow tainting.
• Enforce privacy compliance through a multi-LLM voting committee guided by GDPR, CCPA, and PIPL.
• Evaluate AgentRaft on a large real-world toolset to measure DOE coverage, detection efficiency, and auditing cost reductions.

提案手法

• Construct a Cross-Tool Function Call Graph (FCG) to model inter-tool data dependencies in LLM agents.
• Perform static function pair dependency analysis and LLM-validated dependency pruning to define valid call chains.
• Synthesize high-valid, source-to-sink call-chain prompts to deterministically trigger deep-layer tool execution.
• Execute taint-tracking in a runtime environment to monitor data propagation from source to sink.
• Apply a multi-LLM voting committee, guided by GDPR/CCPA/PIPL, to judge data necessity (D_nec) and detect DOE (D_trans outside D_int and D_nec).
• Evaluate DOE detection efficacy and cost savings across 6,675 tools and four agent scenarios.

実験結果

リサーチクエスチョン

• RQ1 What is the prevalence of Data Over-Exposure in cross-tool data flows of LLM agents?
• RQ2 How effective is AgentRaft at detecting DOE compared with baselines across diverse tool ecosystems?
• RQ3 How much does the multi-LLM voting mechanism improve DOE judgment accuracy over single-model judges?
• RQ4 What are the efficiency and cost benefits of automated privacy auditing at scale?

主な発見

• DOE is a systemic risk, with 57.07% of potential tool call paths exposing sensitive data.
• 65.42% of transmitted data fields are identified as over-exposed.
• AgentRaft achieves 69.15% discovery within 50 prompts and ~99% coverage at 150 prompts.
• Multi-LLM voting improves DOE identification by 87.24% within 150 prompts.
• Auditing costs per tool-chain are reduced by 88.6% compared with non-guided baselines.