[Paper Review] Agentic Retrieval-Augmented Generation for Time Series Analysis

Chidaksh Ravuru, Sagar Srinivas Sakhinana+1 more|arXiv (Cornell University)|Aug 18, 2024

Time Series Analysis and Forecasting4 citations

TL;DR

This paper proposes an agentic Retrieval-Augmented Generation (RAG) framework for time series analysis that uses a hierarchical multi-agent architecture to improve forecasting, anomaly detection, and classification. By leveraging specialized sub-agents with fine-tuned small language models (SLMs) and a shared prompt pool of distilled historical patterns, the framework achieves state-of-the-art performance across multiple benchmarks, outperforming task-specific models and ablated variants.

ABSTRACT

Time series modeling is crucial for many applications, however, it faces challenges such as complex spatio-temporal dependencies and distribution shifts in learning from historical context to predict task-specific outcomes. To address these challenges, we propose a novel approach using an agentic Retrieval-Augmented Generation (RAG) framework for time series analysis. The framework leverages a hierarchical, multi-agent architecture where the master agent orchestrates specialized sub-agents and delegates the end-user request to the relevant sub-agent. The sub-agents utilize smaller, pre-trained language models (SLMs) customized for specific time series tasks through fine-tuning using instruction tuning and direct preference optimization, and retrieve relevant prompts from a shared repository of prompt pools containing distilled knowledge about historical patterns and trends to improve predictions on new data. Our proposed modular, multi-agent RAG approach offers flexibility and achieves state-of-the-art performance across major time series tasks by tackling complex challenges more effectively than task-specific customized methods across benchmark datasets.

Motivation & Objective

• Address the limitations of fixed-window time series models that fail to capture long-range dependencies and complex trends.
• Overcome the lack of inherent time series pattern recognition in small language models (SLMs) trained on text.
• Improve generalization and reasoning in time series tasks by integrating external knowledge through retrieval-augmented generation.
• Develop a modular, flexible framework that outperforms task-specific models across diverse time series applications.
• Enable effective adaptation of SLMs to time series tasks through instruction tuning, direct preference optimization (DPO), and retrieval from distilled prompt pools.

Proposed method

• Employ a hierarchical, multi-agent architecture with a master agent that routes user requests to specialized sub-agents based on task type.
• Use fine-tuned small language models (SLMs) such as Llama 3-8B, optimized via instruction tuning and direct preference optimization (DPO) for specific time series tasks.
• Implement a shared, task-agnostic prompt pool containing distilled knowledge of historical time series patterns and trends for retrieval.
• Enable agentic reasoning through iterative retrieval and refinement cycles, allowing sub-agents to dynamically access relevant historical patterns.
• Integrate retrieval-augmented generation (RAG) principles into time series modeling by retrieving task-specific prompts from a knowledge base of distilled patterns.
• Apply self-extension techniques to enhance reasoning and generalization by retrieving and combining multiple relevant prompt templates during inference.

Experimental results

Research questions

• RQ1 Can a modular, agentic RAG framework improve performance across diverse time series tasks compared to task-specific models?
• RQ2 How does retrieval of distilled historical patterns from a shared prompt pool affect prediction accuracy in time series forecasting and anomaly detection?
• RQ3 To what extent do components like DPO, SAS (self-attention scaling), and instruction tuning contribute to the framework’s performance?
• RQ4 Does the agentic, multi-step reasoning capability of the framework mitigate distribution shifts and long-range dependency issues in time series data?
• RQ5 Can small language models (SLMs) achieve state-of-the-art results in time series analysis when augmented with retrieval and agent-based orchestration?

Key findings

• The SelfExtend-Agentic-RAG framework with Llama 3-8B achieved 95.82% accuracy, 95.02% precision, and 95.24% recall on the METR-LA classification benchmark, significantly outperforming the baseline without DPO (88.67% accuracy).
• On the PEMS-BAY dataset, the framework achieved 96.01% accuracy, 95.01% precision, and 95.53% recall in classification, surpassing the baseline without DPO (91.01% accuracy).
• For anomaly detection, the framework achieved 99.47% precision, 98.15% recall, and 92.59% F1 on SWaT, compared to 95.49% precision and 87.04% F1 in the baseline without DPO.
• In 12-sequence-to-12-sequence forecasting, the framework achieved RMSE of 4.03, MAE of 2.02, and MAPE of 5.05 on METR-LA at horizon 3, outperforming the baseline without DPO (RMSE: 4.19, MAE: 2.12, MAPE: 5.36).
• Ablation studies confirmed that all components—DPO, SAS, IT (instruction tuning), and prompt retrieval—significantly contribute to performance, with the full framework consistently outperforming ablated variants across all tasks and datasets.
• The framework demonstrated superior robustness to distribution shifts and improved generalization across diverse time series tasks, including forecasting, anomaly detection, and classification, without task-specific architectural redesign.