[Paper Review] SWARM-SLR AIssistant: A Unified Framework for Scalable Systematic Literature Review Automation

Tim Wittenborg, Allard Oelen+1 more|arXiv (Cornell University)|Mar 5, 2026

Scientific Computing and Data Management0 citations

TL;DR

Introduces the SWARM-SLR AIssistant, a modular, agent-based framework that unifies SWARM-SLR with an AI-assisted workflow and a centralized tool registry to enable scalable, human-in-the-loop SLR automation.

ABSTRACT

Despite a growing ecosystem of tools supporting Systematic Literature Reviews (SLRs), integrating them into user-friendly workflows remains challenging. The Streamlined Workflow for Automating Machine-Actionable Systematic Literature Reviews (SWARM-SLR) unified the tool annotation and provided a cohesive yet modular workflow, but faced scalability and usability issues. We introduce the SWARM-SLR AIssistant, a unified framework that combines the SWARM-SLR's structured methodology with an agent-based assistant that integrates research tools in a modular interface. The first SWARM-SLR stage is integrated, enabling conversational, LLM-guided support and persistent data storage. To address the tool assessment bottleneck, we propose a centralized tool registry that allows developers to annotate and register tools autonomously using a shared metadata schema. Preliminary evaluation shows improved usability, but challenges remain in balancing efficiency, accessibility, and transparency. Further development is needed to realize scalable SLR automation.

Motivation & Objective

• Improve usability and accessibility of SWARM-SLR by integrating it into an AI-assisted workspace with persistent storage.
• Propose a centralized, decentralized-friendly tool registry to annotate and register tools using a shared metadata schema.
• Integrate the first stage of SWARM-SLR into an AIssistant to provide conversational, LLM-guided guidance.
• Evaluate the preliminary implementation with real users to identify usability benefits and remaining challenges.

Proposed method

• Develop a modular AIssistant workspace that can call external configurable tools via standardized tool descriptions.
• Embed the first five SWARM-SLR steps as separate agents within the AIssistant interface.
• Design a centralized tool registry that maps SWARM-SLR requirements to metadata for tool annotation.
• Provide a persistent data layer to share intermediate and final SLR results between humans and machines.
• Use a crowdsourced, extension-market style approach inspired by bio.tools to enable tool onboarding and autonomous annotation by developers.

Figure 1: Overview of the SWARM-SLR AIssistant. The SWARM-SLR requirements are mapped to tool registry properties, which are used to annotate tools. The resulting tool catalogue can be used from within the AIssistant, providing a uniform interface for human and machine to access tools and data.

Experimental results

Research questions

• RQ1 Can the AIssistant provide effective LLM-guided support for the first stage of SWARM-SLR while maintaining a persistent shared data layer?
• RQ2 Does a centralized tool registry improve usability and scalability of tool annotation and integration in SWARM-SLR AIssistant?
• RQ3 What are the usability, transparency, and resource trade-offs when integrating diverse research tools into a unified AI-assisted SLR workflow?
• RQ4 How usable is the integrated system for researchers in real-world SLR contexts, and what challenges remain for broader adoption?

Key findings

• Preliminary evaluation with 18 participants showed improved usability of the AIssistant compared to the original Jupyter Notebook implementation.
• Participants reported positive sentiment toward the AIssistant's usability and accessibility improvements.
• Users highlighted remaining concerns about tool transparency, hallucinations, and resource efficiency.
• The integration provides a uniform interface for human and machine access to tools and data, enabling shared access to intermediate and final results.
• A centralized tool registry is proposed to simplify tool annotation and lower the barrier to contributing tools.

Figure 2: The conceptual design of the AIssistant tool registry provides a central tool curation point, mapping requirements to metadata schema. Users can browse task-specific tools similar to an extension marketplace.