[Paper Review] From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?

Rodrigo Pedro, Daniel Castro+2 more|arXiv (Cornell University)|Aug 3, 2023

Web Application Security Vulnerabilities17 citations

TL;DR

The paper defines prompt-to-SQL (P2SQL) injections in Langchain-based LLM-integrated web apps, analyzes attack variants across seven LLMs, and proposes four defense extensions with a real-world evaluation.

ABSTRACT

Large Language Models (LLMs) have found widespread applications in various domains, including web applications, where they facilitate human interaction via chatbots with natural language interfaces. Internally, aided by an LLM-integration middleware such as Langchain, user prompts are translated into SQL queries used by the LLM to provide meaningful responses to users. However, unsanitized user prompts can lead to SQL injection attacks, potentially compromising the security of the database. Despite the growing interest in prompt injection vulnerabilities targeting LLMs, the specific risks of generating SQL injection attacks through prompt injections have not been extensively studied. In this paper, we present a comprehensive examination of prompt-to-SQL (P$_2$SQL) injections targeting web applications based on the Langchain framework. Using Langchain as our case study, we characterize P$_2$SQL injections, exploring their variants and impact on application security through multiple concrete examples. Furthermore, we evaluate 7 state-of-the-art LLMs, demonstrating the pervasiveness of P$_2$SQL attacks across language models. Our findings indicate that LLM-integrated applications based on Langchain are highly susceptible to P$_2$SQL injection attacks, warranting the adoption of robust defenses. To counter these attacks, we propose four effective defense techniques that can be integrated as extensions to the Langchain framework. We validate the defenses through an experimental evaluation with a real-world use case application.

Motivation & Objective

• Characterize variants of P2SQL injections targeting Langchain-based web applications.
• Assess how different LLMs affect the feasibility and impact of P2SQL attacks.
• Propose and preliminarily evaluate defenses integrated as Langchain extensions to mitigate P2SQL threats.

Proposed method

• Describe Langchain's SQLDatabaseChain/SQLDatabaseAgent processing to identify injection points.
• Define a threat model and construct seven representative P2SQL attack scenarios (unrestricted and restricted prompting, direct and indirect).
• Experimentally evaluate attacks using GPT-3.5-turbo-0301 and additional models across multiple prompts and configurations.
• Develop four Langchain extensions: database permission hardening, SQL query rewriting, auxiliary LLM validation, and in-prompt data preloading.
• Validate defenses on a real-world use-case web application with PostgreSQL backend.

Experimental results

Research questions

• RQ1 RQ1: What P2SQL injection variants can target Langchain-based web apps and what is their security impact?
• RQ2 RQ2: How does the choice of LLM affect the feasibility and success of P2SQL attacks across Langchain implementations (SQL chain vs. SQL agent)?
• RQ3 RQ3: What defenses can effectively mitigate P2SQL attacks with acceptable performance overhead?

Key findings

• Langchain-based LLM-integrated apps are highly vulnerable to P2SQL injections across tested LLMs.
• Unrestricted default Langchain prompts allow arbitrary SQL queries that can read or write whole databases.
• Prompt restrictions may be bypassed, and indirect attacks via embedded prompts in data payloads can manipulate answers.
• SQL agents enable more complex, multi-step P2SQL attacks than SQL chains.
• Four defenses (database permission hardening, SQL query rewriting, auxiliary LLM validation, in-prompt data preloading) reduce risk with acceptable overhead.
• Evaluation on a real-world use case demonstrates practical effectiveness, though automation and transparency of defenses require further work.