[Paper Review] Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal

Rahul Pankajakshan, Sumitra Biswal+2 more|arXiv (Cornell University)|Mar 20, 2024

Big Data and Business Intelligence7 citations

TL;DR

The paper proposes an OWASP-based risk assessment process for LLM security, combining scenario analysis, dependency mapping, and impact analysis to create a stakeholder-focused threat matrix, demonstrated via a university use case.

ABSTRACT

The rapid integration of Large Language Models (LLMs) across diverse sectors has marked a transformative era, showcasing remarkable capabilities in text generation and problem-solving tasks. However, this technological advancement is accompanied by significant risks and vulnerabilities. Despite ongoing security enhancements, attackers persistently exploit these weaknesses, casting doubts on the overall trustworthiness of LLMs. Compounding the issue, organisations are deploying LLM-integrated systems without understanding the severity of potential consequences. Existing studies by OWASP and MITRE offer a general overview of threats and vulnerabilities but lack a method for directly and succinctly analysing the risks for security practitioners, developers, and key decision-makers who are working with this novel technology. To address this gap, we propose a risk assessment process using tools like the OWASP risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. Through this analysis, we assess the likelihood of a cyberattack. Subsequently, we conduct a thorough impact analysis to derive a comprehensive threat matrix. We also map threats against three key stakeholder groups: developers engaged in model fine-tuning, application developers utilizing third-party APIs, and end users. The proposed threat matrix provides a holistic evaluation of LLM-related risks, enabling stakeholders to make informed decisions for effective mitigation strategies. Our outlined process serves as an actionable and comprehensive tool for security practitioners, offering insights for resource management and enhancing the overall system security.

Motivation & Objective

• Motivate the need for structured risk assessment in LLM-based systems due to evolving security threats.
• Adapt and apply the OWASP Risk Rating Methodology to LLM-specific risks.
• Develop a scenario-driven, semi-quantitative process to estimate likelihood and impact.
• Create a stakeholder-focused threat matrix to guide mitigation and resource allocation.
• Demonstrate the framework with a hypothetical university virtual assistant use case.

Proposed method

• Utilize OWASP risk rating methodology to compute risk as Likelihood × Impact.
• Conduct scenario analysis by defining threat agents, motives, skills, and opportunities.
• Map dependent system components against vulnerability factors to derive likelihood.
• Perform impact analysis covering technical and business consequences.
• Aggregate results into a threat matrix tailored to three stakeholder groups (LLM fine-tuning, API integration, end users).
• Provide an illustrative use case to showcase workflow and mitigation guidance.

Figure 1: The system design of the university virtual assistant emphasizes the diverse security measures implemented for each component. An authenticated user’s prompt undergoes optimization and validation/filtering for prompt injection within the prompt optimization module. Contextual information f

Experimental results

Research questions

• RQ1 How can the OWASP risk rating methodology be adapted to assess LLM-specific threats?
• RQ2 How can dependency mapping and scenario analysis be integrated to estimate likelihood of attacks on LLM-based systems?
• RQ3 What constitutes a stakeholder-focused threat matrix for LLM security, and how can it inform mitigation strategies?
• RQ4 What lessons does a university use-case reveal about resource allocation and security maturity in LLM deployments?

Key findings

• A structured threat matrix for LLMs is feasible and aids practitioners in prioritizing mitigations across stakeholder groups.
• The proposed three-step risk analysis (scenario analysis, dependency mapping, impact analysis) yields a semi-quantitative risk rating aligned with OWASP guidance.
• The matrix distinguishes traditional cybersecurity risks from LLM-specific threats like prompt injection and model manipulation.
• The use case demonstrates how risk ratings can guide resource management and security improvements in practical deployments.