[论文解读] Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal

Rahul Pankajakshan, Sumitra Biswal+2 more|arXiv (Cornell University)|Mar 20, 2024

Big Data and Business Intelligence被引用 7

一句话总结

本文提出基于 OWASP 的大语言模型（LLM）安全风险评估流程，结合情景分析、依赖映射和影响分析，创建面向利益相关者的威胁矩阵，并通过一个大学用例进行演示。

ABSTRACT

The rapid integration of Large Language Models (LLMs) across diverse sectors has marked a transformative era, showcasing remarkable capabilities in text generation and problem-solving tasks. However, this technological advancement is accompanied by significant risks and vulnerabilities. Despite ongoing security enhancements, attackers persistently exploit these weaknesses, casting doubts on the overall trustworthiness of LLMs. Compounding the issue, organisations are deploying LLM-integrated systems without understanding the severity of potential consequences. Existing studies by OWASP and MITRE offer a general overview of threats and vulnerabilities but lack a method for directly and succinctly analysing the risks for security practitioners, developers, and key decision-makers who are working with this novel technology. To address this gap, we propose a risk assessment process using tools like the OWASP risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. Through this analysis, we assess the likelihood of a cyberattack. Subsequently, we conduct a thorough impact analysis to derive a comprehensive threat matrix. We also map threats against three key stakeholder groups: developers engaged in model fine-tuning, application developers utilizing third-party APIs, and end users. The proposed threat matrix provides a holistic evaluation of LLM-related risks, enabling stakeholders to make informed decisions for effective mitigation strategies. Our outlined process serves as an actionable and comprehensive tool for security practitioners, offering insights for resource management and enhancing the overall system security.

研究动机与目标

• 鉴于日益演变的安全威胁，推动在基于 LLM 的系统中进行结构化风险评估的必要性。
• 将 OWASP 风险评级方法学改编并应用于 LLM 特定风险。
• 开发以情景驱动的半定量过程以估计可能性和影响。
• 创建面向利益相关者的威胁矩阵，以指导缓解措施和资源分配。
• 通过一个假设的大学虚拟助理用例演示该框架。

提出的方法

• 利用 OWASP 风险评级方法将风险计算为 Likelihood × Impact。
• 通过定义威胁主体、动机、技能和机会来进行情景分析。
• 将依赖的系统组件映射到脆弱性因素以推导可能性。
• 执行对技术和业务后果的影响分析。
• 将结果汇总为面向三个利益相关者群体的威胁矩阵（LLM 微调、API 集成、最终用户）。
• 提供一个示例用例以展示工作流程和缓解指南。

Figure 1: The system design of the university virtual assistant emphasizes the diverse security measures implemented for each component. An authenticated user’s prompt undergoes optimization and validation/filtering for prompt injection within the prompt optimization module. Contextual information f

实验结果

研究问题

• RQ1 如何将 OWASP 风险评级方法学改编以评估 LLM 特定威胁？
• RQ2 如何将依赖映射和情景分析整合以估计对基于 LLM 的系统的攻击可能性？
• RQ3 什么构成面向利益相关者的 LLM 安全威胁矩阵，及其如何为缓解策略提供指引？
• RQ4 大学用例揭示了在 LLM 部署中的资源分配与安全成熟度方面哪些经验教训？

主要发现

• 可实现的面向 LLM 的结构化威胁矩阵有助于实务人员在各利益相关者群体之间优先考虑缓解措施。
• 所提出的三步风险分析（情景分析、依赖映射、影响分析）生成的半定量风险评级与 OWASP 指引保持一致。
• 该矩阵将传统的网络安全风险与如提示注入和模型操纵等 LLM 特定威胁区分开来。
• 用例展示了风险评级如何在实际部署中指导资源管理和安全改进。