[论文解读] Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility

Wentao Ye, Mingfeng Ou+9 more|arXiv (Cornell University)|May 15, 2023

Topic Modeling被引用 13

一句话总结

本论文提出一个自动化工作流，用于研究 LLM 的鲁棒性、的一致性和可信度，涵盖跨多个模型的超过一百万次查询，揭示脆弱性并提出相对训练指数（RTI）以衡量用于 LLM 评估的数据集可信度。

ABSTRACT

The recent popularity of large language models (LLMs) has brought a significant impact to boundless fields, particularly through their open-ended ecosystem such as the APIs, open-sourced models, and plugins. However, with their widespread deployment, there is a general lack of research that thoroughly discusses and analyzes the potential risks concealed. In that case, we intend to conduct a preliminary but pioneering study covering the robustness, consistency, and credibility of LLMs systems. With most of the related literature in the era of LLM uncharted, we propose an automated workflow that copes with an upscaled number of queries/responses. Overall, we conduct over a million queries to the mainstream LLMs including ChatGPT, LLaMA, and OPT. Core to our workflow consists of a data primitive, followed by an automated interpreter that evaluates these LLMs under different adversarial metrical systems. As a result, we draw several, and perhaps unfortunate, conclusions that are quite uncommon from this trendy community. Briefly, they are: (i)-the minor but inevitable error occurrence in the user-generated query input may, by chance, cause the LLM to respond unexpectedly; (ii)-LLMs possess poor consistency when processing semantically similar query input. In addition, as a side finding, we find that ChatGPT is still capable to yield the correct answer even when the input is polluted at an extreme level. While this phenomenon demonstrates the powerful memorization of the LLMs, it raises serious concerns about using such data for LLM-involved evaluation in academic development. To deal with it, we propose a novel index associated with a dataset that roughly decides the feasibility of using such data for LLM-involved evaluation. Extensive empirical studies are tagged to support the aforementioned claims.

研究动机与目标

• 推动对 LLM 风险的系统性评估，超越传统的 NLP 指标。
• 提出一个自动化工作流，以扩展对 LLM 的鲁棒性、一致性和可信度评估。
• 引入统一的数据原语和自动解释器，以处理大规模查询响应。
• 开发针对现实世界 LLM 使用情境的威胁模型和攻击方案。
• 引入 RTI 作为数据集可信度指数，以指导 LLM 评估的数据集选择。

提出的方法

• 使用 gpt-3.5-turbo API，以及开源的 LLaMA 和 OPT 模型作为基础。
• 形成一个通用数据原语：(prompt, p, q, o, a) 以带有多个混乱选项地构造问答数据。
• 通过词、字符、视觉扰动实现自动化攻击，以模拟现实输入错误。
• 定义鲁棒性和一致性威胁模型以及五种与 LLM 使用情景相符的攻击方案。
• 通过逐步扰动输入来计算 RTI，以确定记忆效应和数据集可靠性。
• 在项目 URL 提供开源数据集和样本。

实验结果

研究问题

• RQ1 前沿 LLM 对对抗性结构化输入和常见用户错误的鲁棒性如何？
• RQ2 当语义相似的输入被重新表述时，LLM 的回答的一致性如何？
• RQ3 我们能否使用一个以记忆为驱动的指数（RTI）来量化基于 LLM 的评估的数据集可信度？
• RQ4 鲁棒性、一致性和记忆性对学术评估 LLMs 有哪些实际影响？

主要发现

• 轻微的输入扰动可能导致 LLM 出现意外回应。
• 在处理语义上相似的查询时，LLMs 的一致性较差。
• 即使输入被严重污染，ChatGPT 仍然可能给出正确答案，表明记忆作用。
• RTI 提供了数据集记忆程度及其用于 LLM 评估的相对适用性的度量。
• 该研究强调在 LLM 相关评估中谨慎使用污染或记忆化的数据集，并提供开源资源。