[论文解读] GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models

Emilio Ferrara|arXiv (Cornell University)|Oct 1, 2023

Artificial Intelligence in Healthcare and Education被引用 19

一句话总结

本文综述GenAI与大语言模型的恶意用途，提出一个伤害-意图分类法，编目概念验证的误用情景，并提出缓解与监管考量。

ABSTRACT

Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identities orchestrate malicious campaigns, and where targeted misinformation or scams are crafted with unparalleled precision. Welcome to the darker side of GenAI applications. This article is not just a journey through the meanders of potential misuse of GenAI and LLMs, but also a call to recognize the urgency of the challenges ahead. As we navigate the seas of misinformation campaigns, malicious content generation, and the eerie creation of sophisticated malware, we'll uncover the societal implications that ripple through the GenAI revolution we are witnessing. From AI-powered botnets on social media platforms to the unnerving potential of AI to generate fabricated identities, or alibis made of synthetic realities, the stakes have never been higher. The lines between the virtual and the real worlds are blurring, and the consequences of potential GenAI's nefarious applications impact us all. This article serves both as a synthesis of rigorous research presented on the risks of GenAI and misuse of LLMs and as a thought-provoking vision of the different types of harmful GenAI applications we might encounter in the near future, and some ways we can prepare for them.

研究动机与目标

• 激发并记录生成式AI和大语言模型在网络安全、伦理与社会结构方面的风险。
• 提出一个将伤害映射到恶意意图的分类法（欺骗、宣传、不诚实）。
• 调查概念验证的滥用情景并讨论对政策与实践的现实世界影响。
• 强调监管背景（欧盟与中国）并概述缓解策略与持续监测需求。

提出的方法

• 界定生成式AI和大语言模型并解释其生成机制。
• 引入一个3x4的伤害-意图分类法，将伤害（对个人的伤害、金融/经济损害、信息操控、社会/基础设施损害）与意图（欺骗、宣传、不诚实）联系起来。
• 呈现实验概念场景（表2–表3–表5）以及滥用的示例以支撑该分类法。
• 讨论监管环境与伦理指引，并提出缓解措施与风险-收益分析方法。

Figure 1. Charting the Landscape of Nefarious Applications of Generative Artificial Intelligence and Large Language Models

实验结果

研究问题

• RQ1 GenAI和LLMs有哪些明显或可能的恶意应用？
• RQ2 如何系统地对伤害和攻击者意图进行分类以预测滥用？
• RQ3 哪些具体的概念验证场景能够证明GenAI的滥用及其含义？
• RQ4 应对GenAI滥用的缓解、治理和监管方法有哪些建议？

主要发现

• 引入一个伤害-意图分类法（3x4矩阵），将伤害类型与恶意意图联系起来。
• 总结了横跨冒充、错误信息、欺骗与宣传的概念验证滥用情景（表2–表3–表5）。
• 记录现实世界的示例和风险（如AI生成的声音冒充、合成身份）并讨论其含义。
• 讨论欧盟和中国的监管视角，并强调持续监测与伦理准则的必要性。
• 提出技术与社会技术缓解策略的建议，以及风险-收益分析的重要性。

Figure 2. Mind Map of Abuse and Malicious Applications of GenAI and Large Language Models.