[Paper Review] GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models

Emilio Ferrara|arXiv (Cornell University)|Oct 1, 2023

Artificial Intelligence in Healthcare and Education19 citations

TL;DR

The paper surveys nefarious uses of GenAI and LLMs, presents a harm-intent taxonomy, catalogs proof-of-concept misuse scenarios, and proposes mitigation and regulatory considerations.

ABSTRACT

Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identities orchestrate malicious campaigns, and where targeted misinformation or scams are crafted with unparalleled precision. Welcome to the darker side of GenAI applications. This article is not just a journey through the meanders of potential misuse of GenAI and LLMs, but also a call to recognize the urgency of the challenges ahead. As we navigate the seas of misinformation campaigns, malicious content generation, and the eerie creation of sophisticated malware, we'll uncover the societal implications that ripple through the GenAI revolution we are witnessing. From AI-powered botnets on social media platforms to the unnerving potential of AI to generate fabricated identities, or alibis made of synthetic realities, the stakes have never been higher. The lines between the virtual and the real worlds are blurring, and the consequences of potential GenAI's nefarious applications impact us all. This article serves both as a synthesis of rigorous research presented on the risks of GenAI and misuse of LLMs and as a thought-provoking vision of the different types of harmful GenAI applications we might encounter in the near future, and some ways we can prepare for them.

Motivation & Objective

• Motivate and document the risks of Generative AI and Large Language Models for cybersecurity, ethics, and societal structures.
• Propose a taxonomy mapping harms to malicious intents (deception, propaganda, dishonesty).
• Survey proof-of-concept misuse scenarios and discuss real-world implications for policy and practice.
• Highlight regulatory contexts (EU and China) and outline mitigation strategies and ongoing monitoring needs.

Proposed method

• Define Generative AI and Large Language Models and explain their generation mechanisms.
• Introduce a 3x4 harm-intent taxonomy linking harms (Harm to Person, Financial/Economic Damage, Information Manipulation, Societal/Infrastructural Damage) with intents (Deception, Propaganda, Dishonesty).
• Present proof-of-concept scenarios (Tables 2–3–5) and illustrative examples of misuse to ground the taxonomy.
• Discuss regulatory landscapes and ethical guidance, and propose mitigation and risk-benefit analysis approaches.

Figure 1. Charting the Landscape of Nefarious Applications of Generative Artificial Intelligence and Large Language Models

Experimental results

Research questions

• RQ1 What nefarious applications of GenAI and LLMs are evident or plausible?
• RQ2 How can harms and attacker intents be systematically categorized to anticipate misuse?
• RQ3 What concrete proof-of-concept scenarios demonstrate GenAI misuse, and what are their implications?
• RQ4 What mitigation, governance, and regulatory approaches are recommended to counter GenAI abuse?

Key findings

• Introduces a harm-intent taxonomy (a 3x4 matrix) linking types of harm to malicious intents.
• Summarizes proof-of-concept misuse scenarios across impersonation, misinformation, deception, and propaganda (Tables 2–3–5).
• Documents real-world exemplars and risks (e.g., AI-generated voice impersonation, synthetic identities) and discusses their implications.
• Discusses regulatory perspectives in the EU and China and emphasizes the need for continuous monitoring and ethical guidelines.
• Offers recommendations for technical and socio-technical mitigation strategies and the importance of risk-benefit analyses.

Figure 2. Mind Map of Abuse and Malicious Applications of GenAI and Large Language Models.