[论文解读] Devising and Detecting Phishing: Large Language Models vs. Smaller Human Models

Fredrik Heiding, Bruce Schneier+3 more|arXiv (Cornell University)|Aug 23, 2023

Spam and Phishing Detection被引用 14

一句话总结

本文比较 GPT-4 生成的钓鱼邮件质量与人工 V-Triad 模型，并评估四种 LLM 在检测钓鱼意图方面的能力，以及对 AI 支持的钓鱼进行经济分析。

ABSTRACT

AI programs, built using large language models, make it possible to automatically create phishing emails based on a few data points about a user. They stand in contrast to traditional phishing emails that hackers manually design using general rules gleaned from experience. The V-Triad is an advanced set of rules for manually designing phishing emails to exploit our cognitive heuristics and biases. In this study, we compare the performance of phishing emails created automatically by GPT-4 and manually using the V-Triad. We also combine GPT-4 with the V-Triad to assess their combined potential. A fourth group, exposed to generic phishing emails, was our control group. We utilized a factorial approach, sending emails to 112 randomly selected participants recruited for the study. The control group emails received a click-through rate between 19-28%, the GPT-generated emails 30-44%, emails generated by the V-Triad 69-79%, and emails generated by GPT and the V-Triad 43-81%. Each participant was asked to explain why they pressed or did not press a link in the email. These answers often contradict each other, highlighting the need for personalized content. The cues that make one person avoid phishing emails make another person fall for them. Next, we used four popular large language models (GPT, Claude, PaLM, and LLaMA) to detect the intention of phishing emails and compare the results to human detection. The language models demonstrated a strong ability to detect malicious intent, even in non-obvious phishing emails. They sometimes surpassed human detection, although often being slightly less accurate than humans. Finally, we make an analysis of the economic aspects of AI-enabled phishing attacks, showing how large language models can increase the incentives of phishing and spear phishing by reducing their costs.

研究动机与目标

• 评估自动生成的钓鱼邮件（GPT-4）在效果上与人工制作的邮件（V-Triad）相比的表现。
• 评估将 GPT-4 与 V-Triad 结合是否能提升钓鱼质量与速度。
• 在检测钓鱼意图并提供降低风险的指导方面测试多种 LLM（GPT、Claude、PaLM、LLaMA）。
• 研究 AI 支持的钓鱼攻击的经济含义。

提出的方法

• 招募 112 名参与者并收集个性化背景数据以定制钓鱼邮件。
• 使用四种方法创建钓鱼邮件：对照组（任意）、GPT-4、V-Triad，以及 GPT-4 + V-Triad。
• 分批发送邮件并测量点击率；在研究结束后对参与者进行事后访谈/简报。
• 询问参与者为何点击或未点击链接，以分析易感性。
• 使用四种 LLM（GPT、Claude、Bard、LLaMA）检测钓鱼意图，并与人类检测进行比较。
• 从攻击者角度对传统、鱼叉式和 AI 增强钓鱼进行经济成本-收益分析。

实验结果

研究问题

• RQ1 GPT-4 生成的钓鱼邮件在点击率方面与 V-Triad 生成的邮件相比如何？
• RQ2 将 GPT-4 与 V-Triad 结合是否能提升钓鱼效果或缩短创建时间？
• RQ3 LLMs 是否能比人类更准确地检测钓鱼意图，以及在不同邮件类别中的表现如何？
• RQ4 AI 驱动钓鱼相对于传统方法的经济激励与成本有哪些？

主要发现

• 对照组邮件的点击率为 19-28%。
• GPT-4 生成的邮件的点击率为 30-44%。
• V-Triad 生成的邮件的点击率为 69-79%。
• GPT+V-Triad 邮件的点击率为 43-81%。
• Claude 对对照邮件的恶意意图检测为 75%，GPT 为 25%，GPT+V-Triad 为 25%（在引导怀疑之前；引导后，Claude 检测为 75%（对照）、100%（GPT）、100%（V-Triad）、100%（GPT+V-Triad））。
• LLMs 表现出强大的检测恶意意图的能力，有时甚至超过人类，但结果随提示和模型而异。