[논문 리뷰] Scamming the Scammers: Using ChatGPT to Reply Mails for Wasting Time and Resources

Enrico Cambiaso, Luca Caviglione|arXiv (Cornell University)|2023. 02. 10.

Spam and Phishing Detection인용 수 15

한 줄 요약

The paper explores using ChatGPT to generate replies to scam emails to waste scammers’ time and resources, presenting a preliminary testbed and results showing feasibility of engagement over multiple threads and days.

ABSTRACT

The use of Artificial Intelligence (AI) to support cybersecurity operations is now a consolidated practice, e.g., to detect malicious code or configure traffic filtering policies. The recent surge of AI, generative techniques and frameworks with efficient natural language processing capabilities dramatically magnifies the number of possible applications aimed at increasing the security of the Internet. Specifically, the ability of ChatGPT to produce textual contents while mimicking realistic human interactions can be used to mitigate the plague of emails containing scams. Therefore, this paper investigates the use of AI to engage scammers in automatized and pointless communications, with the goal of wasting both their time and resources. Preliminary results showcase that ChatGPT is able to decoy scammers, thus confirming that AI is an effective tool to counteract threats delivered via mail. In addition, we highlight the multitude of implications and open research questions to be addressed in the perspective of the ubiquitous adoption of AI.

연구 동기 및 목표

• Motivate the use of AI to mitigate mail-based scams and reduce resource waste.
• Evaluate the feasibility of ChatGPT-generated replies to engage scammers in unproductive conversations.
• Provide a preliminary quantitative assessment of engagement duration and mail volume in scam interactions.
• Identify open research questions and ethical considerations for AI-assisted scam countermeasures.

제안 방법

• Set up a realistic mail account domain and observed 30 days of incoming messages, spanning 60 days total for the test.
• Use Office365 anti-spam filters to identify potential scammers and manually verify them for the test set.
• Feed the full scammer message text to ChatGPT to generate a reply, treating ChatGPT as a black-box generator.
• Add a preamble in prompts to elicit helpful replies while avoiding disclosure of personal data, and mimic human response delays to extend conversations.
• Reply to scammers by quoting prior messages and sending ChatGPT-generated content, without actually sending from ChatGPT (AI used only to generate text).
• Analyze thread length, message counts, and time spans to assess engagement with scammers.

실험 결과

연구 질문

• RQ1 Can ChatGPT generate convincing, contextually appropriate replies to scam emails?
• RQ2 To what extent can AI-generated replies sustain scammer engagement over time (in terms of message count and duration)?
• RQ3 What practical limitations and operational considerations arise when using AI to interact with scammers in production-like settings?
• RQ4 What ethical, privacy, and security questions emerge from AI-enabled “scam the scammer” approaches?

주요 결과

• ChatGPT can generate replies that sustain conversations with scammers for multiple messages and days.
• Some scam threads persisted up to 18 AI-generated mails and about 27 days of interaction.
• A portion of scammers stopped replying due to external factors or after initial exchanges, while others continued, including cases where scammers requested money or personal data.
• SMTP errors and attacker behavior influenced thread termination, indicating limitations in real-world email delivery.
• The study highlights several engineering, privacy, and ethical questions that must be addressed for AI-based scam countermeasures.
• Results are preliminary but demonstrate the feasibility of using generative AI to decoy scammers and waste attacker resources.