[Paper Review] The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

Miles Brundage, Shahar Avin+24 more|arXiv (Cornell University)|Feb 20, 2018

Network Security and Intrusion DetectionComputer Science489 citations

TL;DR

This report surveys potential security threats from malicious uses of AI and proposes forecasting, prevention, and mitigation strategies, plus areas for further research.

ABSTRACT

This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in which AI may influence the threat landscape in the digital, physical, and political domains, we make four high-level recommendations for AI researchers and other stakeholders. We also suggest several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. Finally, we discuss, but do not conclusively resolve, the long-term equilibrium of attackers and defenders.

Motivation & Objective

• Assess how AI can influence threats in digital, physical, and political domains.
• Develop forecasts of malicious AI capabilities and attacker models.
• Propose practical recommendations for researchers and stakeholders to prevent and mitigate harms.
• Identify promising research directions to broaden defenses and hinder attacks.

Proposed method

• Survey existing and potential threat landscapes across digital, physical, and political domains.
• Forecast threat trajectories and attacker-defender dynamics.
• Recommend high-level actions for researchers and stakeholders to reduce risk.
• Discuss long-term equilibrium between attackers and defenders without conclusive resolution.

Experimental results

Research questions

• RQ1 How might AI influence security threats across digital, physical, and political contexts?
• RQ2 What forecast-based and proactive measures can improve prevention and mitigation of malicious AI use?
• RQ3 What recommendations can strengthen defenses and reduce attacker effectiveness?
• RQ4 What research agendas could expand defense capabilities against AI-enabled threats?

Key findings

• The paper outlines a landscape of potential AI-enabled threats across multiple domains.
• It proposes forecasting, prevention, and mitigation as core strategies for addressing malicious AI use.
• The report offers high-level recommendations for researchers and other stakeholders.
• It suggests several future research areas to expand defenses or reduce attack effectiveness.
• The long-term attacker–defender dynamics are discussed but not conclusively resolved.