[Paper Review] Regulating AI: Applying insights from behavioural economics and psychology to the application of article 5 of the EU AI Act

Huixin Zhong, O'Neill, Eamonn+1 more|arXiv (Cornell University)|Jul 24, 2023

Digital Transformation in LawEconomics, Econometrics and Finance3 citations

TL;DR

This paper applies behavioural economics and cognitive psychology to clarify and strengthen Article 5 of the EU AI Act, defining 'subliminal', 'manipulative', and 'deceptive' techniques, identifying 3 subliminal and 5 manipulative techniques AI may use, and proposing targeted revisions to enhance legal protection against AI-driven user manipulation.

ABSTRACT

Article 5 of the European Union's Artificial Intelligence Act is intended to regulate AI use to prevent potentially harmful consequences. Nevertheless, applying this legislation practically is likely to be challenging because of ambiguously used terminologies and because it fails to specify which manipulation techniques may be invoked by AI, potentially leading to significant harm. This paper aims to bridge this gap by defining key terms and demonstrating how AI may invoke these techniques, drawing from insights in psychology and behavioural economics. First, this paper provides definitions of the terms "subliminal techniques", "manipulative techniques" and "deceptive techniques". Secondly, we identified from the literature in cognitive psychology and behavioural economics three subliminal and five manipulative techniques and exemplify how AI might implement these techniques to manipulate users in real-world case scenarios. These illustrations may serve as a practical guide for stakeholders to detect cases of AI manipulation and consequently devise preventive measures. Article 5 has also been criticised for offering inadequate protection. We critically assess the protection offered by Article 5, proposing specific revisions to paragraph 1, points (a) and (b) of Article 5 to increase its protective effectiveness.

Motivation & Objective

• To address the ambiguity in Article 5 of the EU AI Act regarding terms like 'subliminal', 'manipulative', and 'deceptive' techniques.
• To identify and define specific psychological and behavioural techniques that AI systems may exploit to influence users.
• To provide practical, real-world case scenarios illustrating how AI could deploy these techniques in ways that may cause harm.
• To critically assess the current protective scope of Article 5 and propose concrete revisions to enhance its effectiveness.
• To support regulators, developers, and auditors in detecting and preventing AI manipulation through evidence-based criteria.

Proposed method

• Defined 'subliminal techniques', 'manipulative techniques', and 'deceptive techniques' using insights from cognitive psychology and behavioural economics.
• Searched academic literature to identify and categorize three subliminal and five manipulative techniques with empirical grounding.
• Mapped each identified technique to plausible AI implementation scenarios in real-world applications (e.g., recommendation systems, chatbots).
• Analyzed the current wording of Article 5, paragraph 1, points (a) and (b), to assess gaps in legal protection.
• Proposed specific textual revisions to Article 5 to improve clarity, scope, and enforceability based on psychological mechanisms.
• Used case-based reasoning to demonstrate how defined techniques could lead to user harm, supporting regulatory design.

Experimental results

Research questions

• RQ1 Which psychological and behavioural mechanisms constitute subliminal, manipulative, and deceptive techniques in AI systems?
• RQ2 How can AI systems practically implement these techniques in real-world applications to influence user decisions?
• RQ3 What are the key shortcomings in the current wording of Article 5 of the EU AI Act regarding protection against such techniques?
• RQ4 How can the legal language of Article 5 be revised to better capture and regulate these AI-driven influence mechanisms?
• RQ5 To what extent do the identified techniques pose a risk of causing significant harm to users under current regulatory frameworks?

Key findings

• Three subliminal techniques—priming, sensory overload, and temporal framing—were identified as having psychological foundations and potential for AI deployment.
• Five manipulative techniques—default bias, scarcity, social proof, loss aversion, and choice architecture—were found to be widely supported by behavioural economics and applicable to AI systems.
• Case scenarios demonstrated that AI systems can exploit these techniques in areas such as digital advertising, financial advice, and content recommendation to subtly influence user choices.
• The current wording of Article 5, particularly points (a) and (b) of paragraph 1, fails to clearly define or prohibit key manipulation mechanisms, resulting in inadequate legal protection.
• The paper proposes specific textual revisions to Article 5 to enhance clarity and scope, particularly by explicitly referencing psychological mechanisms like loss aversion and default bias.
• The integration of behavioural science insights into regulatory language can significantly improve the detectability and enforceability of AI manipulation rules.