[Paper Review] Cybersecurity of AI medical devices: risks, legislation, and challenges

Elisabetta Biasin, Erik Kamenjašević+1 more|arXiv (Cornell University)|Mar 6, 2023

Artificial Intelligence in Healthcare and Education4 citations

TL;DR

This paper examines cybersecurity risks in AI-powered medical devices, analyzing threats like data poisoning and code extraction, and evaluates the EU's regulatory framework—including MDR, NIS 2, GDPR, and the AI Act—highlighting integration challenges and compliance ambiguities in incident reporting and critical infrastructure definitions.

ABSTRACT

Medical devices and artificial intelligence systems rapidly transform healthcare provisions. At the same time, due to their nature, AI in or as medical devices might get exposed to cyberattacks, leading to patient safety and security risks. This book chapter is divided into three parts. The first part starts by setting the scene where we explain the role of cybersecurity in healthcare. Then, we briefly define what we refer to when we talk about AI that is considered a medical device by itself or supports one. To illustrate the risks such medical devices pose, we provide three examples: the poisoning of datasets, social engineering, and data or source code extraction. In the second part, the paper provides an overview of the European Union's regulatory framework relevant for ensuring the cybersecurity of AI as or in medical devices (MDR, NIS Directive, Cybersecurity Act, GDPR, the AI Act proposal and the NIS 2 Directive proposal). Finally, the third part of the paper examines possible challenges stemming from the EU regulatory framework. In particular, we look toward the challenges deriving from the two legislative proposals and their interaction with the existing legislation concerning AI medical devices' cybersecurity. They are structured as answers to the following questions: (1) how will the AI Act interact with the MDR regarding the cybersecurity and safety requirements?; (2) how should we interpret incident notification requirements from the NIS 2 Directive proposal and MDR?; and (3) what are the consequences of the evolving term of critical infrastructures? [This is a draft chapter. The final version will be available in Research Handbook on Health, AI and the Law edited by Barry Solaiman & I. Glenn Cohen, forthcoming 2023, Edward Elgar Publishing Ltd]

Motivation & Objective

• To identify and analyze cybersecurity risks specific to AI as or within medical devices, including data poisoning, social engineering, and source code theft.
• To map and evaluate the existing and proposed EU regulatory frameworks governing AI medical device security, including MDR, NIS Directive, Cybersecurity Act, GDPR, and the AI Act.
• To assess the interoperability and potential conflicts between the AI Act and MDR regarding safety and cybersecurity requirements.
• To clarify the interpretation and implementation of incident notification obligations under the NIS 2 Directive proposal and MDR.
• To explore the implications of the evolving definition of critical infrastructure on AI medical device regulation.

Proposed method

• Systematic analysis of cybersecurity threats targeting AI medical devices using three illustrative case studies: dataset poisoning, social engineering, and source code/data exfiltration.
• Comprehensive review and comparison of relevant EU legislation, including MDR, NIS Directive, Cybersecurity Act, GDPR, and proposed AI Act and NIS 2 Directive.
• Legal and regulatory analysis to assess the interaction between the AI Act and MDR, particularly on safety and cybersecurity requirements.
• Interpretive analysis of incident notification obligations under the NIS 2 Directive proposal and MDR, focusing on alignment and potential overlaps.
• Examination of the evolving scope of 'critical infrastructure' under EU law and its implications for AI medical devices.
• Synthesis of regulatory challenges through structured responses to three core questions on regulatory interaction, notification, and infrastructure definitions.

Experimental results

Research questions

• RQ1 How will the AI Act interact with the MDR in defining and enforcing cybersecurity and safety requirements for AI medical devices?
• RQ2 How should incident notification requirements under the NIS 2 Directive proposal and MDR be interpreted and implemented in practice?
• RQ3 What are the consequences of the evolving definition of critical infrastructure under EU law for the regulation of AI medical devices?
• RQ4 What regulatory gaps or overlaps exist between the proposed AI Act and existing legislation such as MDR and GDPR?
• RQ5 How do emerging cybersecurity threats like data poisoning and source code extraction challenge current regulatory frameworks for AI medical devices?

Key findings

• AI medical devices face significant cybersecurity threats, including data poisoning, social engineering attacks, and unauthorized extraction of model weights or source code.
• The MDR and the AI Act may create regulatory overlap or conflict, particularly in defining the scope of cybersecurity and safety requirements for AI medical devices.
• Incident notification requirements under the NIS 2 Directive proposal and MDR may lead to compliance ambiguity due to differing thresholds and reporting timelines.
• The evolving definition of 'critical infrastructure' in EU law may expand the regulatory scope to include AI medical devices, increasing compliance obligations.
• The interaction between GDPR, MDR, and the AI Act raises concerns about legal fragmentation and inconsistent enforcement across regulatory domains.
• The paper concludes that regulatory clarity is urgently needed to ensure consistent, enforceable, and interoperable cybersecurity standards for AI medical devices across the EU.