[Paper Review] CT-GAN: Malicious Tampering of 3D Medical Imagery using Deep Learning

Yisroel Mirsky, Tom Mahler+2 more|arXiv (Cornell University)|Jan 11, 2019

Digital Media Forensic Detection46 references108 citations

TL;DR

CT-GAN demonstrates automated, realistic injection/removal of evidence (e.g., lung cancer) in 3D CT scans using conditional GANs, and verifies susceptibility of radiologists and AI to such tampering.

ABSTRACT

In 2018, clinics and hospitals were hit with numerous attacks leading to significant data breaches and interruptions in medical services. An attacker with access to medical records can do much more than hold the data for ransom or sell it on the black market. In this paper, we show how an attacker can use deep-learning to add or remove evidence of medical conditions from volumetric (3D) medical scans. An attacker may perform this act in order to stop a political candidate, sabotage research, commit insurance fraud, perform an act of terrorism, or even commit murder. We implement the attack using a 3D conditional GAN and show how the framework (CT-GAN) can be automated. Although the body is complex and 3D medical scans are very large, CT-GAN achieves realistic results which can be executed in milliseconds. To evaluate the attack, we focused on injecting and removing lung cancer from CT scans. We show how three expert radiologists and a state-of-the-art deep learning AI are highly susceptible to the attack. We also explore the attack surface of a modern radiology network and demonstrate one attack vector: we intercepted and manipulated CT scans in an active hospital network with a covert penetration test. Demo video: https://youtu.be/_mkRAArj-x0 Source code: https://github.com/ymirsky/CT-GAN

Motivation & Objective

• Expose a realistic, automated method for tampering 3D medical imagery by injecting or removing evidence of disease.
• Assess the threat by testing radiologists and AI lung cancer screening systems on tampered CT scans.
• Demonstrate an attack vector within a hospital PACS network and discuss practical countermeasures.
• Provide reproducible datasets and code to enable further investigation and defense.

Proposed method

• Use two conditional GANs (cGANs) for in-painting to inject (unhealthy) or remove (healthy) tissue in localized 3D CT regions.
• Process involves locating target evidence regions, extracting a rectangular cuboid, resizing (interpolating) it, applying the cGAN modification, and reinserting the cuboid into the scan.
• Enhance realism with histogram equalization, tissue-density masking (Hounsfield units), and Gaussian noise to reduce border artifacts.
• Automate the attack to function in air-gapped PACS environments and validate with expert radiologists and a state-of-the-art AI lung cancer screening model.
• Demonstrate a hospital penetration test to showcase practical attack surface and potential interception paths.

Experimental results

Research questions

• RQ1 Can a 3D medical image (CT scan) be realistically tampered with to inject or remove evidence of disease using deep learning?
• RQ2 How effective are radiologists and machine learning models at detecting tampered 3D medical imagery?
• RQ3 What are plausible attack surfaces and vectors within a modern PACS/RIS network for deploying such tampering?
• RQ4 What countermeasures can mitigate automated tampering and improve detection?

Key findings

• Three radiologists correctly diagnosed 99% of injected cancer cases as malignant.
• Three radiologists judged 94% of cancer-removed cases as healthy.
• After informing radiologists about the attack, 60% of injections were still misdiagnosed as cancer, and 87% of removals were misdiagnosed as cancer.
• A state-of-the-art AI lung cancer screening model misdiagnosed 100% of tampered patients.
• The attack is feasible in realistic hospital networks and can be automated for air-gapped systems.