[Paper Review] DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4

Zhengliang Liu, Huang, Yue+16 more|arXiv (Cornell University)|Mar 20, 2023

Topic ModelingComputer Science89 citations

TL;DR

The paper proposes DeID-GPT, a GPT-4/ChatGPT-based zero-shot framework to automatically de-identify free-text medical notes by embedding HIPAA identifiers into prompts and processing notes to redact PHI while preserving text meaning. It evaluates on the i2b2/UTHealth de-identification dataset and highlights prompt design as the key driver of performance.

ABSTRACT

The digitization of healthcare has facilitated the sharing and re-using of medical data but has also raised concerns about confidentiality and privacy. HIPAA (Health Insurance Portability and Accountability Act) mandates removing re-identifying information before the dissemination of medical records. Thus, effective and efficient solutions for de-identifying medical data, especially those in free-text forms, are highly needed. While various computer-assisted de-identification methods, including both rule-based and learning-based, have been developed and used in prior practice, such solutions still lack generalizability or need to be fine-tuned according to different scenarios, significantly imposing restrictions in wider use. The advancement of large language models (LLM), such as ChatGPT and GPT-4, have shown great potential in processing text data in the medical domain with zero-shot in-context learning, especially in the task of privacy protection, as these models can identify confidential information by their powerful named entity recognition (NER) capability. In this work, we developed a novel GPT4-enabled de-identification framework (``DeID-GPT") to automatically identify and remove the identifying information. Compared to existing commonly used medical text data de-identification methods, our developed DeID-GPT showed the highest accuracy and remarkable reliability in masking private information from the unstructured medical text while preserving the original structure and meaning of the text. This study is one of the earliest to utilize ChatGPT and GPT-4 for medical text data processing and de-identification, which provides insights for further research and solution development on the use of LLMs such as ChatGPT/GPT-4 in healthcare. Codes and benchmarking data information are available at https://github.com/yhydhx/ChatGPT-API.

Motivation & Objective

• Motivate HIPAA-compliant sharing of free-text medical data by removing identifying information.
• Leverage zero-shot in-context learning of GPT-4/ChatGPT to identify and redact PHI in clinical notes without fine-tuning.
• Design high-quality prompts that integrate HIPAA identifiers to guide de-identification while preserving text meaning.

Proposed method

• Use GPT-4/ChatGPT in a zero-shot framework with prompts that embed HIPAA identifiers to specify de-identification tasks.
• Process original clinical notes through prompts to generate de-identified outputs.
• Map and redact PHI in accordance with HIPAA categories within the prompt-driven workflow.
• Evaluate performance on the 2014 i2b2/UTHealth de-identification dataset using synthetic surrogates.
• Provide open-source code and benchmarking data for reproducibility.

Experimental results

Research questions

• RQ1 Can GPT-4/ChatGPT achieve high-accuracy de-identification of free-text medical notes without fine-tuning?
• RQ2 How do prompt designs and HIPAA-identifier integration affect de-identification performance?
• RQ3 How well does zero-shot de-identification preserve the original meaning and structure of clinical notes?

Key findings

• DeID-GPT achieves the highest accuracy among compared methods in masking private information from unstructured clinical text.
• The approach maintains the original structure and meaning of notes while redacting PHI.
• Prompt engineering, specifically incorporating HIPAA identifiers into prompts, is the primary driver of performance.
• The study is among the earliest to apply ChatGPT/GPT-4 to medical text processing and de-identification.
• Code and benchmarking data are publicly available for reproducibility.