[Paper Review] MASK: A flexible framework to facilitate de-identification of clinical texts

Nikola Milošević, Gangamma Kalappa+3 more|arXiv (Cornell University)|May 24, 2020

Topic Modeling12 references4 citations

TL;DR

MASK is a flexible, open-source Python framework for de-identifying clinical texts by combining customizable named entity recognition (NER) and masking strategies. It supports state-of-the-art models like BiLSTM with ELMo and CRF-based methods, achieving F1-scores up to 0.98 for names and 0.97 overall on the i2b2 2014 dataset, significantly accelerating and improving de-identification for research use.

ABSTRACT

Medical health records and clinical summaries contain a vast amount of important information in textual form that can help advancing research on treatments, drugs and public health. However, the majority of these information is not shared because they contain private information about patients, their families, or medical staff treating them. Regulations such as HIPPA in the US, PHIPPA in Canada and GDPR regulate the protection, processing and distribution of this information. In case this information is de-identified and personal information are replaced or redacted, they could be distributed to the research community. In this paper, we present MASK, a software package that is designed to perform the de-identification task. The software is able to perform named entity recognition using some of the state-of-the-art techniques and then mask or redact recognized entities. The user is able to select named entity recognition algorithm (currently implemented are two versions of CRF-based techniques and BiLSTM-based neural network with pre-trained GLoVe and ELMo embedding) and masking algorithm (e.g. shift dates, replace names/locations, totally redact entity).

Motivation & Objective

• To address the challenge of de-identifying clinical texts while preserving research utility.
• To provide a modular, extensible framework that supports multiple NER and masking algorithms.
• To reduce the cost and time of manual de-identification, which can exceed $500,000 for 50,000 records.
• To enable researchers to select optimal NER and masking methods per entity type, improving accuracy and privacy compliance.
• To support integration as a library or tool in larger research systems, with extensibility for new models and data formats.

Proposed method

• MASK uses a plugin-based architecture to allow interchangeable named entity recognition (NER) and masking algorithms.
• It supports two CRF-based NER models using lexical and dictionary features, and two BiLSTM-based models with pre-trained ELMo and GLoVe embeddings.
• NER models are trained or fine-tuned on clinical text datasets, including i2b2 2014 and ICES premise data.
• Masking strategies include entity replacement, date shifting, and full redaction, configurable per entity type.
• The framework supports multiple input formats via customizable reader plugins for training and inference.
• It enables end-to-end de-identification by first detecting PHI/PII via NER, then applying user-selected masking techniques.

Experimental results

Research questions

• RQ1 Can a modular framework improve the accuracy and efficiency of clinical text de-identification compared to monolithic tools?
• RQ2 How do different NER models—CRF-based and deep learning-based—perform on clinical de-identification tasks?
• RQ3 What is the trade-off between speed, accuracy, and generalizability across different NER models in clinical settings?
• RQ4 Can fine-tuning pre-trained embeddings improve performance on low-resource clinical entity types?
• RQ5 To what extent can a flexible, plugin-based architecture support diverse research and clinical use cases?

Key findings

• The BiLSTM model with residual connections and ELMo embeddings achieved the highest overall F1-score of 0.97 on the i2b2 2014 test set.
• The NAME entity class achieved an F1-score of 0.98, indicating strong performance for person names.
• The CRF model with lexical and dictionary features outperformed many prior systems and was significantly faster than deep learning models.
• The BiLSTM model with GLoVe embeddings underperformed, likely due to the use of non-clinical pre-trained embeddings.
• The ELMo-based model generalized well across domains and required only small fine-tuning datasets for adaptation.
• Training the ELMo model took about one hour per epoch on CPU, and inference took up to 10 seconds per narrative, while CRF models processed each narrative in under 0.5 seconds.