[Paper Review] MatchMiner-AI: An Open-Source Solution for Cancer Clinical Trial Matching

Altreuter, Jennifer, Pavel Trukhanov+25 more|arXiv (Cornell University)|Dec 23, 2024

Biomedical Text Mining and Ontologies4 citations

TL;DR

MatchMiner-AI is an open-source, privacy-preserving AI platform that uses synthetic electronic health record (EHR) data to match cancer patients to relevant clinical trials. It leverages text embeddings and custom classifiers to rank trial-patient pairings with 90% relevance in top-20 recommendations—significantly outperforming baseline models—demonstrating strong clinical utility without using real protected health data.

ABSTRACT

Background Clinical trials are essential to advancing cancer treatments, yet fewer than 10% of adults with cancer enroll in trials, and many studies fail to meet accrual targets. Artificial intelligence (AI) could improve identification of appropriate trials for patients, but sharing AI models trained on protected health information remains difficult due to privacy restrictions. Methods We developed MatchMiner-AI, an open-source platform for clinical trial search and ranking trained entirely on synthetic electronic health record (EHR) data. The system extracts core clinical criteria from longitudinal EHR text and embeds patient summaries and trial "spaces" (target populations) in a shared vector space for rapid retrieval. It then applies custom text classifiers to assess whether each patient-trial pairing is a clinically reasonable consideration. The pipeline was evaluated on real clinical data. Results Across retrospective evaluations on real EHR data, the fine-tuned pipeline outperformed baseline text-embedding approaches. For trial-enrolled patients, 90% of the top 20 recommended trials were relevant matches (compared to 17% for the baseline model). Similar improvements were noted for patients who received standard-of-care treatments (88% of the top 20 matches were relevant, compared to 14% for baseline). Text classification modules demonstrated strong discrimination (AUROC 0.94-0.98) for evaluating candidate patient-trial space pair eligibility; incorporating these components consistently increased mean average precision to ~ 0.90 across patient- and trial-centric use cases. Synthetic training data, model weights, inference tools, and demonstration frontends are publicly available. Conclusions MatchMiner-AI demonstrates an openly accessible, privacy-preserving approach to distilling a clinical trial matching AI pipeline from LLM-generated synthetic EHR data.

Motivation & Objective

• To address low clinical trial enrollment rates in cancer patients by developing an AI-driven matching system.
• To overcome privacy barriers in sharing AI models trained on protected health information.
• To create a fully open-source, reproducible pipeline for clinical trial matching using synthetic EHR data.
• To improve the accuracy and relevance of patient-trial recommendations beyond baseline embedding methods.
• To enable deployment of a scalable, privacy-compliant solution for oncology trial matching in clinical settings.

Proposed method

• The system generates synthetic EHR data to train a clinical trial matching pipeline without using real patient data.
• It extracts core clinical criteria from longitudinal EHR text using large language model (LLM)-based processing.
• Patient summaries and trial 'spaces' (target populations) are embedded into a shared vector space for efficient similarity retrieval.
• Custom text classifiers are fine-tuned to assess clinical eligibility of patient-trial pairings with high discrimination.
• The pipeline combines retrieval via vector similarity with binary classification to refine top-ranked matches.
• All model weights, inference tools, and demonstration frontends are publicly released for community use.

Experimental results

Research questions

• RQ1 Can an AI-driven clinical trial matching system achieve high relevance in recommendations without using real protected health data?
• RQ2 How does a retrieval-and-classification pipeline using synthetic EHR data compare to baseline embedding-only models in clinical trial matching accuracy?
• RQ3 To what extent can text classifiers improve mean average precision in patient-trial pair eligibility assessment?
• RQ4 Can a fully open-source, privacy-preserving system achieve performance comparable to real-data-trained models?
• RQ5 What is the impact of combining vector-space retrieval with rule-based text classification on top-20 recommendation relevance?

Key findings

• For trial-enrolled patients, 90% of the top 20 recommended trials were clinically relevant, compared to only 17% with the baseline model.
• For patients receiving standard-of-care treatments, 88% of the top 20 matches were relevant, versus 14% for the baseline.
• Text classification modules achieved AUROC scores between 0.94 and 0.98, indicating strong discrimination in eligibility assessment.
• Incorporating text classifiers increased mean average precision to approximately 0.90 across both patient- and trial-centric use cases.
• The system's performance was validated on real clinical data despite being trained exclusively on synthetic EHR data.
• All components, including synthetic training data, model weights, and inference tools, are publicly available for reuse and extension.