[Paper Review] Gene-associated Disease Discovery Powered by Large Language Models

Jiayu Chang, Shiyu Wang+3 more|arXiv (Cornell University)|Jan 16, 2024

Biomedical Text Mining and OntologiesBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This paper proposes a Large Language Model (LLM)-powered framework to automate the discovery of gene-disease associations by retrieving and summarizing relevant scientific literature. Using PubMed API and LLMs like GPT-4, the system identifies and ranks diseases linked to specific genes with high accuracy, achieving a Hit Ratio of 0.8 at top-20 ranking for Alzheimer’s disease, demonstrating strong performance in real-time, context-aware disease prediction.

ABSTRACT

The intricate relationship between genetic variation and human diseases has been a focal point of medical research, evidenced by the identification of risk genes regarding specific diseases. The advent of advanced genome sequencing techniques has significantly improved the efficiency and cost-effectiveness of detecting these genetic markers, playing a crucial role in disease diagnosis and forming the basis for clinical decision-making and early risk assessment. To overcome the limitations of existing databases that record disease-gene associations from existing literature, which often lack real-time updates, we propose a novel framework employing Large Language Models (LLMs) for the discovery of diseases associated with specific genes. This framework aims to automate the labor-intensive process of sifting through medical literature for evidence linking genetic variations to diseases, thereby enhancing the efficiency of disease identification. Our approach involves using LLMs to conduct literature searches, summarize relevant findings, and pinpoint diseases related to specific genes. This paper details the development and application of our LLM-powered framework, demonstrating its potential in streamlining the complex process of literature retrieval and summarization to identify diseases associated with specific genetic variations.

Motivation & Objective

• To address the limitations of static, outdated disease-gene databases by enabling real-time, dynamic discovery of gene-disease associations.
• To automate the labor-intensive process of sifting through thousands of medical literature articles to identify relevant disease links for specific genes.
• To improve clinical decision-making by providing timely, evidence-based disease predictions grounded in current research.
• To evaluate the effectiveness of LLMs in understanding and ranking gene-disease relationships using contextual evidence from scientific abstracts.

Proposed method

• Leverages PubMed API to retrieve top-K scientific literature articles based on gene-specific queries, using relevance and recency as criteria.
• Employs fine-tuned prompts to instruct LLMs (e.g., GPT-4) to extract and summarize disease associations from article abstracts and full texts.
• Uses in-context learning in LLMs to rank diseases by perceived strength of association, based on linguistic cues like 'most common' or '20% associated'.
• Applies a retrieval hyperparameter N to control the number of literature references used for LLM inference, optimizing for knowledge density and relevance.
• Employs Hit Ratio (HR) at top-K as the primary evaluation metric to measure recall performance in identifying target diseases like Alzheimer’s.
• Validates the framework on a curated subset of the DisGeNET database, focusing on known gene-disease pairs for benchmarking.

Experimental results

Research questions

• RQ1 Can LLMs effectively retrieve and summarize relevant scientific literature to identify diseases associated with specific genes?
• RQ2 How does the number of retrieved literature references (N) affect the accuracy of disease ranking by LLMs?
• RQ3 To what extent can LLMs understand contextual cues in medical literature to assess the strength of gene-disease associations?
• RQ4 How does the framework’s performance compare to traditional static databases in terms of recall and real-time relevance?

Key findings

• The Hit Ratio for Alzheimer’s disease reached 0.8 at top-20 ranking, indicating strong performance in identifying the correct disease among top-ranked outputs.
• As the number of retrieved literature references (N) increased, the frequency of Alzheimer’s disease appearing in the top-K results also increased, showing improved recall with greater knowledge coverage.
• LLMs demonstrated contextual understanding by correctly identifying Alzheimer’s disease as the primary association for PSEN1, citing phrases like 'most common cause of familial Alzheimer’s disease'.
• The system achieved a steady rise in HR as K increased from 2 to 20, with notable gains observed at K=10, 15, and 20, indicating robustness across different ranking thresholds.
• The framework successfully ranked Alzheimer’s disease at the top for PSEN1, correctly identifying it as the primary disease linked to the gene based on abstract content.
• The results confirm that LLMs can effectively process and reason over complex medical literature to extract and rank disease associations with high precision.