[Paper Review] Archive of PubTator 3.0 source code and trained models

Chih-Hsuan Wei, Alexis Allot+8 more|PubMed|Jan 19, 2024

Biomedical Text Mining and OntologiesBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

PubTator 3.0 is an AI-powered biomedical literature resource that provides over one billion entity and relation annotations across 36 million PubMed abstracts and 6 million PMC full-text articles using state-of-the-art NLP models. It enables advanced semantic and relation searches with higher precision and recall than PubMed or Google Scholar, and integrates with large language models like GPT-4 to improve response factuality and verifiability.

ABSTRACT

PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases, and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.

Motivation & Objective

• To address the growing challenge of navigating the exponentially expanding biomedical literature by enabling efficient, semantic-aware information retrieval.
• To develop a scalable, AI-powered system that automatically annotates key biomedical entities and their relationships in scientific literature.
• To improve the precision and recall of literature search beyond traditional keyword-based systems like PubMed and Google Scholar.
• To integrate with large language models to enhance the factual reliability and verifiability of AI-generated responses in biomedical contexts.
• To provide a publicly accessible, up-to-date, and scalable resource for researchers to accelerate knowledge discovery in biomedicine.

Proposed method

• The system employs state-of-the-art deep learning models fine-tuned on biomedical corpora to extract and classify entities such as proteins, diseases, chemicals, and genetic variants.
• It applies neural relation extraction models to identify semantic relationships between annotated entities in text, such as protein-disease or drug-chemical interactions.
• Annotations are precomputed and indexed across a comprehensive corpus of 36 million PubMed abstracts and 6 million PMC full-text articles, updated weekly.
• The system exposes an online interface and a programmatic API that support semantic and relation-based queries using entity synonyms and normalized identifiers.
• It integrates with GPT-4 via API calls to validate and refine LLM-generated responses, improving factual consistency and traceability to source literature.
• The pipeline leverages transfer learning from large language models pre-trained on biomedical text, enabling high performance with minimal fine-tuning.

Experimental results

Research questions

• RQ1 Can an AI-powered system achieve higher precision and recall in biomedical literature search compared to PubMed and Google Scholar?
• RQ2 To what extent can precomputed entity and relation annotations improve the efficiency and accuracy of semantic search in biomedical literature?
• RQ3 How does integrating a large language model like GPT-4 with a curated knowledge base like PubTator 3.0 enhance the factuality and verifiability of AI-generated answers?
• RQ4 Can a scalable, automated system maintain high-quality annotations across a large, dynamically updated corpus of biomedical literature?
• RQ5 What is the impact of synonym normalization and entity linking on the effectiveness of semantic search in complex biomedical queries?

Key findings

• PubTator 3.0 retrieves significantly more relevant articles than PubMed or Google Scholar for entity pair queries, with higher precision in the top 20 results.
• The system achieves over one billion entity and relation annotations across 36 million abstracts and 6 million full-text articles from the PMC open access subset.
• The integration of GPT-4 with PubTator 3.0 APIs results in more factually consistent and verifiable responses compared to standalone LLM inference.
• The precomputed annotations enable low-latency, scalable semantic search, supporting large-scale data analysis and knowledge discovery.
• The system is updated weekly, ensuring access to the most current biomedical literature with consistent entity and relation labeling.
• The use of standardized entity identifiers and synonym mapping enhances cross-document entity linking and query resolution.