[Paper Review] Predicting Anti-microbial Resistance using Large Language Models

Hyunwoo Yoo, Bahrad A. Sokhansanj+2 more|arXiv (Cornell University)|Jan 1, 2024

Machine Learning in BioinformaticsBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This paper proposes a novel ensemble model that combines a nucleotide sequence-based large language model (LLM) with a biomedical text LLM (BioBERT) fine-tuned on PubMed and resistance gene databases to predict antibiotic resistance (AMR) drug classes. By integrating sequence data with biological background knowledge from text and using LLM-based data augmentation for rare classes, the method achieves state-of-the-art performance on a new benchmark, outperforming sequence-only models.

ABSTRACT

During times of increasing antibiotic resistance and the spread of infectious diseases like COVID-19, it is important to classify genes related to antibiotic resistance. As natural language processing has advanced with transformer-based language models, many language models that learn characteristics of nucleotide sequences have also emerged. These models show good performance in classifying various features of nucleotide sequences. When classifying nucleotide sequences, not only the sequence itself, but also various background knowledge is utilized. In this study, we use not only a nucleotide sequence-based language model but also a text language model based on PubMed articles to reflect more biological background knowledge in the model. We propose a method to fine-tune the nucleotide sequence language model and the text language model based on various databases of antibiotic resistance genes. We also propose an LLM-based augmentation technique to supplement the data and an ensemble method to effectively combine the two models. We also propose a benchmark for evaluating the model. Our method achieved better performance than the nucleotide sequence language model in the drug resistance class prediction.

Motivation & Objective

• To improve the accuracy of antibiotic resistance gene classification by integrating biological sequence data with textual knowledge from scientific literature.
• To address data scarcity in rare antibiotic resistance classes through LLM-based data augmentation.
• To develop a unified classification framework by harmonizing labels from multiple databases (CARD and MEGARes) using the EBI ARO ontology.
• To evaluate the performance of a dual-model ensemble combining sequence and text LLMs against existing state-of-the-art AMR prediction methods.
• To establish a benchmark for evaluating AMR prediction models using multi-source, multi-attribute resistance gene data.

Proposed method

• Fine-tuned a multi-species nucleotide sequence LLM (NT) using a 6-mer tokenizer and LoRA for low-rank adaptation to predict drug resistance classes.
• Fine-tuned BioBERT on biomedical text from PubMed and full-text articles to extract resistance attributes such as Gene Family and Resistance Mechanism.
• Integrated data from CARD and MEGARes using the EBI ARO ontology to unify heterogeneous resistance gene labels into a single classification system.
• Applied BioGPT prompting to generate synthetic text descriptions for rare resistance classes, enhancing training data diversity.
• Combined the two models via weighted soft voting ensemble to leverage complementary strengths of sequence and text representations.
• Generated paired-end sequencing reads using ART simulator to evaluate model robustness on real-world metagenomic data.

Experimental results

Research questions

• RQ1 Can combining sequence-based and text-based LLMs improve the accuracy of antibiotic resistance drug class prediction compared to sequence-only models?
• RQ2 How effective is LLM-based data augmentation in improving model performance for rare antibiotic resistance classes with limited training samples?
• RQ3 To what extent does integrating multi-database resistance gene annotations (CARD and MEGARes) using a common ontology enhance model generalization and consistency?
• RQ4 How does the ensemble of sequence and text LLMs compare to existing state-of-the-art AMR prediction tools like AMR-meta, AMR++, Meta-MARC, and DeepARG?
• RQ5 Does the model maintain strong performance when evaluated on simulated metagenomic sequencing reads, indicating real-world applicability?

Key findings

• The proposed ensemble model achieved better macro F1 scores than the sequence-only LLM baseline, demonstrating the value of integrating textual biological knowledge.
• The use of BioGPT for data augmentation significantly improved performance on rare resistance classes, reducing data imbalance effects.
• Harmonizing CARD and MEGARes labels using the EBI ARO ontology enabled consistent, unified classification across diverse databases.
• The model demonstrated strong competitiveness on simulated metagenomic reads, showing robustness to sequencing-level noise and variability.
• The weighted soft voting ensemble outperformed individual models, confirming the complementary nature of sequence and text representations.
• The benchmark proposed in the study provides a standardized evaluation framework for future AMR prediction models.