[Paper Review] Machine Learning for Antimicrobial Resistance

John Santerre, James J. Davis+2 more|arXiv (Cornell University)|Jul 5, 2016

Computational Drug Discovery Methods5 references16 citations

TL;DR

This paper demonstrates that ensemble machine learning techniques, particularly Random Forest, can effectively classify antimicrobial resistance (AMR) phenotypes from genomic data with classification accuracies ranging from 80% to 92%, even on small datasets. It shows that these models reliably identify known resistance-associated gene regions, highlighting the potential of applied machine learning for global health challenges in underserved communities.

ABSTRACT

Biological datasets amenable to applied machine learning are more available today than ever before, yet they lack adequate representation in the Data-for-Good community. Here we present a work in progress case study performing analysis on antimicrobial resistance (AMR) using standard ensemble machine learning techniques and note the successes and pitfalls such work entails. Broadly, applied machine learning (AML) techniques are well suited to AMR, with classification accuracies ranging from mid-90% to low- 80% depending on sample size. Additionally, these techniques prove successful at identifying gene regions known to be associated with the AMR phenotype. We believe that the extensive amount of biological data available, the plethora of problems presented, and the global impact of such work merits the consideration of the Data- for-Good community.

Motivation & Objective

• To demonstrate the feasibility of applying off-the-shelf machine learning techniques to public biological datasets for antimicrobial resistance (AMR) prediction.
• To address the underrepresentation of biological datasets in the Data-for-Good community by showcasing real-world impact in global health.
• To identify gene regions associated with AMR phenotypes using feature importance from trained models, even with limited training data.
• To encourage the Data-for-Good community to engage with computational biology by highlighting accessible, high-impact problems in AMR.
• To explore three distinct objectives: maximizing classification accuracy, improving generalization across species/antibiotics, and enabling rapid feature selection for outbreak response.

Proposed method

• Transformed genotype-to-phenotype classification into a supervised machine learning task using k-mer representation of genomic sequences.
• Applied standard ensemble learning techniques, primarily Random Forest (RF), to classify AMR phenotypes from whole-genome sequencing data.
• Used k-mer frequency counts (k = 10 to 30) as input features, with increasing k leading to higher feature dimensionality and larger disk sizes.
• Performed 80/20 train-test splits to evaluate model performance across varying sample sizes, with accuracy tracked as a function of training set size.
• Conducted feature importance analysis using Random Forest to identify top gene regions associated with resistance, assessing stability across decreasing sample sizes.
• Evaluated model performance across three bacterial species: *Streptococcus pneumoniae*, *Acinetobacter baumannii*, and *Mycobacterium tuberculosis*, each with resistance to specific antibiotics.

Experimental results

Research questions

• RQ1 Can standard machine learning models achieve high accuracy in predicting antimicrobial resistance phenotypes from genomic data?
• RQ2 How stable are the identified resistance-associated gene regions when trained on progressively smaller datasets?
• RQ3 To what extent can machine learning models generalize across different bacterial species or antibiotics for AMR prediction?
• RQ4 Can feature importance from trained models reliably identify known resistance genes, even with limited training data?
• RQ5 What are the practical implications of using off-the-shelf ML tools on public biological datasets for real-world public health applications?

Key findings

• Random Forest models achieved classification accuracies between 80% and 92% on AMR phenotype prediction, depending on sample size and species.
• Model accuracy plateaued at larger sample sizes, indicating diminishing returns beyond a certain dataset size.
• Top gene regions associated with resistance remained stable in feature importance rankings even with as few as 25 training isolates.
• The models successfully identified known resistance-associated gene regions, validating their biological relevance.
• Feature importance analysis revealed consistent clustering of mutations in specific genomic regions, suggesting biological coherence in resistance mechanisms.
• Despite lower accuracy on small datasets, the models retained sufficient predictive power to identify key resistance loci, supporting use in early-outbreak scenarios.