[Paper Review] Whole Genome Transformer for Gene Interaction Effects in Microbiome Habitat Specificity

Zhufeng Li, Sandeep Suresh Cranganore+2 more|arXiv (Cornell University)|May 9, 2024

CRISPR and Genetic EngineeringBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This paper introduces a Whole Genome Transformer framework that leverages large-scale pre-trained models to predict microbial habitat specificity from complete genome sequences, using attention-based attribution to identify key gene interaction networks. The method achieves strong predictive performance and uncovers biologically relevant gene associations, including known and novel candidates for experimental validation in diverse environments.

ABSTRACT

Leveraging the vast genetic diversity within microbiomes offers unparalleled insights into complex phenotypes, yet the task of accurately predicting and understanding such traits from genomic data remains challenging. We propose a framework taking advantage of existing large models for gene vectorization to predict habitat specificity from entire microbial genome sequences. Based on our model, we develop attribution techniques to elucidate gene interaction effects that drive microbial adaptation to diverse environments. We train and validate our approach on a large dataset of high quality microbiome genomes from different habitats. We not only demonstrate solid predictive performance, but also how sequence-level information of entire genomes allows us to identify gene associations underlying complex phenotypes. Our attribution recovers known important interaction networks and proposes new candidates for experimental follow up.

Motivation & Objective

• To address the challenge of predicting complex microbial phenotypes, particularly habitat specificity, from whole-genome sequences.
• To overcome limitations of existing models that focus on single genes or short DNA sequences by modeling large-scale genomic segments.
• To identify and interpret gene-gene interactions that drive microbial adaptation to diverse environments.
• To develop attribution techniques that reveal functional gene networks underlying complex phenotypes.
• To provide a framework that recovers known biological networks and proposes novel candidates for experimental follow-up.

Proposed method

• The framework employs a pre-trained transformer model (ESM) fine-tuned on complete microbial genome sequences to generate contextualized gene embeddings.
• It uses sequence-level attention mechanisms to capture long-range dependencies and interactions across entire genomes.
• A gradient-based attribution method is applied to identify genes with high contribution to habitat specificity predictions.
• Gene interaction networks are reconstructed from attribution scores, where hubs represent genes with multiple significant interactions.
• The method integrates functional annotations (e.g., KEGG Orthology) to interpret the biological relevance of identified gene clusters.
• The approach is validated on a large, high-quality dataset of microbiome genomes from diverse habitats, including aquatic and soil environments.

Experimental results

Research questions

• RQ1 Which gene interactions across entire microbial genomes are most predictive of habitat specificity?
• RQ2 Can transformer-based models trained on full genomes outperform models restricted to single genes or short sequences in predicting microbial phenotypes?
• RQ3 To what extent can attribution methods recover known biological networks involved in microbial adaptation?
• RQ4 What novel gene interaction candidates does the model identify that may be relevant for experimental validation?
• RQ5 How do gene interaction networks differ between free-living microbes (e.g., Prochlorococcus) and soil-dwelling microbes (e.g., Acidimicrobiia)?

Key findings

• The Whole Genome Transformer achieves strong predictive performance in classifying microbial habitat specificity using only genomic sequences.
• The model successfully recovers known functional gene networks, such as those involved in energy metabolism and stress response, confirming biological plausibility.
• Hub genes identified via attribution are enriched in operons and co-regulated gene clusters, suggesting functional coherence.
• The method identifies novel candidate gene interactions in understudied taxa, such as Prochlorococcus marinus and Acidimicrobiia, that may drive environmental adaptation.
• Gene interaction networks reconstructed from attribution scores show distinct topological patterns across habitats, reflecting ecological specialization.
• The approach demonstrates that modeling entire genomes enables detection of complex, non-local interactions that single-gene models miss.