[Paper Review] MuSe-GNN: Learning Unified Gene Representation From Multimodal Biological Graph Data

Tianyu Liu, Yuge Wang+2 more|arXiv (Cornell University)|Sep 29, 2023

Bioinformatics and Genomic Networks12 citations

TL;DR

MuSe-GNN learns unified gene embeddings by sharing weights across multimodal gene graphs and using reconstruction, weighted similarity, and contrastive losses to capture cross-dataset gene function similarity. It outperforms baselines across tissues and modalities.

ABSTRACT

Discovering genes with similar functions across diverse biomedical contexts poses a significant challenge in gene representation learning due to data heterogeneity. In this study, we resolve this problem by introducing a novel model called Multimodal Similarity Learning Graph Neural Network, which combines Multimodal Machine Learning and Deep Graph Neural Networks to learn gene representations from single-cell sequencing and spatial transcriptomic data. Leveraging 82 training datasets from 10 tissues, three sequencing techniques, and three species, we create informative graph structures for model training and gene representations generation, while incorporating regularization with weighted similarity learning and contrastive learning to learn cross-data gene-gene relationships. This novel design ensures that we can offer gene representations containing functional similarity across different contexts in a joint space. Comprehensive benchmarking analysis shows our model's capacity to effectively capture gene function similarity across multiple modalities, outperforming state-of-the-art methods in gene representation learning by up to 97.5%. Moreover, we employ bioinformatics tools in conjunction with gene representations to uncover pathway enrichment, regulation causal networks, and functions of disease-associated or dosage-sensitive genes. Therefore, our model efficiently produces unified gene representations for the analysis of gene functions, tissue functions, diseases, and species evolution.

Motivation & Objective

• Address data heterogeneity in multimodal biological data to learn unified gene representations across tissues, sequencing techniques, and species.
• Develop a weight-sharing GNN framework that integrates multiple gene co-expression graphs into a joint embedding space.
• Regularize embeddings with weighted similarity and contrastive learning to reveal cross-dataset gene function relationships.
• Demonstrate improved downstream utility of embeddings for pathway analysis, disease studies, and gene function prediction.

Proposed method

• Construct high-variance gene graphs for each dataset using HVGs and noise-robust processing (scTransform, CS-CORE, SPARK-X).
• Use cross-graph Transformer with dataset-specific and shared GT layers to enforce weight sharing across graphs.
• Decode node embeddings with dataset-specific MLP decoders to reconstruct co-expression networks (Graph Auto-encoder style).
• Train with a multi-component loss: graph reconstruction BCE loss, weighted cosine-similarity loss across common HVGs, and self-supervised InfoNCE contrastive loss for cross-graph consistency.
• Employ a final objective that combines BCE, weighted Cosine Similarity, and InfoNCE losses with tunable weight λc.
• Evaluate using multiple biological benchmarks and six performance metrics across tissues.

Figure 1: The workflow of MuSe-GNN, the visualization of gene embeddings, and the problems of two exitsing methods, GIANT and Gene2vec. (a) The process of learning gene embeddings by MuSe-GNN. Here we highlight the difference between single-cell data and spatial data, and the major applications of g

Experimental results

Research questions

• RQ1 Can a unified gene embedding space be learned from heterogeneous multimodal biological graphs while preserving functional similarity across datasets?
• RQ2 Does weight-sharing across dataset-specific graph transformers improve cross-dataset gene clustering compared to non-shared baselines?
• RQ3 Do weighted similarity and contrastive losses improve recovery of known gene functional groups and pathways across datasets?
• RQ4 Can the learned embeddings improve downstream tasks such as gene function prediction and pathway analysis across tissues and diseases.

Key findings

• MuSe-GNN outperforms baselines across tissues on six evaluation metrics, achieving up to 97.5% improvement in comprehensive assessment.
• In heart and lung, MuSe-GNN is 20.1% and 97.5% better than the second-best method, respectively.
• Embeddings from MuSe-GNN integrate multimodal data into a co-embedded space, revealing functional gene clusters shared across modalities.
• GOEA and IPA analyses on embeddings identify biologically meaningful pathways, causal networks, and disease associations.
• Gene-function prediction using MuSe-GNN embeddings achieves higher accuracy than baselines like Geneformer and raw data.

Figure 2: The overall model architecture and the design of loss functions for MuSe-GNN. The color of nodes in the green block represents common/different genes across two datasets. The brown block represents the network architecture of MuSe-GNN, and the blue block represents different loss function