[Paper Review] CrossLLM-Mamba: Multimodal State Space Fusion of LLMs for RNA Interaction Prediction

Rabeya Tus Sadia, Qiang Ye+1 more|arXiv (Cornell University)|Feb 23, 2026

RNA and protein synthesis mechanisms0 citations

TL;DR

CrossLLM-Mamba redefines RNA interaction prediction as a bidirectional state-space alignment task, enabling dynamic cross-modal fusion of BioLLM embeddings via a BiMamba encoder, achieving state-of-the-art results across RNA–protein, RNA–small molecule, and RNA–RNA interactions.

ABSTRACT

Accurate prediction of RNA-associated interactions is essential for understanding cellular regulation and advancing drug discovery. While Biological Large Language Models (BioLLMs) such as ESM-2 and RiNALMo provide powerful sequence representations, existing methods rely on static fusion strategies that fail to capture the dynamic, context-dependent nature of molecular binding. We introduce CrossLLM-Mamba, a novel framework that reformulates interaction prediction as a state-space alignment problem. By leveraging bidirectional Mamba encoders, our approach enables deep ``crosstalk'' between modality-specific embeddings through hidden state propagation, modeling interactions as dynamic sequence transitions rather than static feature overlaps. The framework maintains linear computational complexity, making it scalable to high-dimensional BioLLM embeddings. We further incorporate Gaussian noise injection and Focal Loss to enhance robustness against hard-negative samples. Comprehensive experiments across three interaction categories, RNA-protein, RNA-small molecule, and RNA-RNA demonstrate that CrossLLM-Mamba achieves state-of-the-art performance. On the RPI1460 benchmark, our model attains an MCC of 0.892, surpassing the previous best by 5.2\%. For binding affinity prediction, we achieve Pearson correlations exceeding 0.95 on riboswitch and repeat RNA subtypes. These results establish state-space modeling as a powerful paradigm for multi-modal biological interaction prediction.

Motivation & Objective

• Motivate improved RNA-associated interaction prediction beyond static fusion by modeling dynamic crosstalk between modalities.
• Propose a bidirectional state-space fusion framework (BiMamba) to enable continuous information flow between modality embeddings.
• Maintain linear computational complexity to scale with high-dimensional BioLLM embeddings.
• Improve robustness to hard negatives and class imbalance via Gaussian noise injection and Focal Loss.
• Demonstrate generalization across three interaction categories: RNA–protein, RNA–RNA, and RNA–small molecule.

Proposed method

• Encode proteins with ESM-2, RNA with RiNALMo, and small molecules with MoleBERT to obtain modality-specific embeddings.
• Project embeddings into a shared latent space with linear projection plus Gaussian noise for robustness (X_A = W_A E_A + b_A + N(0, sigma^2), X_B = W_B E_B + b_B + N(0, sigma^2)).
• Apply Bidirectional Mamba (BiMamba) to each modality to capture forward and reverse context and produce X_enc.
• Construct a Cross-Mamba fusion sequence by stacking [X_A_enc, X_B_enc] and process with a BiMamba mixer to model interaction flows (S_mixed).
• Aggregate via global mean pooling and pass through an MLP to predict interaction probability or affinity; use Focal Loss for classification and a composite MSE-Pearson loss for affinity.
• Adopt linear complexity cross-modal fusion to avoid quadratic scaling of cross-attention.

Experimental results

Research questions

• RQ1 Can bidirectional state-space fusion of BioLLM embeddings capture dynamic RNA interaction crosstalk better than static fusion methods?
• RQ2 Does BiMamba-based cross-modal fusion maintain linear scaling with high-dimensional BioLLM embeddings while delivering state-of-the-art performance?
• RQ3 How robust is CrossLLM-Mamba to hard negatives and class imbalance in RNA interaction datasets?
• RQ4 How well does the framework generalize across RNA–protein, RNA–RNA, and RNA–small molecule interaction modalities?

Key findings

• On RNA–protein interaction (RPI1460) CrossLLM-Mamba achieves MCC 0.892 and ACC 0.935, surpassing the previous best MCC by 0.892 vs 0.848 in BioLLMNet.
• For RPI1460, F1 = 0.933, Precision = 0.901, Recall = 0.971, AUC-ROC = 0.957.
• Across RNA–small molecule binding affinity, the model yields Pearson correlations above 0.95 for riboswitches (0.9562) and repeats (0.9521).
• In RNA–RNA plant miRNA–lncRNA transfer tasks, CrossLLM-Mamba achieves up to 75% accuracy in MTR-ATH (training on M. truncatula, testing on A. thaliana), outperforming some baselines in four of six transfer settings.
• Ablation studies show Cross-Mamba fusion significantly outperforms concatenation, bidirectionality improves MCC by 2.7%, and Gaussian noise plus Focal Loss aid generalization and hard-negative handling.