[Paper Review] TransMA: an explainable multi-modal deep learning model for predicting properties of ionizable lipid nanoparticles in mRNA delivery

Kun Wu, Zixu Wang+5 more|arXiv (Cornell University)|Jul 8, 2024

RNA Interference and Gene DeliveryBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

TransMA is a novel explainable multi-modal deep learning model that predicts transfection efficiency of ionizable lipid nanoparticles (LNPs) by fusing 3D spatial and 1D sequential molecular features using a molecule 3D Transformer and molecule Mamba, respectively, with a novel mol-attention mechanism for feature alignment. It achieves state-of-the-art performance on the largest existing LNP dataset and identifies key atoms responsible for transfection efficiency changes, especially in transfection cliff pairs.

ABSTRACT

As the primary mRNA delivery vehicles, ionizable lipid nanoparticles (LNPs) exhibit excellent safety, high transfection efficiency, and strong immune response induction. However, the screening process for LNPs is time-consuming and costly. To expedite the identification of high-transfection-efficiency mRNA drug delivery systems, we propose an explainable LNPs transfection efficiency prediction model, called TransMA. TransMA employs a multi-modal molecular structure fusion architecture, wherein the fine-grained atomic spatial relationship extractor named molecule 3D Transformer captures three-dimensional spatial features of the molecule, and the coarse-grained atomic sequence extractor named molecule Mamba captures one-dimensional molecular features. We design the mol-attention mechanism block, enabling it to align coarse and fine-grained atomic features and captures relationships between atomic spatial and sequential structures. TransMA achieves state-of-the-art performance in predicting transfection efficiency using the scaffold and cliff data splitting methods on the current largest LNPs dataset, including Hela and RAW cell lines. Moreover, we find that TransMA captures the relationship between subtle structural changes and significant transfection efficiency variations, providing valuable insights for LNPs design. Additionally, TransMA's predictions on external transfection efficiency data maintain a consistent order with actual transfection efficiencies, demonstrating its robust generalization capability. The code, model and data are made publicly available at https://github.com/wklix/TransMA/tree/master. We hope that high-accuracy transfection prediction models in the future can aid in LNPs design and initial screening, thereby assisting in accelerating the mRNA design process.

Motivation & Objective

• To accelerate the discovery of high-efficiency mRNA delivery systems by enabling accurate, automated prediction of ionizable lipid nanoparticle (LNP) transfection efficiency.
• To address the limitations of existing models by integrating multi-modal molecular features—3D spatial and 1D sequential representations—into a unified architecture.
• To enhance model interpretability by identifying specific atoms that drive significant changes in transfection efficiency, particularly in transfection cliff pairs.
• To improve generalization by validating predictions on external, untrained datasets across different cell lines and formulations.

Proposed method

• TransMA employs a multi-modal molecular structure fusion architecture combining a molecule 3D Transformer to extract fine-grained 3D spatial features of the molecular structure.
• It uses a molecule Mamba model to capture coarse-grained, one-dimensional molecular sequence features.
• A novel mol-attention mechanism block is designed to align and fuse the 3D spatial and 1D sequential features by computing attention between atomic representations.
• The model is trained on the largest current LNP dataset using scaffold and cliff data splitting protocols to evaluate robustness and generalization.
• Interpretability is achieved by analyzing attention scores from the mol-attention block to identify atoms most influential in transfection efficiency changes.
• External validation is performed on a dataset of 15 LNPs not seen during training, with predictions compared to reported transfection efficiencies.

Figure 1: The model takes as input multimodal structural information of ionizable lipids, including three-dimensional structural details: atomic type sequences, three-dimensional coordinates, distance matrices, bond type matrices, and one-dimensional SMILES representation molecular structural inform

Experimental results

Research questions

• RQ1 Can a multi-modal deep learning model that fuses 3D spatial and 1D sequential molecular features outperform existing models in predicting LNP transfection efficiency?
• RQ2 To what extent can attention mechanisms in the model identify specific atoms responsible for large differences in transfection efficiency, particularly in transfection cliff pairs?
• RQ3 Does the model generalize well to external datasets not used in training, especially across different cell lines and formulations?
• RQ4 How do subtle structural changes—such as a single carbon or nitrogen atom variation—translate into significant transfection efficiency differences, and can the model detect these?

Key findings

• TransMA achieves state-of-the-art performance in predicting transfection efficiency on the largest existing LNP dataset, outperforming prior models such as AGILE and TransLNP under both scaffold and cliff data splitting protocols.
• The model maintains high accuracy in predicting transfection efficiency on external datasets without fine-tuning, with predicted values maintaining consistent ordering relative to true values across different cell lines and formulations.
• In transfection cliff pairs, atoms with high attention scores (e.g., 0.84 and 0.86) correspond to the structural differences (one C and one N atom), confirming the model’s ability to identify key atoms driving efficiency changes.
• The multiplicative transfection difference between two molecules in a cliff pair was up to 10^3.89 times, and TransMA correctly predicted the higher-efficiency molecule, demonstrating robustness to extreme structural changes.
• The mol-attention mechanism enables effective fusion of multi-modal features, revealing atom-level structure-transfection relationships critical for rational LNP design.
• Despite high performance, prediction accuracy remains limited by dataset scarcity and the inherent challenge of transfection cliffs, where small structural changes cause large efficiency shifts.

Figure 2: Left: The architecture of the molecule 3D Transformer adopts a pre-training and fine-tuning approach. The pre-training task involves masked language modeling and noise coordinate prediction. Fine-tuning is conducted using molecular prediction heads for transfection efficiency prediction. R