[Paper Review] Drug Synergistic Combinations Predictions via Large-Scale Pre-Training and Graph Structure Learning

Zhigang Hu, Qinze Yu+6 more|arXiv (Cornell University)|Jan 14, 2023

Computational Drug Discovery Methods4 citations

TL;DR

This paper proposes a novel deep learning framework that leverages large-scale pre-trained models and graph structure learning to predict synergistic drug combinations. By integrating multi-modal drug, protein, and disease representations with a message-passing graph and pseudo-relation learning, the model achieves state-of-the-art performance, including a 10% improvement on an independent AstraZeneca test set and over 15% higher AUROC than the second-best model on unseen drugs.

ABSTRACT

Drug combination therapy is a well-established strategy for disease treatment with better effectiveness and less safety degradation. However, identifying novel drug combinations through wet-lab experiments is resource intensive due to the vast combinatorial search space. Recently, computational approaches, specifically deep learning models have emerged as an efficient way to discover synergistic combinations. While previous methods reported fair performance, their models usually do not take advantage of multi-modal data and they are unable to handle new drugs or cell lines. In this study, we collected data from various datasets covering various drug-related aspects. Then, we take advantage of large-scale pre-training models to generate informative representations and features for drugs, proteins, and diseases. Based on that, a message-passing graph is built on top to propagate information together with graph structure learning flexibility. This is first introduced in the biological networks and enables us to generate pseudo-relations in the graph. Our framework achieves state-of-the-art results in comparison with other deep learning-based methods on synergistic prediction benchmark datasets. We are also capable of inferencing new drug combination data in a test on an independent set released by AstraZeneca, where 10% of improvement over previous methods is observed. In addition, we're robust against unseen drugs and surpass almost 15% AU ROC compared to the second-best model. We believe our framework contributes to both the future wet-lab discovery of novel drugs and the building of promising guidance for precise combination medicine.

Motivation & Objective

• To address the challenge of identifying synergistic drug combinations efficiently, avoiding costly wet-lab experiments.
• To overcome limitations of prior models that lack generalization to new drugs and cell lines.
• To integrate multi-modal biological data (drugs, proteins, diseases) into a unified predictive framework.
• To develop a robust, generalizable model capable of handling domain shift and unseen drug combinations.
• To enable accurate inference on novel drug combinations beyond the training distribution.

Proposed method

• Utilizes large-scale pre-training to generate rich, informative representations for drugs, proteins, and diseases from diverse data sources.
• Constructs a message-passing graph that propagates information across nodes while enabling flexible graph structure learning to discover pseudo-relations.
• Introduces a self-training strategy to refine predictions and improve model generalization.
• Employs dual predictive heads for drug-drug interaction (DDI) and drug-target interaction (DTI) to enhance feature learning.
• Applies multi-head graph attention networks (GAT) with 4, 8, and 12 attention heads for effective graph representation learning.
• Uses a multi-layer perceptron (MLP) with 3072, 768, and 128 dimensions for final synergistic effect classification.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework effectively predict synergistic drug combinations using multi-modal biological data?
• RQ2 How well does the model generalize to novel drugs and cell lines not seen during training?
• RQ3 Does incorporating pre-trained representations and graph structure learning improve prediction performance compared to existing methods?
• RQ4 To what extent does the self-training strategy and dual predictive heads enhance model robustness and accuracy?
• RQ5 How does the model perform on independent, real-world test sets such as the AstraZeneca dataset?

Key findings

• The proposed model achieves state-of-the-art performance on benchmark datasets, outperforming existing deep learning-based methods in AUROC, AUPRC, and F1-Score.
• On the independent AstraZeneca test set, the model shows a 10% improvement in performance over previous methods.
• The model demonstrates robustness to unseen drugs, surpassing the second-best model by over 15% in AUROC on the unseen drug evaluation.
• Ablation studies confirm that both the self-training strategy and the DDI/DTI predictive modules significantly improve model performance.
• The use of rich, pre-trained embeddings is critical—replacing them with one-hot encoding or fingerprints leads to substantial performance drops.
• The model maintains stable performance across varying cell line frequencies, indicating strong generalization and resistance to data distribution shifts.