[Paper Review] Attention based convolutional neural network for predicting RNA-protein binding sites

Xiaoyong Pan, Junchi Yan|arXiv (Cornell University)|Dec 6, 2017

RNA and protein synthesis mechanisms2 references20 citations

TL;DR

This paper proposes iDeepA, an attention-based convolutional neural network that predicts RNA-protein binding sites using raw RNA sequences. By integrating 1D-CNN with dual-level attention mechanisms—on sequence and feature map dimensions—it enhances feature learning and motif localization, achieving state-of-the-art AUC of 0.921 on CLIP-seq benchmark data, with notable improvements on small datasets.

ABSTRACT

RNA-binding proteins (RBPs) play crucial roles in many biological processes, e.g. gene regulation. Computational identification of RBP binding sites on RNAs are urgently needed. In particular, RBPs bind to RNAs by recognizing sequence motifs. Thus, fast locating those motifs on RNA sequences is crucial and time-efficient for determining whether the RNAs interact with the RBPs or not. In this study, we present an attention based convolutional neural network, iDeepA, to predict RNA-protein binding sites from raw RNA sequences. We first encode RNA sequences into one-hot encoding. Next, we design a deep learning model with a convolutional neural network (CNN) and an attention mechanism, which automatically search for important positions, e.g. binding motifs, to learn discriminant high-level features for predicting RBP binding sites. We evaluate iDeepA on publicly gold-standard RBP binding sites derived from CLIP-seq data. The results demonstrate iDeepA achieves comparable performance with other state-of-the-art methods.

Motivation & Objective

• To develop a deep learning model that accurately predicts RNA-protein binding sites from raw RNA sequences.
• To improve model interpretability and performance by incorporating attention mechanisms to focus on biologically relevant sequence motifs.
• To address the limitations of existing methods on small training datasets by enhancing feature learning through attention.
• To evaluate the model on a large-scale, gold-standard CLIP-seq dataset across 24 RBPs.

Proposed method

• RNA sequences are encoded into one-hot matrices representing nucleotide presence (A, C, G, U).
• A 1D-CNN processes the one-hot matrix, applying convolution, ReLU activation, and max-pooling to extract hierarchical features.
• Two attention mechanisms are applied: one on the sequence dimension (over time steps) and one on the feature map dimension (via transposed hidden states).
• Attention weights are computed via a feedforward network producing soft attention scores, where $ \alpha_t = \frac{\exp(e_t)}{\sum_i \exp(e_i)} $, and the context vector is $ O = \sum_t h_t \alpha_t $.
• The outputs from the CNN and both attention layers are concatenated and passed through two fully connected layers with sigmoid activation for binary classification.
• Model training uses categorical cross-entropy loss with RMSProp optimizer over 30 epochs.

Experimental results

Research questions

• RQ1 Can an attention-augmented CNN improve the identification of biologically relevant RNA-protein binding motifs compared to standard CNNs?
• RQ2 Does the attention mechanism enhance performance on RBPs with limited training data?
• RQ3 How does iDeepA compare to state-of-the-art methods like DeepBind and GraphProt in terms of AUC across diverse RBPs?
• RQ4 Can the attention mechanism localize functionally important subsequences in RNA sequences for interpretability?

Key findings

• iDeepA achieves an average AUC of 0.921 across 24 RBP experiments, matching the performance of DeepBind and outperforming GraphProt (0.887) and deepnet-rbp (0.902).
• On the RBP C17ORF85 with only 4,000 training samples, iDeepA achieves an AUC of 0.839, an 11% improvement over DeepBind’s 0.755.
• The attention mechanism enhances learning efficiency on small datasets, suggesting better generalization and focus on critical motifs.
• MILCNN underperforms due to potential disruption of binding sites when sequences are split into fixed-length subsequences.
• The model shows comparable performance to state-of-the-art methods, indicating robustness and scalability on diverse RBP datasets.