[Paper Review] Feature versus Raw Sequence: Deep Learning Comparative Study on Predicting Pre-miRNA

Jaya Thomas, Sonia Thomas+1 more|arXiv (Cornell University)|Oct 17, 2017

Genomics and Phylogenetic Studies16 references3 citations

TL;DR

This study compares deep learning models using raw miRNA sequences versus engineered features for pre-miRNA prediction. A 6-layer CNN on raw sequences achieved 99.5% accuracy, slightly outperforming a 3-layer DBN on 20 selected features (99.0% accuracy), demonstrating that well-designed raw-sequence models can replace feature-based approaches when sufficient data is available.

ABSTRACT

Should we input known genome sequence features or input sequence itself in deep learning framework? As deep learning more popular in various applications, researchers often come to question whether to generate features or use raw sequences for deep learning. To answer this question, we study the prediction accuracy of precursor miRNA prediction of feature-based deep belief network and sequence-based convolution neural network. Tested on a variant of six-layer convolution neural net and three-layer deep belief network, we find the raw sequence input based convolution neural network model performs similar or slightly better than feature based deep belief networks with best accuracy values of 0.995 and 0.990, respectively. Both the models outperform existing benchmarks models. The results shows us that if provided large enough data, well devised raw sequence based deep learning models can replace feature based deep learning models. However, construction of well behaved deep learning model can be very challenging. In cased features can be easily extracted, feature-based deep learning models may be a better alternative.

Motivation & Objective

• To determine whether raw RNA sequences or pre-extracted sequence features yield better performance in deep learning models for pre-miRNA prediction.
• To evaluate the effectiveness of convolutional neural networks (CNNs) trained directly on raw nucleotide sequences versus deep belief networks (DBNs) using biologically relevant features.
• To compare the performance of these deep learning models against existing benchmark methods such as SVM, random forests, and naive Bayes classifiers.
• To investigate whether end-to-end learning from raw sequences can eliminate the need for manual feature engineering in pre-miRNA classification.

Proposed method

• Trained a 6-layer convolutional neural network (CNN) on raw nucleotide sequences using filters of size 18 with stride 4 and 20 filters, followed by two fully connected layers with 90 and 2 neurons respectively.
• Implemented a second CNN architecture with two convolutional layers (filter sizes 12 and 6, stride 1), followed by max pooling (window size 6, stride 4), and two fully connected layers ending in a 2-neuron output layer.
• Used dropout with a ratio of 0.3 at the output layer to prevent overfitting in both CNN models.
• Constructed a deep belief network (DBN) with three hidden layers (100, 70, 35 neurons) trained on 58 and 20 biologically relevant features including free energy, melting temperature, and base-pairing statistics.
• Performed 20 independent runs for DBN models to account for stochasticity and averaged results for stability.
• Evaluated models using accuracy, F1-score, and compared results against established methods like SVM, MiRANN, and Triplet-SVM.

Experimental results

Research questions

• RQ1 Does a deep learning model trained directly on raw RNA sequences outperform a model using handcrafted sequence features for pre-miRNA prediction?
• RQ2 How do the performance metrics (accuracy, F1-score) of CNNs on raw sequences compare to DBNs on feature sets across different feature subsets?
• RQ3 Can raw-sequence-based deep learning models achieve higher accuracy than traditional machine learning models like SVM and random forests in pre-miRNA classification?
• RQ4 What is the impact of model depth and architecture design on performance when training on limited biological sequence data?

Key findings

• The raw-sequence-based 6-layer CNN achieved the highest accuracy of 0.995, significantly outperforming the feature-based DBN model.
• The DBN model trained on 20 selected features achieved an accuracy of 0.990, slightly higher than the 0.968 accuracy obtained with 58 features.
• Both deep learning models (CNN and DBN) outperformed existing benchmark methods such as SVM, naive Bayes, and random forests in pre-miRNA prediction.
• The CNN model with a two-convolutional-layer architecture (filter sizes 12 and 6, stride 1) followed by max pooling achieved high performance, indicating that deeper architectures can enhance feature extraction.
• Despite the potential of raw-sequence models, constructing well-performing deep learning architectures remains challenging due to data limitations and hyperparameter sensitivity.
• When high-quality, easily extractable features are available, feature-based DBNs may still be a preferable alternative due to their robustness and lower architectural complexity.