[Paper Review] MUST-CNN: A Multilayer Shift-and-Stitch Deep Convolutional Architecture for Sequence-based Protein Structure Prediction

Zeming Lin, Jack Lanchantin+1 more|arXiv (Cornell University)|May 10, 2016

Protein Structure and Dynamics28 references16 citations

TL;DR

MUST-CNN proposes a novel multilayer shift-and-stitch convolutional neural network for end-to-end, per-position protein property prediction from amino acid sequences. By enabling full-resolution, efficient dense prediction through hierarchical shift-and-stitch operations, it achieves state-of-the-art performance on secondary structure and solvent accessibility prediction with faster inference and simpler architecture than prior methods.

ABSTRACT

Predicting protein properties such as solvent accessibility and secondary structure from its primary amino acid sequence is an important task in bioinformatics. Recently, a few deep learning models have surpassed the traditional window based multilayer perceptron. Taking inspiration from the image classification domain we propose a deep convolutional neural network architecture, MUST-CNN, to predict protein properties. This architecture uses a novel multilayer shift-and-stitch (MUST) technique to generate fully dense per-position predictions on protein sequences. Our model is significantly simpler than the state-of-the-art, yet achieves better results. By combining MUST and the efficient convolution operation, we can consider far more parameters while retaining very fast prediction speeds. We beat the state-of-the-art performance on two large protein property prediction datasets.

Motivation & Objective

• To develop a deep learning model that predicts protein properties like secondary structure and solvent accessibility directly from primary amino acid sequences.
• To overcome the limitations of window-based multilayer perceptron (MLP) models, such as slow training and restricted receptive fields.
• To enable full-resolution, per-position labeling on entire protein sequences using a computationally efficient convolutional architecture.
• To eliminate the need for sliding window inference by introducing a novel shift-and-stitch technique that maintains high model capacity and speed.
• To establish a generic, end-to-end framework for sequence-level per-position labeling applicable to diverse biological and NLP tasks.

Proposed method

• Introduces the multilayer shift-and-stitch (MUST) technique to enable full-resolution, dense prediction across entire protein sequences without pooling-induced resolution loss.
• Applies hierarchical convolutional layers with parameter sharing to capture long-range dependencies in protein sequences.
• Uses a shift-and-stitch mechanism to compute scores for all possible positions simultaneously, avoiding repeated forward passes over sliding windows.
• Employs end-to-end training on full-length sequences, allowing the model to learn global context and local patterns jointly.
• Leverages efficient convolution operations combined with the MUST technique to scale model capacity while maintaining fast inference speeds.
• Applies dropout regularization adaptively across different datasets to improve generalization without architectural changes.

Experimental results

Research questions

• RQ1 Can a deep convolutional neural network achieve superior performance on protein property prediction compared to traditional window-based MLP models?
• RQ2 Can the shift-and-stitch technique enable full-resolution, per-position labeling in a computationally efficient manner for long protein sequences?
• RQ3 Does end-to-end training on full sequences outperform models that rely on local windowing and feature engineering?
• RQ4 How does model capacity and inference speed compare between MUST-CNN and existing state-of-the-art models like GSN, LSTM, and CNF?
• RQ5 Can a single, unified architecture generalize across multiple protein property prediction tasks with minimal hyperparameter tuning?

Key findings

• MUST-CNN achieves 89.6% Q3 accuracy on the ssp (3-class secondary structure) task in the CB513 dataset, outperforming previous state-of-the-art methods.
• On the CullPDB dataset, MUST-CNN achieves 68.4% Q8 accuracy, surpassing the prior best result of 67.4% from the LSTM model.
• The model predicts one million amino acids in under 2 seconds, demonstrating exceptional inference speed despite high model capacity.
• The model achieves a 1% improvement in Q3 accuracy over the previous best published result on the 4prot dataset, despite using a significantly simpler architecture.
• The model generalizes well across datasets, with consistent performance across different protein property tasks using only dropout hyperparameter adjustments.
• The shift-and-stitch mechanism enables full-resolution prediction without sacrificing speed, allowing the model to learn long-range dependencies more effectively than windowed approaches.