[Paper Review] DeepQA: Improving the estimation of single protein model quality with deep belief networks

Renzhi Cao, Debswapna Bhattacharya+2 more|arXiv (Cornell University)|Jul 15, 2016

Protein Structure and Dynamics46 references4 citations

TL;DR

DeepQA proposes a deep belief network (DBN)-based method for single protein model quality assessment, integrating diverse features like energy, physicochemical properties, and structural information to improve quality estimation. It outperforms SVMs and standard neural networks, achieving state-of-the-art performance on the CASP11 dataset and excelling in selecting high-quality models from large pools of low-quality ab initio models.

ABSTRACT

Protein quality assessment (QA) by ranking and selecting protein models has long been viewed as one of the major challenges for protein tertiary structure prediction. Especially, estimating the quality of a single protein model, which is important for selecting a few good models out of a large model pool consisting of mostly low-quality models, is still a largely unsolved problem. We introduce a novel single-model quality assessment method DeepQA based on deep belief network that utilizes a number of selected features describing the quality of a model from different perspectives, such as energy, physio-chemical characteristics, and structural information. The deep belief network is trained on several large datasets consisting of models from the Critical Assessment of Protein Structure Prediction (CASP) experiments, several publicly available datasets, and models generated by our in-house ab initio method. Our experiment demonstrate that deep belief network has better performance compared to Support Vector Machines and Neural Networks on the protein model quality assessment problem, and our method DeepQA achieves the state-of-the-art performance on CASP11 dataset. It also outperformed two well-established methods in selecting good outlier models from a large set of models of mostly low quality generated by ab initio modeling methods. DeepQA is a useful tool for protein single model quality assessment and protein structure prediction. The source code, executable, document and training/test datasets of DeepQA for Linux is freely available to non-commercial users at http://cactus.rnet.missouri.edu/DeepQA/.

Motivation & Objective

• To address the persistent challenge of estimating the quality of individual protein models in large, heterogeneous model pools.
• To improve the accuracy of protein structure prediction by enabling better selection of high-quality models from low-quality ones.
• To develop a machine learning framework that integrates multi-faceted features (energy, physicochemical, structural) for robust quality estimation.
• To demonstrate superior performance of deep belief networks over traditional models like SVMs and feedforward neural networks in single-model quality assessment.
• To provide a freely available, non-commercial tool for researchers to enhance protein structure prediction workflows.

Proposed method

• The method employs a deep belief network (DBN) trained on multiple datasets, including CASP experiments, public protein model collections, and in-house ab initio models.
• Input features include energy terms, physicochemical properties, and structural descriptors to represent model quality from multiple perspectives.
• The DBN is pre-trained using greedy layer-wise unsupervised learning on visible and hidden layers before fine-tuning with supervised learning.
• The model is fine-tuned using labeled data where model quality is assessed via metrics such as GDT_TS or RMSD to the native structure.
• Feature engineering selects relevant, non-reduundant descriptors to improve generalization and reduce noise in the input space.
• The final model outputs a quality score predicting the likelihood of a model being close to the native structure.

Experimental results

Research questions

• RQ1 Can a deep belief network effectively integrate multi-source features to improve single protein model quality estimation?
• RQ2 How does the performance of a DBN compare to traditional models like SVMs and feedforward neural networks in protein quality assessment?
• RQ3 To what extent can DeepQA identify high-quality models from large pools dominated by low-quality ab initio models?
• RQ4 Does the DBN-based approach generalize across diverse datasets, including CASP and public model collections?
• RQ5 Can the method achieve state-of-the-art performance on benchmark datasets like CASP11?

Key findings

• DeepQA achieved state-of-the-art performance on the CASP11 dataset, outperforming existing methods in single-model quality assessment.
• The deep belief network significantly outperformed both support vector machines and standard neural networks in ranking model quality.
• DeepQA demonstrated superior capability in selecting high-quality outlier models from large sets of low-quality ab initio models.
• The integration of diverse features—energy, physicochemical, and structural—enhanced the model’s predictive accuracy and robustness.
• The method showed strong generalization across multiple datasets, including CASP experiments and public model repositories.
• The source code, executables, documentation, and training/test datasets are publicly available for non-commercial use.