[Paper Review] Comparison of different versions of SignalP and TargetP for diatom plastid protein predictions with ASAFind

Ansgar Gruber, Cedar McKay+2 more|arXiv (Cornell University)|Mar 4, 2023

Machine Learning in Bioinformatics4 citations

TL;DR

This study evaluates SignalP 5.0 and TargetP 2.0 against earlier versions for predicting signal peptides in diatom plastid-targeted proteins using the ASAFind pipeline. Although SignalP 5.0 shows reduced sensitivity in detecting signal peptides compared to prior versions, its integration with ASAFind maintains high prediction performance; TargetP 2.0 outperforms other tools in identifying diatom signal and mitochondrial transit peptides, making it a superior input for ASAFind predictions.

ABSTRACT

Plastid targeted proteins of diatoms and related algae can be predicted with high sensitivity and specificity using the ASAFind method published in 2015. ASAFind predictions rely on SignalP predictions of endoplasmic reticulum (ER) targeting signal peptides. Recently (in 2019), a new version of SignalP was released, SignalP 5.0. We tested the ability of SignalP 5.0 to recognize signal peptides of nucleus-encoded, plastid-targeted diatom pre-proteins, and to identify the signal peptide cleavage site. The results were compared to manual predictions of the characteristic cleavage site motif, and to previous versions of SignalP. SignalP 5.0 is less sensitive than the previous versions of SignalP in this specific task, and also in the detection of signal peptides of non-plastid proteins in diatoms. However, in combination with ASAFind, the resulting prediction performance for plastid proteins is high. In addition, we tested the multi-location prediction tool TargetP for its suitability to provide signal peptide information to ASAFind. The newest version, TargetP 2.0, had the highest prediction performances for diatom signal peptides and mitochondrial transit peptides compared to other versions of SignalP and TargetP, thus it provides a good basis for ASAFind predictions.

Motivation & Objective

• To assess the performance of SignalP 5.0 and previous SignalP versions in detecting signal peptides of nucleus-encoded, plastid-targeted diatom pre-proteins.
• To evaluate whether SignalP 5.0 improves or degrades sensitivity and specificity in signal peptide recognition compared to earlier versions.
• To test the suitability of TargetP versions, especially TargetP 2.0, as a source of signal peptide information for the ASAFind pipeline.
• To determine the optimal combination of signal peptide prediction tools with ASAFind for high-accuracy plastid protein prediction in diatoms.
• To compare manual cleavage site motif predictions with computational tool outputs for validation.

Proposed method

• The study uses ASAFind, a method published in 2015 that predicts plastid-targeted proteins based on signal peptide detection via SignalP.
• SignalP 5.0 and earlier versions (SignalP 4.1 and 3.0) are applied to predict signal peptides in diatom proteomes.
• TargetP 2.0 and earlier versions are tested for their ability to predict signal peptides and mitochondrial transit peptides in diatoms.
• Manual curation of known cleavage site motifs in diatom pre-proteins serves as a gold standard for evaluating tool accuracy.
• Performance is assessed using sensitivity, specificity, and agreement with manually identified cleavage sites.
• Raw data from predictions are compared across tools and used to evaluate the impact of tool choice on downstream ASAFind plastid protein prediction accuracy.

Experimental results

Research questions

• RQ1 Does SignalP 5.0 maintain or improve sensitivity in detecting signal peptides of nucleus-encoded, plastid-targeted diatom pre-proteins compared to previous SignalP versions?
• RQ2 How does the performance of SignalP 5.0 in identifying signal peptide cleavage sites compare to manual predictions of the characteristic motif in diatom proteins?
• RQ3 Can TargetP 2.0 outperform SignalP versions in detecting signal peptides and mitochondrial transit peptides in diatom proteomes?
• RQ4 What is the impact of using different signal peptide prediction tools on the overall accuracy of ASAFind in predicting plastid-targeted proteins in diatoms?
• RQ5 Is there a significant performance degradation in SignalP 5.0 for non-plastid proteins in diatoms compared to earlier versions?

Key findings

• SignalP 5.0 exhibits lower sensitivity than previous SignalP versions in detecting signal peptides of nucleus-encoded, plastid-targeted diatom pre-proteins.
• SignalP 5.0 shows reduced performance in identifying signal peptides in non-plastid proteins in diatoms compared to earlier versions.
• Despite lower sensitivity, SignalP 5.0 maintains high prediction performance when used in combination with ASAFind for plastid protein prediction.
• TargetP 2.0 achieves the highest prediction performance among all tested versions for both diatom signal peptides and mitochondrial transit peptides.
• TargetP 2.0 provides a more reliable signal peptide input for ASAFind than any tested version of SignalP, making it the preferred tool for diatom plastid protein prediction.
• Manual predictions of cleavage site motifs serve as a robust benchmark, confirming that SignalP 5.0 underperforms relative to earlier versions in this specific task.