[Paper Review] RNA Secondary Structure Prediction By Learning Unrolled Algorithms

Xinshi Chen, Yu Li+3 more|arXiv (Cornell University)|Feb 13, 2020

RNA and protein synthesis mechanisms31 references37 citations

TL;DR

E2Efold learns to predict RNA base-pairing matrices end-to-end by embedding hard structural constraints via an unrolled constrained-optimization post-processing network, achieving superior accuracy (especially for pseudoknots) and competitive inference speed.

ABSTRACT

In this paper, we propose an end-to-end deep learning model, called E2Efold, for RNA secondary structure prediction which can effectively take into account the inherent constraints in the problem. The key idea of E2Efold is to directly predict the RNA base-pairing matrix, and use an unrolled algorithm for constrained programming as the template for deep architectures to enforce constraints. With comprehensive experiments on benchmark datasets, we demonstrate the superior performance of E2Efold: it predicts significantly better structures compared to previous SOTA (especially for pseudoknotted structures), while being as efficient as the fastest algorithms in terms of inference time.

Motivation & Objective

• Motivate end-to-end learning for RNA secondary structure prediction while respecting hard structural constraints.
• Avoid nested-structure limitation by predicting a base-pairing matrix directly and enforcing constraints via an unrolled post-processing algorithm.
• Couple a transformer-based Deep Score Network with a differentiable Post-Processing Network trained jointly.
• Demonstrate superior performance on benchmark datasets, including pseudoknotted structures, and compare with state-of-the-art methods.

Proposed method

• Predict base-pairing scores with a Deep Score Network that outputs an LxL symmetric score matrix U_theta(x).
• Enforce RNA structure constraints during post-processing via a Post-Processing Network derived from an unrolled constrained optimization algorithm.
• Formulate post-processing as a convex-relaxed optimization over A in [0,1] with symmetry and hard constraints, using an A = T(hatA) transform to encode feasibility.
• Unroll the optimization into PP_phi with learnable hyperparameters, enabling end-to-end training alongside U_theta.
• Directly optimize differentiable surrogates of F1 (precision/recall) to improve base-pair prediction quality.
• Pre-train with logistic regression loss, then jointly train U_theta and PP_phi to maximize trajectory-based -F1 losses.

Experimental results

Research questions

• RQ1 Can an end-to-end model predict RNA secondary structures while inherently satisfying hard structural constraints including pseudoknots?
• RQ2 Does integrating an unrolled constrained-optimization post-processing step during training improve accuracy and efficiency compared to decoupled setups?
• RQ3 How does E2Efold perform on benchmark datasets relative to state-of-the-art methods, particularly for pseudoknotted structures?

Key findings

• E2Efold achieves superior F1 scores compared to SOTA on benchmark datasets, including strong pseudoknot handling.
• On RNAstralign, E2Efold delivers higher accuracy and maintains fast inference times comparable to LinearFold.
• Across benchmarks, E2Efold improves precision and recall, with notable gains in F1 score.
• The integrated end-to-end training with a differentiable unrolled post-processing step yields better performance than post-processing-only variants.
• Pseudoknot-containing predictions are improved, with E2Efold matching or exceeding baselines that explicitly handle pseudoknots.