[Paper Review] LinearPartition: Linear-Time Approximation of RNA Folding Partition Function and Base Pairing Probabilities

He Zhang, Liang Zhang+2 more|arXiv (Cornell University)|Dec 31, 2019

RNA and protein synthesis mechanisms31 references4 citations

TL;DR

LinearPartition is a linear-time approximation algorithm for computing RNA folding partition functions and base pairing probabilities, using beam search to prune low-probability structures. It achieves orders-of-magnitude speedup over classical methods like Vienna RNAfold—reducing a 2.5-day computation to 1.3 minutes on a 32,753 nt sequence—while improving correlation with ground truth base pairing probabilities and enhancing accuracy on long-distance pairs and large RNA families.

ABSTRACT

RNA secondary structure prediction is widely used to understand RNA function. Recently, there has been a shift away from the classical minimum free energy (MFE) methods to partition function-based methods that account for folding ensembles and can therefore estimate structure and base pair probabilities. However, the classical partition function algorithm scales cubically with sequence length, and is therefore a slow calculation for long sequences. This slowness is even more severe than cubic-time MFE-based methods due to a larger constant factor in runtime. Inspired by the success of our recently proposed LinearFold algorithm that predicts the approximate MFE structure in linear time, we design a similar linear-time heuristic algorithm, LinearPartition, to approximate the partition function and base pairing probabilities, which is shown to be orders of magnitude faster than Vienna RNAfold and CONTRAfold (e.g., 2.5 days vs. 1.3 minutes on a sequence with length 32,753 nt). More interestingly, the resulting base pairing probabilities are even better correlated with the ground truth structures. LinearPartition also leads to a small accuracy improvement when used for downstream structure prediction on families with the longest length sequences (16S and 23S rRNA), as well as a substantial improvement on long-distance base pairs (500+ nt apart).

Motivation & Objective

• To address the cubic-time bottleneck of classical partition function algorithms for long RNA sequences.
• To develop a scalable method that approximates base pairing probabilities without sacrificing accuracy for long RNAs.
• To improve runtime efficiency for downstream applications such as MEA, ThreshKnot, and stochastic sampling.
• To explore whether beam pruning in a left-to-right dynamic programming framework enhances both speed and accuracy.
• To enable practical computation of partition functions and base pairing probabilities on full-length mRNAs and ribosomal RNAs.

Proposed method

• LinearPartition uses a left-to-right dynamic programming approach with beam search pruning to limit the number of active states to a fixed beam size $b$.
• It approximates the partition function by summing over only the $O(nb^2)$ most probable structures within the beam, rather than all $O(n^3)$ possible structures.
• The algorithm maintains a set of partial structures at each position, retaining only the top $b$ states based on their cumulative partition function values.
• Unlike LinearFold, which uses $k$-best parsing to reduce $O(nb^2)$ to $O(nb\log b)$, LinearPartition computes the full $O(nb^2)$ sum to preserve accuracy in partition function estimation.
• The method is designed to be exact when beam size $b$ is sufficiently large, and becomes increasingly accurate as $b$ increases.
• It leverages the observation that base pairing probabilities grow linearly with sequence length under reasonable thresholds, justifying the $O(n)$ approximation.

Experimental results

Research questions

• RQ1 Can a linear-time heuristic approximation of the RNA partition function maintain high accuracy in base pairing probability estimation?
• RQ2 Does beam search pruning in a left-to-right dynamic programming framework improve both speed and accuracy compared to exact cubic-time methods?
• RQ3 How does the approximation quality of LinearPartition compare to Vienna RNAfold in terms of correlation with experimental ground truth structures?
• RQ4 Can LinearPartition improve downstream structure prediction accuracy, especially for long RNAs and long-distance base pairs?
• RQ5 What is the impact of beam size on the accuracy and runtime of the partition function approximation?

Key findings

• LinearPartition reduces runtime from $O(n^3)$ to $O(nb^2)$, achieving a 1,200-fold speedup on a 32,753 nt sequence (2.5 days vs. 1.3 minutes).
• The base pairing probabilities from LinearPartition show higher correlation with experimental ground truth structures than those from Vienna RNAfold.
• On 16S and 23S rRNA families, LinearPartition improves accuracy in downstream MEA structure prediction compared to RNAfold.
• For long-distance base pairs (>500 nt apart), LinearPartition achieves substantial accuracy gains over classical methods.
• With beam size $b=100$, the root mean square deviation (RMSD) of predicted base pairing probabilities is below 0.005, indicating high approximation fidelity.
• Increasing beam size from $b=50$ to $b=100$ improves PPV and sensitivity, which stabilize beyond $b=100$, though some families (e.g., 16S rRNA) show diminishing returns or slight degradation at higher $b$.