[Paper Review] Global Thresholding and Multiple Pass Parsing

Joshua Goodman|ArXiv.org|Aug 13, 1997

Blind Source Separation Techniques7 references47 citations

TL;DR

This paper introduces three novel parsing optimization techniques—beam thresholding with priors, global thresholding, and multiple-pass parsing—combined with an automated parameter search algorithm, achieving up to 30 times faster parsing than traditional beam search while maintaining identical performance levels on PCFGs. The methods leverage probabilistic grammar structures and global information to drastically reduce search space without sacrificing accuracy.

ABSTRACT

We present a variation on classic beam thresholding techniques that is up to an order of magnitude faster than the traditional method, at the same performance level. We also present a new thresholding technique, global thresholding, which, combined with the new beam thresholding, gives an additional factor of two improvement, and a novel technique, multiple pass parsing, that can be combined with the others to yield yet another 50% improvement. We use a new search algorithm to simultaneously optimize the thresholding parameters of the various algorithms.

Motivation & Objective

• To address the performance bottleneck in statistical parsing caused by exponential growth in possible nonterminals during CKY chart parsing.
• To improve parsing speed without degrading precision or recall by introducing new thresholding and parsing strategies.
• To optimize multiple thresholding parameters simultaneously using a gradient descent-based algorithm for maximum efficiency.
• To extend the applicability of thresholding techniques to more complex formalisms like SBTG and STAG.

Proposed method

• Introduces beam thresholding with priors, which incorporates the prior probability of nonterminals being in the correct parse to improve pruning decisions beyond intra-cell probability alone.
• Proposes global thresholding, which uses the likelihood of a nonterminal being part of a globally probable parse across the entire sentence to guide pruning across all chart cells.
• Develops multiple-pass parsing, where a fast, simplified grammar is used in the first pass to eliminate unlikely constituents, followed by a second, more accurate and slower pass on a pruned search space.
• Employs a gradient descent-based optimization algorithm to simultaneously tune thresholding parameters across beam, global, and multiple-pass techniques for maximal speedup.
• Applies the techniques to CKY chart parsing of PCFGs, using inside-outside probabilities and entropy as performance metrics.
• Validates the methods on a held-out corpus of 31 sentences, measuring precision, recall, and entropy to assess performance and speed trade-offs.

Experimental results

Research questions

• RQ1 Can incorporating prior probabilities of nonterminals into beam thresholding significantly improve parsing speed without reducing accuracy?
• RQ2 Can global thresholding, which uses sentence-level probability information, outperform cell-local beam thresholding in terms of speed and accuracy?
• RQ3 Does multiple-pass parsing, using a fast first pass to prune the search space, lead to measurable speedups in the second, more accurate pass?
• RQ4 Can an automated parameter search algorithm effectively optimize multiple thresholding parameters in a high-dimensional space?
• RQ5 To what extent can these techniques be generalized to other probabilistic formalisms such as SBTG and STAG?

Key findings

• Beam thresholding with priors alone achieves nearly an order of magnitude speedup over traditional beam thresholding while maintaining the same performance level.
• Global thresholding improves efficiency by up to threefold compared to the new beam thresholding method, with typical gains around 50%.
• Combining global thresholding and beam thresholding yields a two- to threefold speedup over beam thresholding alone.
• Multiple-pass parsing provides an additional 50% improvement in speed, even after accounting for the cost of two parsing passes.
• When all three thresholding techniques and the automated parameter search are combined, the parser runs an estimated 30 times faster than traditional beam search with no loss in performance.
• The automated parameter optimization algorithm achieved a factor of two speedup with no performance loss, demonstrating strong practical utility in real-world grammar applications.