[Paper Review] An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities

Andreas Stolcke|ArXiv.org|Nov 28, 1994

Natural Language Processing TechniquesComputer Science35 references288 citations

TL;DR

This paper presents an efficient, Earley-based probabilistic context-free parsing algorithm that computes prefix probabilities, substring generation probabilities, Viterbi parses, and expected production counts in a single left-to-right pass. It extends Earley's top-down control structure to handle sparse grammars efficiently without requiring grammar normalization, enabling incremental probability computation and robust parsing for ungrammatical inputs.

ABSTRACT

We describe an extension of Earley's parser for stochastic context-free grammars that computes the following quantities given a stochastic context-free grammar and an input string: a) probabilities of successive prefixes being generated by the grammar; b) probabilities of substrings being generated by the nonterminals, including the entire string being generated by the grammar; c) most likely (Viterbi) parse of the string; d) posterior expected number of applications of each grammar production, as required for reestimating rule probabilities. (a) and (b) are computed incrementally in a single left-to-right pass over the input. Our algorithm compares favorably to standard bottom-up parsing methods for SCFGs in that it works efficiently on sparse grammars by making use of Earley's top-down control structure. It can process any context-free rule format without conversion to some normal form, and combines computations for (a) through (d) in a single algorithm. Finally, the algorithm has simple extensions for processing partially bracketed inputs, and for finding partial parses and their likelihoods on ungrammatical inputs.

Motivation & Objective

• To develop a parsing algorithm that computes prefix probabilities incrementally during left-to-right input processing.
• To support efficient parsing on sparse grammars without requiring conversion to normal form.
• To unify computation of multiple probabilistic parsing quantities—substring probabilities, Viterbi parses, and expected production counts—in a single algorithm.
• To enable robust parsing for partially bracketed or ungrammatical inputs through simple extensions.

Proposed method

• Extends Earley's parsing framework with probabilistic inference using forward and inner probabilities.
• Computes forward probabilities incrementally as the input is scanned left to right.
• Uses prediction and completion steps with probability propagation to maintain state probabilities.
• Handles recursion via prediction and completion loops, including for epsilon (null) productions.
• Computes expected production counts via backward pass using outer probabilities.
• Supports robust parsing by seeding the chart with partial bracketing and assembling partial parses.

Experimental results

Research questions

• RQ1 How can prefix probabilities be computed incrementally during left-to-right parsing of a stochastic context-free grammar?
• RQ2 Can a top-down Earley-style algorithm efficiently handle sparse grammars without grammar normalization?
• RQ3 How can Viterbi parses and expected production counts be computed within the same parsing framework?
• RQ4 What extensions allow robust parsing of ungrammatical or partially bracketed inputs?

Key findings

• The algorithm computes prefix probabilities and substring generation probabilities in a single left-to-right pass with optimal complexity for sparse grammars.
• It achieves performance comparable to bottom-up methods on fully parameterized grammars while outperforming them on sparse grammars due to top-down control.
• The method supports exact computation of Viterbi parses and expected production counts without requiring grammar transformation.
• Extensions allow parsing of partially bracketed inputs and identification of partial parses with their likelihoods.
• The algorithm avoids the need for grammar conversion to normal form, preserving original rule structure.
• Empirical results show the algorithm is efficient and scalable, with complexity scaling linearly with input length and sub-cubic with grammar size.