[Paper Review] Robust Probabilistic Predictive Syntactic Processing

Brian Roark|ArXiv.org|May 9, 2001

Natural Language Processing TechniquesComputer Science85 references17 citations

TL;DR

This paper presents a robust, probabilistic, top-down, left-to-right syntactic parser that incrementally builds fully connected syntactic structures using conditional probability models. It demonstrates significant reductions in perplexity and word error rate (WER) over trigram language models in speech recognition, even with only 5% of the trigram training data, and achieves improved performance on spontaneous speech via EM-based model adaptation.

ABSTRACT

This thesis presents a broad-coverage probabilistic top-down parser, and its application to the problem of language modeling for speech recognition. The parser builds fully connected derivations incrementally, in a single pass from left-to-right across the string. We argue that the parsing approach that we have adopted is well-motivated from a psycholinguistic perspective, as a model that captures probabilistic dependencies between lexical items, as part of the process of building connected syntactic structures. The basic parser and conditional probability models are presented, and empirical results are provided for its parsing accuracy on both newspaper text and spontaneous telephone conversations. Modifications to the probability model are presented that lead to improved performance. A new language model which uses the output of the parser is then defined. Perplexity and word error rate reduction are demonstrated over trigram models, even when the trigram is trained on significantly more data. Interpolation on a word-by-word basis with a trigram model yields additional improvements.

Motivation & Objective

• To develop a broad-coverage, probabilistic, top-down parser that processes input incrementally from left to right.
• To model syntactic dependencies in a way that reflects psycholinguistic processing of natural language.
• To apply the parser to language modeling for speech recognition, improving on standard n-gram models.
• To evaluate the parser’s robustness on noisy, spontaneous speech and grammatically ill-formed input.
• To explore interpolation and EM-based adaptation to enhance performance on low-resource or challenging test sets.

Proposed method

• Uses a probabilistic top-down parsing strategy with left-corner traversal to incrementally construct syntactic derivations.
• Employs conditional probability models over syntactic categories and lexical items to guide parsing decisions.
• Applies a selective left-corner transform to flatten grammar structure while preserving constituent relationships.
• Integrates the parser’s output into a language model for speech recognition, replacing or augmenting n-gram models.
• Uses word-by-word interpolation with trigram models to combine strengths of both approaches.
• Employs EM (Expectation-Maximization) to re-estimate model parameters on test data, improving generalization on spontaneous speech.

Experimental results

Research questions

• RQ1 Can a top-down, left-to-right, probabilistic parser model syntactic dependencies in a way that reflects human sentence processing?
• RQ2 Does a parser-based language model reduce perplexity and word error rate more effectively than standard trigram models?
• RQ3 Can the parser handle spontaneous, noisy, or ungrammatical speech input robustly without pre-processing?
• RQ4 To what extent does interpolation with trigram models improve performance when using limited training data?
• RQ5 Can EM-based adaptation improve the parser’s language model performance on low-resource or out-of-domain speech data?

Key findings

• The parser-based language model reduced perplexity and word error rate (WER) compared to trigram models, even when trained on only 5% of the trigram data.
• On Switchboard test sets, the parser model achieved a WER of 39.0 on sentences of length ≥10, outperforming the lattice trigram model (WER 39.3) on the same subset.
• Interpolation between the parser model and trigram model yielded additional improvements in WER, demonstrating complementary strengths.
• The model showed robustness to ill-formed input, with no significant performance degradation on spontaneous speech, suggesting minimal grammaticality bias.
• EM-based adaptation improved the parser model’s performance on Switchboard, enabling it to surpass the lattice trigram baseline.
• Empirical results suggest that syntactic parsing can access long-range dependencies beyond the reach of standard n-gram models, validating its utility in speech recognition.