[Paper Review] Text Chunking using Transformation-Based Learning

Lance Ramshaw, Mitchell P. Marcus|ArXiv.org|May 23, 1995

Natural Language Processing TechniquesComputer Science9 references472 citations

TL;DR

This paper applies transformation-based learning to text chunking by framing it as a tagging problem, where chunk boundaries are encoded in tags attached to words. Using part-of-speech tags and learned transformation rules, the system achieves 92% precision and recall for baseNP chunks and 88% for more complex partitioning chunks on Penn Treebank data, demonstrating the effectiveness of the approach for shallow parsing and information extraction.

ABSTRACT

Eric Brill introduced transformation-based learning and showed that it can do part-of-speech tagging with fairly high accuracy. The same method can be applied at a higher level of textual interpretation for locating chunks in the tagged text, including non-recursive ``baseNP'' chunks. For this purpose, it is convenient to view chunking as a tagging problem by encoding the chunk structure in new tags attached to each word. In automatic tests using Treebank-derived data, this technique achieved recall and precision rates of roughly 92% for baseNP chunks and 88% for somewhat more complex chunks that partition the sentence. Some interesting adaptations to the transformation-based learning approach are also suggested by this application.

Motivation & Objective

• To adapt transformation-based learning—previously used for part-of-speech tagging—to the task of text chunking.
• To evaluate whether transformation rules can effectively learn chunk boundary decisions using local word, POS, and chunk tag contexts.
• To explore how rule-based learning can improve upon baseline tagging heuristics for identifying non-recursive noun phrases and sentence partitions.
• To assess the impact of lexical templates and rule learning on performance across different chunking types.
• To investigate the feasibility of extending transformation-based learning to higher-level syntactic structures beyond simple chunks.

Proposed method

• Represent chunk structure as additional tags attached to each word, transforming chunking into a tagging problem.
• Use Brill’s transformation-based learning framework to automatically induce a sequence of context-sensitive rules that correct initial chunk tag predictions.
• Train on Treebank-derived data, using part-of-speech tags and baseline chunk tags as input features.
• Apply rules iteratively to refine chunk tag assignments based on local context, including neighboring words, POS tags, and existing chunk tags.
• Introduce lexical templates to improve performance on specific word patterns, particularly for challenging cases like VBG/VBN and conjunctions.
• Evaluate performance using standard recall, precision, and error reduction metrics on held-out test sets.

Experimental results

Research questions

• RQ1 Can transformation-based learning effectively learn to identify baseNP and more complex chunk structures in English text?
• RQ2 How does performance vary when lexical templates are included or excluded in the rule learning process?
• RQ3 What types of linguistic errors are most common, and can they be resolved with local context alone?
• RQ4 To what extent do part-of-speech tags and word identities provide sufficient information for accurate chunking?
• RQ5 Can the transformation-based learning framework be extended to model larger syntactic units or dependency-like structures?

Key findings

• The system achieved 92% precision and 92% recall for baseNP chunks, with a 48.7% error reduction compared to the baseline.
• For more complex partitioning chunks (including V and N groups), the system reached 88% precision and 88% recall, with a 56.3% error reduction.
• Lexical templates contributed modestly to baseNP chunking (38.8% error reduction), but more significantly to partitioning chunks (67.9% error reduction).
• The most frequent error class involved VBG and VBN verbs incorrectly tagged as outside baseNPs, indicating limitations of local context for verb phrase interpretation.
• Conjunctions (e.g., 'and', ',') were a major source of error, as the system struggled to distinguish when they were part of a single NP versus separate units.
• Many errors stemmed from semantic distinctions unavailable to local pattern matching, suggesting inherent limits of purely syntactic, local models.