[Paper Review] Memory-Based Shallow Parsing

Walter Daelemans, Sabine Buchholz+1 more|ArXiv.org|Jun 2, 1999

Natural Language Processing Techniques15 references118 citations

TL;DR

This paper presents a memory-based learning (MBL) approach to shallow parsing that integrates POS tagging, chunking, and syntactic relation detection (subject/object) as modular, classification-based tasks. Using the WSJ Treebank, the method achieves 93.8% F1 for NP chunking, 94.7% for VP chunking, 77.1% for subject detection, and 79.0% for object detection, outperforming or matching state-of-the-art methods with high efficiency and flexibility in handling diverse linguistic features.

ABSTRACT

We present a memory-based learning (MBL) approach to shallow parsing in which POS tagging, chunking, and identification of syntactic relations are formulated as memory-based modules. The experiments reported in this paper show competitive results, the F-value for the Wall Street Journal (WSJ) treebank is: 93.8% for NP chunking, 94.7% for VP chunking, 77.1% for subject detection and 79.0% for object detection.

Motivation & Objective

• To develop a flexible, efficient, and accurate approach to shallow parsing using memory-based learning (MBL) for syntactic pattern recognition.
• To address the limitations of eager learning methods in handling exceptions and subregularities in linguistic data by leveraging lazy learning with full training data retention.
• To evaluate MBL on core shallow parsing tasks: NP/VP chunking and subject/object relation detection, comparing performance to existing methods.
• To demonstrate the method's ability to integrate diverse linguistic features (e.g., morphology, WordNet, chunking output) without requiring complex retraining or post-processing.
• To show that MBL can detect nested syntactic relations and support multi-pass relation detection in a single inference pass, unlike some competing approaches.

Proposed method

• The approach formulates POS tagging, chunking, and syntactic relation detection as supervised classification tasks using memory-based learning (MBL), where each instance is a feature vector centered on a target word or tag.
• The system uses two MBL variants: IB1-IG, which computes similarity using information gain-weighted feature mismatches, and IGTree, a decision tree approximation for faster inference.
• Feature vectors include contextual information such as POS tags, word forms, morphological features, and chunk boundaries, enabling rich representation for classification.
• The method employs a cascaded architecture: chunking results are used as input for subject/object detection, allowing hierarchical processing of linguistic structure.
• Similarity-based prediction is performed by retrieving the most similar training instances from memory, with class labels extrapolated directly from the nearest neighbors.
• The approach supports multi-task learning by processing different syntactic relations (e.g., subject, object) in a single pass over a chunked sentence.

Experimental results

Research questions

• RQ1 Can memory-based learning achieve competitive performance on NP and VP chunking tasks compared to existing statistical and rule-based methods?
• RQ2 How does MBL handle linguistic exceptions and subregularities better than eager learning methods, given its retention of all training instances?
• RQ3 To what extent does incorporating additional linguistic features (e.g., word forms, morphological tags, WordNet) improve performance in subject and object detection?
• RQ4 How does the performance of MBL compare to the recently proposed memory-based sequence learning (MBSL) method on the same tasks, particularly in terms of precision, recall, and flexibility?
• RQ5 Can MBL detect nested syntactic relations and support multi-pass relation detection in a single inference pass, unlike MBSL?

Key findings

• The MBL approach achieved an F1 score of 93.8% for NP chunking and 94.7% for VP chunking on the WSJ Treebank, demonstrating strong performance on core chunking tasks.
• For subject detection, the method reached an F1 score of 77.1%, and for object detection, 79.0%, showing solid performance on syntactic relation identification.
• Performance dropped by 2.5% for subjects and 6.9% for objects when lexical information was excluded, highlighting the importance of word-level features.
• F1 scores decreased by over 50% when chunking was omitted before relation detection, underscoring the critical role of accurate chunking as a preprocessing step.
• Using gold-standard chunks instead of predicted ones increased F1 by 5.9% for subjects and 5.1% for objects, indicating that error propagation from chunking affects downstream tasks.
• MBL outperformed or matched state-of-the-art methods without post-processing, and demonstrated superior flexibility in incorporating diverse linguistic features and detecting nested relations.