[Paper Review] Cascaded Grammatical Relation Assignment

Sabine Buchholz, Jorn Veenstra+1 more|ArXiv.org|Jun 2, 1999

Natural Language Processing Techniques15 references74 citations

TL;DR

This paper proposes a cascaded Memory-Based Learning approach for grammatical relation (GR) assignment, where multiple stages sequentially identify chunks (NP, VP, PP, etc.) and their adverbial functions before assigning GRs like subject or object. Despite imperfect lower-level classifiers, the cascade improves overall GR performance, with adverbial function labeling contributing significantly even when its own accuracy is moderate.

ABSTRACT

In this paper we discuss cascaded Memory-Based grammatical relations assignment. In the first stages of the cascade, we find chunks of several types (NP,VP,ADJP,ADVP,PP) and label them with their adverbial function (e.g. local, temporal). In the last stage, we assign grammatical relations to pairs of chunks. We studied the effect of adding several levels to this cascaded classifier and we found that even the less performing chunkers enhanced the performance of the relation finder.

Motivation & Objective

• To investigate whether cascaded grammatical relation assignment using Memory-Based Learning improves performance over monolithic approaches.
• To evaluate the impact of adding multiple processing stages—such as chunking and adverbial function labeling—on the final grammatical relation assignment.
• To determine whether imperfect lower-level classifiers still contribute positively to the overall cascade performance.
• To assess error propagation from earlier stages to later stages in the cascade.
• To explore whether sequential, layered processing enhances grammatical relation detection beyond simultaneous processing.

Proposed method

• Employ Memory-Based Learning (MBL) with IB1, IB1-IG, and IGTree algorithms for all classification stages.
• Use a cascade architecture: first POS tagging, then simultaneous chunking (NP, VP, PP, ADJP, ADVP), followed by adverbial function labeling (e.g., locative, temporal), and finally grammatical relation assignment.
• Leverage overlapping features and similarity-based generalization to handle rare or exceptional cases in low-frequency contexts.
• Train and test on the Penn Treebank II WSJ corpus, using sections 00–19 for training and 20–24 for testing.
• Use perfect test data (manually corrected annotations) to isolate the effect of errors in earlier cascade stages.
• Apply information gain weighting in similarity metrics to prioritize informative features across heterogeneous input sources.

Experimental results

Research questions

• RQ1 Does cascading multiple Memory-Based classifiers improve grammatical relation assignment performance compared to a single-stage system?
• RQ2 To what extent do imperfect lower-level modules (e.g., chunkers or adverbial function taggers) still enhance the final GR assignment?
• RQ3 How much performance gain is achievable by improving intermediate modules like adverbial function labeling or PP chunking?
• RQ4 To what extent is error propagation from earlier stages responsible for performance degradation in later stages?
• RQ5 Can sequential processing of different chunk types (e.g., ADJP before NP) improve overall chunking and GR assignment accuracy?

Key findings

• Adding NP chunking to the cascade increased the F1-score for subject identification by 11.3% and for object identification by 16.2%.
• Adverbial function labeling improved F1-score for locative and temporal adjuncts by 6.3% and 15%, respectively, despite its own F1 of 73.5%.
• Even with imperfect intermediate modules, the full cascade achieved a 71.2% F1-score for grammatical relation assignment on realistic data, rising to 83.5% on perfect test data.
• The largest performance gap between realistic and perfect data occurred in the final GR stage—12.3% improvement when all prior modules were perfect—indicating that error propagation is significant.
• PP chunking and adverbial function labeling each contributed positively to GR assignment, even though their individual F1 scores were relatively low (92.0% and 73.5%, respectively).
• The study demonstrated that cascaded systems can be robust to errors in lower-level components, as even less accurate modules like ADJP and ADVP chunkers still contributed to overall performance gains.