[Paper Review] Left-corner Methods for Syntactic Modeling with Universal Structural Constraints

Hiroshi Noji|arXiv (Cornell University)|Aug 1, 2016

Topic Modeling136 references3 citations

TL;DR

This paper proposes left-corner parsing methods to exploit universal syntactic constraints—particularly center-embedding avoidance—across languages. By extending left-corner parsing for dependency grammars and analyzing 20+ Universal Dependencies treebanks, it demonstrates that center-embedding is rare universally, and using this constraint improves unsupervised grammar induction and parsing performance.

ABSTRACT

The primary goal in this thesis is to identify better syntactic constraint or bias, that is language independent but also efficiently exploitable during sentence processing. We focus on a particular syntactic construction called center-embedding, which is well studied in psycholinguistics and noted to cause particular difficulty for comprehension. Since people use language as a tool for communication, one expects such complex constructions to be avoided for communication efficiency. From a computational perspective, center-embedding is closely relevant to a left-corner parsing algorithm, which can capture the degree of center-embedding of a parse tree being constructed. This connection suggests left-corner methods can be a tool to exploit the universal syntactic constraint that people avoid generating center-embedded structures. We explore such utilities of center-embedding as well as left-corner methods extensively through several theoretical and empirical examinations. Our primary task is unsupervised grammar induction. In this task, the input to the algorithm is a collection of sentences, from which the model tries to extract the salient patterns on them as a grammar. This is a particularly hard problem although we expect the universal constraint may help in improving the performance since it can effectively restrict the possible search space for the model. We build the model by extending the left-corner parsing algorithm for efficiently tabulating the search space except those involving center-embedding up to a specific degree. We examine the effectiveness of our approach on many treebanks, and demonstrate that often our constraint leads to better parsing performance. We thus conclude that left-corner methods are particularly useful for syntax-oriented systems, as it can exploit efficiently the inherent universal constraints in languages.

Motivation & Objective

• To identify language-independent syntactic constraints that can improve syntax-oriented NLP systems.
• To investigate whether center-embedding—known to be cognitively difficult—is universally avoided in natural languages.
• To develop a left-corner parsing algorithm that efficiently captures the degree of center-embedding in dependency trees.
• To evaluate whether constraining search space via center-embedding avoidance improves unsupervised grammar induction.
• To validate the universality of syntactic patterns using multilingual dependency treebanks from the Universal Dependencies project.

Proposed method

• Extends left-corner parsing to dependency grammars to track the degree of center-embedding during sentence processing.
• Defines a stack-based left-corner PDA where stack depth before shifting a token corresponds to its center-embedding degree.
• Uses a formal proof to show that stack depth δe before shifting token e equals m′e + 1, where m′e is the center-embedding degree of e.
• Applies the left-corner parsing framework to unsupervised grammar induction by restricting search to low-degree center-embedding structures.
• Employs a collection of 20+ multilingual dependency treebanks from the Universal Dependencies project for empirical evaluation.
• Implements a parser that tabulates search space efficiently by excluding high-degree center-embedding derivations up to a user-defined threshold.

Experimental results

Research questions

• RQ1 Is center-embedding universally avoided across diverse natural languages, and to what extent?
• RQ2 Can left-corner parsing effectively model and quantify the degree of center-embedding in dependency trees?
• RQ3 Does incorporating center-embedding avoidance as a universal constraint improve the performance of unsupervised grammar induction?
• RQ4 How does the left-corner parsing mechanism relate to the formal definition of center-embedding degree in syntactic structures?
• RQ5 To what extent can universal syntactic constraints reduce the search space in grammar induction without sacrificing accuracy?

Key findings

• Center-embedding is a rare phenomenon across the 20+ languages analyzed in the Universal Dependencies treebanks.
• The left-corner parsing algorithm successfully captures the degree of center-embedding, with stack depth before shifting a token being exactly one more than its center-embedding degree.
• The formal proof establishes that the maximum stack depth in the left-corner PDA corresponds to the maximum center-embedding degree in the parse tree.
• In unsupervised grammar induction, applying the center-embedding constraint leads to improved parsing performance on multiple treebanks.
• The left-corner method efficiently exploits universal syntactic constraints, reducing search space while maintaining or improving model accuracy.
• The results support the hypothesis that center-embedding avoidance is a universal linguistic constraint that can be computationally leveraged in syntax modeling.