[Paper Review] Arc-swift: A Novel Transition System for Dependency Parsing

Peng Qi, Christopher D. Manning|arXiv (Cornell University)|May 12, 2017

Topic Modeling10 references3 citations

TL;DR

This paper introduces arc-swift, a novel transition system for dependency parsing that enables direct, non-local attachments between distant tokens in a single transition, bypassing the need for long sequences of local shifts and reduces. By leveraging global lexical information more directly, arc-swift achieves 3.7–7.6% relative error reduction on Penn Treebank and English Universal Dependencies, significantly outperforming existing systems even with a very small beam size.

ABSTRACT

Transition-based dependency parsers often need sequences of local shift and reduce operations to produce certain attachments. Correct individual decisions hence require global information about the sentence context and mistakes cause error propagation. This paper proposes a novel transition system, arc-swift, that enables direct attachments between tokens farther apart with a single transition. This allows the parser to leverage lexical information more directly in transition decisions. Hence, arc-swift can achieve significantly better performance with a very small beam size. Our parsers reduce error by 3.7--7.6% relative to those using existing transition systems on the Penn Treebank dependency parsing task and English Universal Dependencies.

Motivation & Objective

• To address the challenge of long-distance dependency attachments in transition-based dependency parsing, which require lengthy, non-obvious transition sequences in traditional systems.
• To reduce error propagation by enabling parsing decisions that incorporate global sentence context rather than relying on local, incremental transitions.
• To improve parsing accuracy—particularly for prepositional phrases and conjunctions—by allowing direct attachment of distant tokens in a single operation.
• To design a transition system that avoids spurious ambiguity and is easily implementable using only the standard parser state, unlike prior extensions requiring complex data structures.

Proposed method

• arc-swift introduces two new transition operations: LArc[k] and RArc[k], which allow a token on the stack to directly attach to a dependent anywhere in the sentence, with k denoting the number of stack elements involved.
• The system operates by allowing a right arc or left arc to be applied not just to the top of the stack, but to any of the top k stack elements, enabling direct attachment to a buffer token in one step.
• The parser state is represented as (σ | i_k | ... | i_1, j | β, A), where σ is the stack, i_k to i_1 are stack elements, j is the buffer head, and A is the set of arcs.
• Transitions are defined such that LArc[k] attaches the k-th stack element to a buffer token j, and RArc[k] attaches j to the k-th stack element, with the resulting state updated accordingly.
• The method supports beam search decoding with a very small beam size (e.g., 1–2), yet achieves high performance due to the reduced need for sequential, error-prone decisions.
• The system is designed to be backward compatible with standard transition-based parsing frameworks and does not require modifications to the underlying classifier or feature representation.

Experimental results

Research questions

• RQ1 Can a transition system that allows direct, non-local attachments between distant tokens improve dependency parsing accuracy?
• RQ2 Does enabling long-range attachments in a single transition reduce error propagation compared to sequential local transitions?
• RQ3 To what extent does arc-swift improve parsing performance on challenging constructions such as prepositional phrases and conjunctions?
• RQ4 Can arc-swift achieve high performance with a very small beam size, indicating reduced dependency on extensive search?

Key findings

• arc-swift parsers reduce error by 3.7–7.6% relative on the Penn Treebank dependency parsing task compared to parsers using existing transition systems.
• On English Universal Dependencies, arc-swift achieves similar relative error reductions, demonstrating robustness across different datasets and annotation schemes.
• The system shows significant improvements in parsing accuracy for prepositional phrase and conjunction attachments, which are commonly misparsed in traditional systems.
• Even with a beam size of 1, arc-swift achieves competitive performance, and with a beam size of 2, it outperforms greedy decoding in arc-eager systems, indicating high efficiency and effectiveness.
• Linguistic analysis reveals that arc-swift reduces errors in common constructions such as PP attachments and conjunctions, while maintaining comparable performance on other error types.
• The method avoids spurious ambiguity and is easily implementable using only the standard parser state, unlike prior systems that require tracking spines or additional data structures.