[Paper Review] Parsing All: Syntax and Semantics, Dependencies and Spans

Junru Zhou, Zuchao Li+1 more|arXiv (Cornell University)|Aug 30, 2019

Topic Modeling51 references4 citations

TL;DR

This paper proposes a unified joint model for parsing both syntax and semantics using span and dependency representations, leveraging multi-task learning to enable bidirectional benefits between syntactic and semantic parsing. The model achieves state-of-the-art or competitive results across four tasks: span and dependency semantic role labeling (SRL), and constituent and dependency syntactic parsing on standard benchmarks.

ABSTRACT

Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former. However, few works ever made an attempt to let semantic parsing help syntactic parsing. As linguistic representation formalisms, both syntax and semantics may be represented in either span (constituent/phrase) or dependency, on both of which joint learning was also seldom explored. In this paper, we propose a novel joint model of syntactic and semantic parsing on both span and dependency representations, which incorporates syntactic information effectively in the encoder of neural network and benefits from two representation formalisms in a uniform way. The experiments show that semantics and syntax can benefit each other by optimizing joint objectives. Our single model achieves new state-of-the-art or competitive results on both span and dependency semantic parsing on Propbank benchmarks and both dependency and constituent syntactic parsing on Penn Treebank.

Motivation & Objective

• To address the gap in bidirectional interaction between syntactic and semantic parsing, where prior work mainly focused on syntax aiding semantics but not vice versa.
• To explore the joint learning of both span and dependency representations for both syntax and semantics, which remains underexplored in prior work.
• To design a uniform representation framework that integrates syntactic structure (via joint span) and semantic structure (via SRL constraints) for end-to-end training.
• To empirically verify whether semantic parsing can improve syntactic parsing and vice versa, beyond the conventional unidirectional flow of information.
• To achieve state-of-the-art performance on both semantic and syntactic parsing tasks without relying on external parse supervision at test time.

Proposed method

• Introduces a novel 'joint span' structure based on Head-Driven Phrase Structure Grammar (HPSG), which unifies constituent and dependency information by encoding both phrase structure and head-dependent arcs.
• Proposes a uniform SRL representation that simplifies training and enforces exact inference using SRL constraints, applicable to both span and dependency SRL.
• Employs multi-task learning with a balanced objective to jointly optimize for syntactic and semantic parsing, allowing information to flow in both directions.
• Uses a neural encoder that effectively incorporates syntactic information, with dependency and span structures modeled simultaneously through shared and task-specific heads.
• Trains a separate multi-class classifier for dependency labels alongside the parser, optimizing for both span and dependency structure prediction.
• Applies pre-trained contextual representations (e.g., BERT-like) to enhance feature learning, improving performance across all tasks without test-time auxiliary parsing.

Experimental results

Research questions

• RQ1 Can semantic parsing improve syntactic parsing performance, reversing the typical unidirectional flow of syntactic supervision?
• RQ2 To what extent do span and dependency representations for syntax and semantics benefit from joint modeling?
• RQ3 Does a unified representation for both span and dependency SRL improve generalization and performance compared to separate models?
• RQ4 Can a single model achieve state-of-the-art results across both syntactic and semantic parsing tasks without external parse supervision at test time?
• RQ5 How does the joint span structure, inspired by HPSG, improve the modeling of both constituent and dependency relations in a unified framework?

Key findings

• The proposed joint model achieves new state-of-the-art performance on both span and dependency SRL tasks using the CoNLL-2005 and CoNLL-2009 benchmarks, outperforming prior methods without test-time parse injection.
• The model achieves competitive results on both constituent and dependency parsing tasks using the Penn Treebank, demonstrating strong generalization across syntactic parsing forms.
• In end-to-end mode, the model outperforms the previous SOTA method Strubell et al. (2018) on both in-domain (WSJ) and out-domain (Brown) test sets for span SRL, even without using predicted parses at test time.
• The model achieves new state-of-the-art results on dependency SRL in both end-to-end and given-predicate modes on CoNLL-2009, confirming the effectiveness of the unified SRL representation.
• Ablation studies confirm that joint learning improves both semantic and syntactic parsing, with mutual benefits observed across all four tasks.
• The integration of pre-trained contextual representations further boosts performance, leading to SOTA results across all settings without requiring additional external parsing.