[Paper Review] Cooperative Learning of Disjoint Syntax and Semantics

Serhii Havrylov, Germán Kruszewski+1 more|arXiv (Cornell University)|Feb 25, 2019

Topic Modeling46 references6 citations

TL;DR

This paper proposes a cooperative learning framework that separates syntax and semantics into distinct modules, training them with mixed continuous (gradient descent) and discrete (reinforcement learning) optimization. The model achieves near-perfect accuracy on parsing nested mathematical expressions from a context-free grammar and performs competitively on NLI and sentiment analysis tasks without requiring explicit syntactic supervision.

ABSTRACT

There has been considerable attention devoted to models that learn to jointly infer an expression's syntactic structure and its semantics. Yet, \citet{NangiaB18} has recently shown that the current best systems fail to learn the correct parsing strategy on mathematical expressions generated from a simple context-free grammar. In this work, we present a recursive model inspired by ewcite{ChoiYL18} that reaches near perfect accuracy on this task. Our model is composed of two separated modules for syntax and semantics. They are cooperatively trained with standard continuous and discrete optimization schemes. Our model does not require any linguistic structure for supervision and its recursive nature allows for out-of-domain generalization with little loss in performance. Additionally, our approach performs competitively on several natural language tasks, such as Natural Language Inference or Sentiment Analysis.

Motivation & Objective

• Address the limitation of existing models in learning correct syntactic structures from simple context-free grammars without supervision.
• Overcome the coadaptation problem in joint training of discrete parsers and continuous compositional functions.
• Enable out-of-domain generalization through a recursive, end-to-end trainable architecture with separated modules.
• Demonstrate competitive performance on standard NLP tasks like Natural Language Inference and Sentiment Analysis while learning meaningful syntactic structures.
• Explore the feasibility of using recursive models with latent tree structures as a pre-training method for downstream tasks.

Proposed method

• Decompose the model into two distinct modules: a discrete parser (tree structure generator) and a continuous compositional function (semantic representation learner).
• Train the parser using reinforcement learning with Proximal Policy Optimization (PPO) to stabilize training and control learning pace.
• Use input-dependent control variates to reduce gradient variance in the reinforcement learning objective.
• Apply multiple gradient steps per policy update in PPO to improve sample efficiency and policy learning.
• Combine gradient descent on the compositional function with reinforcement learning on the parser, synchronizing their learning rates to avoid coadaptation.
• Use a Tree-LSTM as the compositional function, differentiable via backpropagation through structure (BPTS), to generate semantic representations from the parsed tree.

Experimental results

Research questions

• RQ1 Can a jointly trained, end-to-end model learn correct syntactic structures from a simple context-free grammar without explicit parsing supervision?
• RQ2 How can discrete (parser) and continuous (compositional function) components be trained cooperatively without one dominating the other?
• RQ3 Does separating syntax and semantics improve generalization to out-of-domain sequences, such as longer or unseen mathematical expressions?
• RQ4 To what extent does learning latent syntactic structure improve performance on downstream NLP tasks like NLI and sentiment analysis?
• RQ5 Can the model avoid collapsing into trivial parsing strategies (e.g., left-branching trees) when trained on natural language data?

Key findings

• The model achieves 70.7% ± 0.3 accuracy on the MultiNLI benchmark, outperforming prior unsupervised latent tree models and matching state-of-the-art results.
• On the SST-2 and SST-5 sentiment analysis tasks, the model attains 90.2% ± 0.2 and 51.5% ± 0.4 accuracy, respectively, matching or exceeding prior RvNN-based models with latent trees.
• The model reaches near-perfect accuracy (99.8%) on the ListOps task, correctly parsing nested mathematical expressions from a context-free grammar.
• The parser learns balanced, meaningful syntactic structures without supervision, as evidenced by high entropy in the parsing policy and generalization to longer sequences.
• The use of PPO with multiple gradient steps and control variates successfully mitigates the coadaptation problem, preventing the parser from simply imitating the compositional function.
• Despite strong performance on formal grammar tasks, models on natural language tasks tend to fall back to trivial parsing strategies, consistent with prior findings.