[Paper Review] Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs

Bailin Wang, Ivan Titov+1 more|arXiv (Cornell University)|Sep 9, 2019

Topic Modeling47 references4 citations

TL;DR

This paper proposes a weakly-supervised neural semantic parser that improves generalization by modeling structured latent alignments between natural language questions and program components, using abstract programs to reduce search space. By enforcing alignment constraints—such as unique span-to-slot mappings—it effectively distinguishes correct from spurious programs, achieving state-of-the-art performance on WikiTableQuestions and WikiSQL.

ABSTRACT

Semantic parsing aims to map natural language utterances onto machine interpretable meaning representations, aka programs whose execution against a real-world environment produces a denotation. Weakly-supervised semantic parsers are trained on utterance-denotation pairs treating programs as latent. The task is challenging due to the large search space and spuriousness of programs which may execute to the correct answer but do not generalize to unseen examples. Our goal is to instill an inductive bias in the parser to help it distinguish between spurious and correct programs. We capitalize on the intuition that correct programs would likely respect certain structural constraints were they to be aligned to the question (e.g., program fragments are unlikely to align to overlapping text spans) and propose to model alignments as structured latent variables. In order to make the latent-alignment framework tractable, we decompose the parsing task into (1) predicting a partial "abstract program" and (2) refining it while modeling structured alignments with differential dynamic programming. We obtain state-of-the-art performance on the WIKITABLEQUESTIONS and WIKISQL datasets. When compared to a standard attention baseline, we observe that the proposed structured-alignment mechanism is highly beneficial.

Motivation & Objective

• Address the challenge of spurious programs in weakly-supervised semantic parsing, where programs execute correctly but do not generalize.
• Reduce the search space of possible programs by introducing abstract programs that capture high-level operations like row and column selection.
• Incorporate structural constraints—such as non-overlapping, bijective alignments between question spans and program slots—as an inductive bias to favor semantically consistent programs.
• Develop a tractable two-stage parsing framework: first predict an abstract program, then instantiate it using structured attention over latent alignments.
• Improve robustness to unseen tables and rare entities by decoupling abstract program generation from low-level entity-specific instantiation.

Proposed method

• Decompose semantic parsing into two stages: (1) sequence-to-sequence model generates an abstract program from a question, and (2) local classifiers instantiate slots using structured attention.
• Model alignments between question spans and program slots as structured latent variables, enforcing constraints like uniqueness and non-overlap.
• Use differential dynamic programming to efficiently compute marginal probabilities over alignment edges during training.
• Apply structured attention to weight input representations based on alignment scores, improving slot instantiation accuracy.
• Introduce a two-stage training process: first train the abstract program generator, then jointly fine-tune the instantiation module with alignment modeling.
• Enforce structural constraints by penalizing alignments that assign multiple operations to the same text span, discouraging spurious programs.

Experimental results

Research questions

• RQ1 Can structured latent alignments improve generalization in weakly-supervised semantic parsing by reducing reliance on spurious programs?
• RQ2 How effective is the use of abstract programs in reducing the search space while preserving expressiveness for table-based semantic parsing?
• RQ3 To what extent do alignment constraints—such as bijectivity and non-overlap—improve the parser’s ability to distinguish correct from spurious programs?
• RQ4 Does modeling alignments as structured latent variables lead to better performance on benchmark datasets like WikiTableQuestions and WikiSQL?
• RQ5 How does the proposed method compare to standard attention mechanisms in handling instantiation errors and coverage gaps?

Key findings

• The proposed parser achieves state-of-the-art performance on both WikiTableQuestions and WikiSQL, outperforming standard attention-based baselines.
• Empirical analysis shows that the structured alignment mechanism significantly reduces instantiation errors—particularly those caused by incorrect column names or filter conditions—compared to standard attention.
• The model exhibits a lower rate of abstraction errors, indicating improved reliability in generating correct abstract programs.
• Coverage errors remain a challenge, primarily due to unsupported operations and entity linking failures, but are less frequent than instantiation errors.
• The structured alignment model effectively discourages spurious programs by violating alignment constraints, such as assigning multiple operations to the same question span.
• Ablation studies confirm that the inductive bias from structured alignments improves robustness, especially in cases where spurious programs produce correct denotations but lack semantic consistency.