[Paper Review] Compositional Semantic Parsing on Semi-Structured Tables

Panupong Pasupat, Percy Liang|arXiv (Cornell University)|Aug 3, 2015

Natural Language Processing Techniques31 references114 citations

TL;DR

This paper introduces a compositional semantic parsing framework for complex question answering over semi-structured HTML tables, using a logical-form-driven parser with strong typing and denotation constraints to manage combinatorial explosion. It achieves 37.1% accuracy on a new 22,033-sample dataset of Wikipedia tables, significantly outperforming baselines at 12.7% (IR) and 24.3% (simple parsing).

ABSTRACT

Two important aspects of semantic parsing for question answering are the breadth of the knowledge source and the depth of logical compositionality. While existing work trades off one aspect for another, this paper simultaneously makes progress on both fronts through a new task: answering complex questions on semi-structured tables using question-answer pairs as supervision. The central challenge arises from two compounding factors: the broader domain results in an open-ended set of relations, and the deeper compositionality results in a combinatorial explosion in the space of logical forms. We propose a logical-form driven parsing algorithm guided by strong typing constraints and show that it obtains significant improvements over natural baselines. For evaluation, we created a new dataset of 22,033 complex questions on Wikipedia tables, which is made publicly available.

Motivation & Objective

• Address the trade-off between knowledge source breadth and logical compositionality in semantic parsing.
• Enable accurate parsing of complex, compositional questions on open-domain, semi-structured tables with unseen relations and entities.
• Develop a parsing system robust to novel relations and entities at test time, without relying on pre-learned lexicons.
• Create a large-scale, publicly available dataset of question-answer pairs grounded in Wikipedia tables to support research in open-domain semantic parsing.
• Improve generalization in semantic parsing by avoiding schema-fixed knowledge bases and instead working directly on raw, unnormalized HTML tables.

Proposed method

• Parse questions into candidate logical forms using a high-coverage, compositional grammar that generates forms over table elements.
• Encode table content as a typed graph, representing relations (column headers), entities (cell values), and their interrelations.
• Apply beam search with pruning based on type consistency and denotation constraints to control the combinatorial explosion of logical forms.
• Use a log-linear model to rerank candidate logical forms based on features including phrase-to-relation alignment and structural consistency.
• Train the parser end-to-end on question-table-answer triples, with no reliance on pre-learned lexicons mapping phrases to relations.
• Enforce strong typing constraints to ensure logical forms are semantically well-formed and executable on the table.

Experimental results

Research questions

• RQ1 Can a semantic parser generalize to complex, compositional questions on semi-structured tables with unseen relations and entities?
• RQ2 How effective is a logical-form-driven parsing approach with strong typing and denotation constraints in managing the combinatorial explosion of possible logical forms?
• RQ3 To what extent can a parser trained on question-table-answer triples generalize to tables not seen during training?
• RQ4 How does the proposed method compare to information retrieval and simple semantic parsing baselines on open-domain, table-based question answering?
• RQ5 What is the impact of logical coverage and compositional depth on model performance in real-world, web-scale table QA?

Key findings

• The proposed semantic parser achieves 37.1% accuracy on the WikiTableQuestions test set, significantly outperforming an information retrieval baseline (12.7%) and a simple semantic parsing baseline (24.3%).
• The system generalizes effectively to tables with previously unseen relations and entities, as training and test tables are disjoint by design.
• The use of strong typing and denotation-based pruning reduces the search space and improves efficiency, enabling scalable parsing over complex tables.
• The dataset contains 22,033 question-answer pairs across 2,108 Wikipedia tables, with 20% reserved for testing to ensure zero-shot generalization evaluation.
• Error analysis shows that 25% of errors stem from oblique phrase-relation alignment (e.g., 'airplane' to 'Model'), indicating a need for better lexical generalization.
• The framework supports a diverse set of logical operations, including comparison, superlatives, aggregation, and arithmetic, demonstrating broad compositional coverage.