[Paper Review] X-SQL: reinforce schema representation with context

Pengcheng He, Yi Mao+2 more|arXiv (Cornell University)|Aug 21, 2019

Topic ModelingComputer Science11 references53 citations

TL;DR

X-SQL introduces a context-enhanced schema representation by integrating BERT-style contextual outputs with schema information, achieving new state-of-the-art results on WikiSQL.

ABSTRACT

In this work, we present X-SQL, a new network architecture for the problem of parsing natural language to SQL query. X-SQL proposes to enhance the structural schema representation with the contextual output from BERT-style pre-training model, and together with type information to learn a new schema representation for down-stream tasks. We evaluated X-SQL on the WikiSQL dataset and show its new state-of-the-art performance.

Motivation & Objective

• Motivate improved semantic parsing from NL to SQL by better integrating unstructured query context with structured schema.
• Develop a context-enhanced schema encoder that leverages global context from a pre-trained model to refine column representations.
• Incorporate schema type information to constrain SQL syntax choices.
• Address limitations of independently trained sub-tasks by using a unified, context-aware architecture.

Proposed method

• Use a sequence encoder similar to BERT, initialized from MT-DNN, with a special [CTX] context output.
• Compute context-enhanced column representations by aligning column tokens with global context via a softmax-weighted sum.
• Predict SQL sub-tasks (S-COL, S-AGG, W-NUM, W-COL, W-OP, W-VAL) with a modular, task-specific network that modulates schema representations using context.
• Incorporate schema type embeddings to guide sub-task predictions, especially for aggregators.
• Adopt a list-wise KL-divergence objective for W-COL to compare column predictions collectively rather than independently.
• Train with a sum of sub-task losses; inference follows a straightforward combination of sub-task outputs.

Experimental results

Research questions

• RQ1 Can incorporating global contextual representations of the natural language query improve schema understanding for NL-to-SQL tasks?
• RQ2 Does adding explicit schema type information and a list-wise learning objective enhance sub-task predictions (e.g., where clause prediction) in WikiSQL?
• RQ3 To what extent does context-enhanced schema representation improve overall SQL accuracy compared to prior models on WikiSQL?
• RQ4 Is the proposed X-SQL architecture robust to execution-guided decoding and its variants?

Key findings

• X-SQL achieves new state-of-the-art on WikiSQL dev and test sets, outperforming SQLova with and without execution guidance.
• Without execution guidance, X-SQL improves logical form accuracy by 2.6 percentage points on the test set (83.3 vs 80.7) and execution accuracy by 2.5 points (88.7 vs 86.2).
• With execution guidance, X-SQL further improves to 86.0% LF and 91.8% EX on the test set, surpassing the best prior model.
• Per-sub-task gains include notable improvements in W-COL and W-VAL, aided by the list-wise KL-divergence objective and context-aware schema modulation.
• X-SQL plus execution guidance is the first model to surpass 90% accuracy on the test set under the WikiSQL benchmark.