[論文レビュー] Content Enhanced BERT-based Text-to-SQL Generation

Tong Guo, Huilin Gao|arXiv (Cornell University)|Oct 16, 2019

Topic Modeling参考文献 5被引用数 46

ひとこと要約

この論文本体は、表の内容と見出しから派生した2つの外部特徴ベクトルをBERTベースの NL2SQL モデルに追加し、WikiSQL の性能を向上させ、最先端の結果を達成する。

ABSTRACT

We present a simple methods to leverage the table content for the BERT-based model to solve the text-to-SQL problem. Based on the observation that some of the table content match some words in question string and some of the table header also match some words in question string, we encode two addition feature vector for the deep model. Our methods also benefit the model inference in testing time as the tables are almost the same in training and testing time. We test our model on the WikiSQL dataset and outperform the BERT-based baseline by 3.7% in logic form and 3.7% in execution accuracy and achieve state-of-the-art.

研究の動機と目的

• Motivate leveraging table content as external knowledge for NL2SQL models.
• Propose two feature vectors aligning question with table cells and headers.
• Integrate these features into a BERT-based architecture for SELECT, AGG, and WHERE components.

提案手法

• Construct question-content and header-content match vectors (QV and HV) via word matching between question and table elements.
• Concatenate question and table header inputs for BERT to obtain contextual representations Q and H.
• Predict SELECT column, SELECT agg, WHERE number, WHERE column, WHERE op, and WHERE value using Q, H, QV, HV with respective probabilities.
• Train end-to-end on WikiSQL with ablation to assess vector contributions.
• Optionally apply execution-guided decoding to further boost results.

実験結果

リサーチクエスチョン

• RQ1 Can external features derived from table contents improve NL2SQL performance over standard BERT baselines?
• RQ2 How do table cell/header matches influence NU2SQL component predictions (SELECT, WHERE, AGG)?
• RQ3 Does incorporating QV and HV at inference time provide benefits when training and testing tables are similar?

主な発見

• Outperforms BERT-based baseline by 3.7 percentage points in both logic form and execution accuracy on WikiSQL.
• Achieves state-of-the-art results on the WikiSQL task with the proposed external features.
• Ablation shows header vector mainly improves WHERE OP while question vector mainly improves WHERE VALUE.
• Adding external feature vectors yields notable gains across SELECT column, WHERE number, WHERE column, WHERE value tasks.
• Execution-guided decoding provides incremental gains when used with the external features.