[Paper Review] Improving Text-to-SQL with Schema Dependency Learning

Binyuan Hui, Xiang Shi+5 more|arXiv (Cornell University)|Mar 7, 2021

Topic ModelingComputer Science36 references24 citations

TL;DR

SDSQL introduces schema dependency learning to guide Text-to-SQL without heavy reliance on execution-guided decoding, achieving state-of-the-art results on WikiSQL and faster inference without EG.

ABSTRACT

Text-to-SQL aims to map natural language questions to SQL queries. The sketch-based method combined with execution-guided (EG) decoding strategy has shown a strong performance on the WikiSQL benchmark. However, execution-guided decoding relies on database execution, which significantly slows down the inference process and is hence unsatisfactory for many real-world applications. In this paper, we present the Schema Dependency guided multi-task Text-to-SQL model (SDSQL) to guide the network to effectively capture the interactions between questions and schemas. The proposed model outperforms all existing methods in both the settings with or without EG. We show the schema dependency learning partially cover the benefit from EG and alleviates the need for it. SDSQL without EG significantly reduces time consumption during inference, sacrificing only a small amount of performance and provides more flexibility for downstream applications.

Motivation & Objective

• Motivate improving Text-to-SQL by reducing dependence on execution-guided decoding while leveraging schema-question interactions.
• Propose a multi-task SDSQL model that jointly learns schema dependency and SQL prediction.
• Show that schema dependency learning improves performance beyond existing sketch-based methods on WikiSQL.
• Demonstrate reduced inference time without EG and analyze component contributions via ablation studies.

Proposed method

• Encode questions and schema headers with BERT and Bi-LSTM to obtain representations.
• Construct a schema dependency learning task with predefined labels (S-Col, S-Agg, W-Col, W-Op, W-Val) and train a biaffine-based predictor to model question-schema edges and labels.
• Use a sketch-based SQL prediction module to predict SQL components in sub-modules.
• Adopt an adaptive multi-task loss to jointly optimize schema dependency and SQL prediction (learnable uncertainties σ1, σ2).
• Conduct experiments on WikiSQL, comparing LF/EX metrics against baselines with and without Execution-Guided (EG) decoding.

Experimental results

Research questions

• RQ1 How does schema dependency learning affect the interaction modeling between questions and schemas in text-to-SQL?
• RQ2 Can SDSQL achieve competitive or superior performance on WikiSQL without employing EG decoding?
• RQ3 What is the impact of the schema dependency task and adaptive multi-task loss on overall performance?
• RQ4 How much inference time is saved when EG is not used, and how does this trade off with accuracy?
• RQ5 Which sub-modules (e.g., W-Col, W-Val) benefit most from schema dependency learning?

Key findings

• SDSQL outperforms all existing models on WikiSQL in both development and test sets without EG.
• Without EG, SDSQL achieves higher LF and EX scores than all baselines listed.
• Using EG with SDSQL still yields the best reported results among compared methods.
• The ablation study shows the schema dependency module contributes more to performance gains than the adaptive loss alone.
• Schema dependency improves particularly the where-column (W-Col) and where-value (W-Val) predictions.