[论文解读] Speech-to-SQL: Towards Speech-driven SQL Query Generation From Natural Language Question

Yuanfeng Song, Raymond Chi-Wing Wong+2 more|arXiv (Cornell University)|Jan 4, 2022

Speech and dialogue systems被引用 7

一句话总结

本文提出Speech-to-SQL，一种新型端到端任务，可直接将自然语言语音转化为结构化SQL查询，绕过易出错的级联ASR+文本转SQL流程。所提出的SpeechSQLNet模型直接利用语音信号，在新构建的SpeechQL数据集上，其准确率高于级联方法和当前最先进文本转SQL模型，证明了语音驱动数据库查询的可行性和优越性。

ABSTRACT

Speech-based inputs have been gaining significant momentum with the popularity of smartphones and tablets in our daily lives, since voice is the most easiest and efficient way for human-computer interaction. This paper works towards designing more effective speech-based interfaces to query the structured data in relational databases. We first identify a new task named Speech-to-SQL, which aims to understand the information conveyed by human speech and directly translate it into structured query language (SQL) statements. A naive solution to this problem can work in a cascaded manner, that is, an automatic speech recognition (ASR) component followed by a text-to-SQL component. However, it requires a high-quality ASR system and also suffers from the error compounding problem between the two components, resulting in limited performance. To handle these challenges, we further propose a novel end-to-end neural architecture named SpeechSQLNet to directly translate human speech into SQL queries without an external ASR step. SpeechSQLNet has the advantage of making full use of the rich linguistic information presented in speech. To the best of our knowledge, this is the first attempt to directly synthesize SQL based on arbitrary natural language questions, rather than a natural language-based version of SQL or its variants with a limited SQL grammar. To validate the effectiveness of the proposed problem and model, we further construct a dataset named SpeechQL, by piggybacking the widely-used text-to-SQL datasets. Extensive experimental evaluations on this dataset show that SpeechSQLNet can directly synthesize high-quality SQL queries from human speech, outperforming various competitive counterparts as well as the cascaded methods in terms of exact match accuracies.

研究动机与目标

• 为解决级联ASR + 文本转SQL流程的局限性，此类流程易受错误累积影响，且依赖高质量ASR。
• 实现直接、端到端的自然语言语音问题到可执行SQL查询的转换，降低非技术用户使用门槛。
• 探索绕过文本的语音驱动接口，尤其对缺乏书面形式的低资源语言具有显著优势。
• 通过在现有文本转SQL数据集基础上添加语音转录，构建新基准数据集SpeechQL，以支持端到端训练与评估。
• 验证端到端语音转SQL模型在从任意口语问题生成准确SQL方面的能力。

提出的方法

• 提出SpeechSQLNet，一种端到端神经架构，可直接将原始语音信号映射为SQL查询，无需依赖中间ASR转录文本。
• 采用语音表征（如MFCC）作为输入，保留文本转录中丢失的丰富语言与语调信息。
• 设计带有注意力机制的序列到序列模型，将语音嵌入映射为结构化SQL语法。
• 在新构建的SpeechQL数据集上联合训练模型，该数据集通过添加语音语句配对数据，扩展了流行的文本转SQL基准。
• 采用混合训练策略，结合语音特征与序列建模，提升对类似ASR错误的鲁棒性。
• 将SpeechSQLNet与级联基线方法（ASR + 文本转SQL）及当前最先进文本转SQL模型进行对比，评估端到端性能。

实验结果

研究问题

• RQ1 端到端语音转SQL模型是否能在SQL生成准确率上超越级联ASR + 文本转SQL流程？
• RQ2 与基于文本的模型相比，直接语音转SQL建模在多大程度上保留了有助于泛化的语言与语调线索？
• RQ3 当输入已为转录文本时，端到端模型对ASR错误的鲁棒性如何？其能否泛化至低资源语言？
• RQ4 语音驱动接口是否能显著减少非技术用户查询关系型数据库的时间与认知负担？
• RQ5 在低资源语言环境下，利用原始语音信号是否优于依赖ASR转录文本，从而实现更好的泛化能力？

主要发现

• SpeechSQLNet在精确匹配准确率上高于级联ASR + 文本转SQL方法，证明了端到端学习在语音转SQL中的优越性。
• 即使在相同SpeechQL数据集上微调，该模型仍优于当前最先进文本转SQL模型，表明直接语音输入具有优势。
• ASR错误率高达33%时，下游文本转SQL模型准确率下降超过36%，凸显级联系统的脆弱性。
• 所提出的SpeechQL数据集可支持端到端语音转SQL模型的可靠评估，并推动语音驱动数据库查询的后续研究。
• 语音转SQL被证明是一种可行且高效的范式，尤其适用于非技术用户和低资源语言群体。
• 结果验证了语音信号中包含对SQL生成有益的非语言信息，而这些信息在转录过程中会丢失。