[论文解读] Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?

E. Shriberg, Rebecca Bates+8 more|ArXiv.org|Jun 11, 2000

Speech and dialogue systems参考文献 34被引用 4

一句话总结

本研究探讨了语调特征（如音高、时长、能量和语速）是否能提升自发会话语音中对话行为（DAs）的自动分类性能。基于Switchboard语料库，作者使用语调特征训练决策树，并将其与词级信息（来自人工转录或自动语音识别输出）结合，结果表明语调显著提升了DA分类准确率，尤其是在与语言模型结合时；同时，多种语调特征在标记对话行为方面表现出冗余性。

ABSTRACT

Identifying whether an utterance is a statement, question, greeting, and so forth is integral to effective automatic understanding of natural dialog. Little is known, however, about how such dialog acts (DAs) can be automatically classified in truly natural conversation. This study asks whether current approaches, which use mainly word information, could be improved by adding prosodic information. The study is based on more than 1000 conversations from the Switchboard corpus. DAs were hand-annotated, and prosodic features (duration, pause, F0, energy, and speaking rate) were automatically extracted for each DA. In training, decision trees based on these features were inferred; trees were then applied to unseen test data to evaluate performance. Performance was evaluated for prosody models alone, and after combining the prosody models with word information -- either from true words or from the output of an automatic speech recognizer. For an overall classification task, as well as three subtasks, prosody made significant contributions to classification. Feature-specific analyses further revealed that although canonical features (such as F0 for questions) were important, less obvious features could compensate if canonical features were removed. Finally, in each task, integrating the prosodic model with a DA-specific statistical language model improved performance over that of the language model alone, especially for the case of recognized words. Results suggest that DAs are redundantly marked in natural conversation, and that a variety of automatically extractable prosodic features could aid dialog processing in speech applications.

研究动机与目标

• 探究语调特征是否能提升自发自然会话中对话行为自动分类的性能。
• 评估语调相对于词级信息在对话行为分类任务中的贡献。
• 确定语调特征在缺少典型特征（如疑问句的F0）时，是否能补偿对话行为分类。
• 评估将语调模型与对话行为特定的统计语言模型结合对分类性能的影响。
• 探索利用语调通过对话行为感知语言建模来提升自动语音识别可行性的方法。

提出的方法

• 本研究使用Switchboard语料库中的1,000多段会话，对话行为采用SWBD-DAMSL标注体系人工标注。
• 为每个对话行为自动提取了语调特征，包括基频（F0）、时长、停顿、能量和语速。
• 在仅使用语调特征以及与词信息（来自真实转录或自动语音识别输出）结合的情况下，训练了决策树模型。
• 在整体对话行为分类及三个子任务上评估性能，并与语言模型基线进行比较。
• 将语调模型与对话行为特定的统计语言模型集成，以评估联合性能提升效果。
• 通过移除典型特征并观察性能变化，分析特征重要性与冗余性。

实验结果

研究问题

• RQ1 语调特征是否能在词级信息之外，进一步提升自发会话语音中对话行为的自动分类性能？
• RQ2 哪些语调特征对特定对话行为类型最具预测力？其与典型特征（如疑问句的F0）相比表现如何？
• RQ3 当典型特征不可用时，非典型语调特征能在多大程度上实现补偿？
• RQ4 将语调模型与对话行为特定语言模型结合，是否能提升分类准确率，尤其是在使用自动识别词的情况下？
• RQ5 当语调作为语言建模中的约束条件时，其在降低自动语音识别中的词错误率方面发挥何种作用？

主要发现

• 语调在所有任务中均显著提升了对话行为分类性能，尤其在与语言模型结合时提升最为明显。
• 将语调决策树与对话行为特定的统计语言模型结合，性能优于仅使用语言模型，尤其在使用自动语音识别输出的词时表现更优。
• 即使移除了典型特征（如F0），其他语调特征（如时长和能量）仍能有效补偿，表明对话行为的语调标记具有冗余性。
• 对于No-Answers，词错误率降低了18%；对于Backchannels，降低了7%；但由于Statements在Switchboard语料库中占主导地位，整体词错误率仅下降0.9%。
• 本研究证明，自然会话中对话行为被多重语调特征冗余标记，多种语调特征共同促进分类的鲁棒性。
• 结果表明，语调是对话行为分类与语音理解系统中可靠且互补的信息来源。