[Paper Review] Incremental Online Spoken Language Understanding.

Prashanth Gurunath Shivakumar, Naveen Kumar+2 more|arXiv (Cornell University)|Oct 23, 2019

Speech and dialogue systems16 references4 citations

TL;DR

This paper proposes an RNN-based incremental spoken language understanding system that processes ASR transcripts in real time, reducing latency without sacrificing accuracy. By introducing an EOS detector for sentence segmentation and enabling early intent detection, the system achieves lower latency than offline methods while maintaining high intent classification performance on the ATIS dataset under streaming conditions.

ABSTRACT

Spoken Language Understanding (SLU) typically comprises of an automatic speech recognition (ASR) followed by a natural language understanding (NLU) module. The two modules process signals in a blocking sequential fashion, i.e., the NLU often has to wait for the ASR to finish processing on an utterance basis, potentially leading to high latencies that render the spoken interaction less natural. In this paper, we propose recurrent neural network (RNN) based incremental processing towards the SLU task of intent detection. The proposed methodology offers lower latencies than a typical SLU system, without any significant reduction in system accuracy. We introduce and analyze different recurrent neural network architectures for incremental and online processing of the ASR transcripts and compare it to the existing offline systems. A lexical End-of-Sentence (EOS) detector is proposed for segmenting the stream of transcript into sentences for intent classification. Intent detection experiments are conducted on benchmark ATIS dataset modified to emulate a continuous incremental stream of words with no utterance demarcation. We also analyze the prospects of early intent detection, before EOS, with our proposed system.

Motivation & Objective

• To reduce latency in spoken language understanding by enabling online, incremental processing of ASR transcripts.
• To address the blocking nature of traditional SLU systems that wait for full utterance completion before NLU processing.
• To develop a lexical End-of-Sentence (EOS) detector for segmenting continuous speech streams into sentences for intent classification.
• To enable early intent detection before full sentence completion, improving responsiveness in real-time interactions.
• To evaluate the trade-off between latency and accuracy in incremental SLU using the ATIS benchmark under streaming conditions.

Proposed method

• Employing recurrent neural networks (RNNs) to process ASR transcripts incrementally, allowing real-time intent detection as words stream in.
• Introducing a lexical End-of-Sentence (EOS) detector to identify sentence boundaries in continuous speech streams, enabling timely intent classification.
• Modifying the ATIS dataset to simulate a continuous, utterance-free stream of words for realistic incremental processing evaluation.
• Training and evaluating RNN architectures for incremental processing, comparing them to standard offline systems in terms of latency and accuracy.
• Implementing early intent detection by classifying intent before the EOS signal, based on partial input sequences.
• Using a streaming inference pipeline where intent is predicted as soon as sufficient context is available, minimizing delay.

Experimental results

Research questions

• RQ1 Can incremental RNN-based processing reduce latency in spoken language understanding without degrading intent detection accuracy?
• RQ2 How effective is a lexical EOS detector in segmenting continuous speech streams for intent classification in real time?
• RQ3 To what extent can early intent detection be achieved before full sentence completion while maintaining acceptable accuracy?
• RQ4 How do different RNN architectures perform under incremental processing compared to standard offline models?
• RQ5 What is the trade-off between latency reduction and accuracy in incremental SLU systems?

Key findings

• The proposed incremental RNN system achieves significantly lower latency than conventional offline SLU systems while maintaining comparable intent detection accuracy.
• The lexical EOS detector effectively identifies sentence boundaries in continuous streams, enabling timely intent classification.
• Early intent detection is feasible, with the system producing reliable predictions before the full sentence is received.
• The incremental RNN approach maintains high accuracy even when processing partial utterances, demonstrating robustness to incomplete input.
• Different RNN architectures show varying performance in incremental settings, with some offering better latency-accuracy trade-offs than others.
• The system outperforms standard offline baselines in latency while matching or slightly underperforming in accuracy, indicating a favorable trade-off for real-time applications.