[Paper Review] Mere account mein kitna balance hai? -- On building voice enabled Banking Services for Multilingual Communities

Akshat Gupta, Sai Krishna Rallabandi+1 more|arXiv (Cornell University)|Oct 9, 2020

Linguistic Variation and Morphology1 references6 citations

TL;DR

This paper proposes a voice-enabled banking system for multilingual communities using acoustic phone units and a Naive Bayes classifier to recognize user intents in mixed-language (Hinglish) speech. Despite a small dataset, combining unigram, bigram, and trigram models with absolute discounting achieved 83% accuracy, demonstrating feasibility for multilingual speech-based banking in low-resource settings.

ABSTRACT

Tremendous progress in speech and language processing has brought language technologies closer to daily human life. Voice technology has the potential to act as a horizontal enabling layer across all aspects of digitization. It is especially beneficial to rural communities in scenarios like a pandemic. In this work we present our initial exploratory work towards one such direction -- building voice enabled banking services for multilingual societies. Speech interaction for typical banking transactions in multilingual communities involves the presence of filled pauses and is characterized by Code Mixing. Code Mixing is a phenomenon where lexical items from one language are embedded in the utterance of another. Therefore speech systems deployed for banking applications should be able to process such content. In our work we investigate various training strategies for building speech based intent recognition systems. We present our results using a Naive Bayes classifier on approximate acoustic phone units using the Allosaurus library.

Motivation & Objective

• To develop a voice-enabled banking service for multilingual communities, particularly in rural India, where language diversity and low digital literacy are challenges.
• To address the linguistic complexity of code-mixed speech—especially Hinglish—common in informal banking interactions.
• To investigate effective training strategies for intent classification using limited, low-resource speech data from multiple Indian languages.
• To create a pseudo-parallel multilingual dataset by acoustic translation of utterances from one language to another, enabling future semi-supervised learning.
• To evaluate the performance of N-gram models on approximate acoustic phone units for intent recognition in low-resource, multilingual speech.

Proposed method

• Collected 100+ task-based dialogues in Hinglish from 11 speakers using a dummy banking app, covering five banking intents.
• Acquired speech data from six Indian languages: Hindi, Marathi, Gujarati, Punjabi, Telugu, and Bhojpuri.
• Converted raw audio into approximate acoustic phone units using the Allosaurus library for downstream classification.
• Applied a Naive Bayes classifier with add-1 smoothing and absolute discounting for N-gram language modeling.
• Used 5-fold cross-validation, leaving out two utterances per fold for testing, excluding the rare 'Withdraw Money' and 'Deposit Money' intents due to single-sample size.
• Combined unigram, bigram, and trigram models with optimal delta values from absolute discounting to improve performance.

Experimental results

Research questions

• RQ1 Can a speech-based intent recognition system effectively handle code-mixed, multilingual utterances common in Indian banking contexts?
• RQ2 How does N-gram model order (unigram, bigram, trigram) affect intent classification accuracy on limited, low-resource speech data?
• RQ3 Does absolute discounting improve intent classification performance compared to add-1 smoothing in low-data regimes?
• RQ4 Can a hybrid N-gram model combining unigram, bigram, and trigram features achieve higher accuracy than individual models?
• RQ5 Can pseudo-parallel data generated via acoustic translation support future semi-supervised learning in multilingual speech applications?

Key findings

• The unigram model with add-1 smoothing achieved the highest accuracy among individual N-gram models at 56%, while trigram performance dropped significantly to 17%.
• Absolute discounting improved test accuracy across all N-gram models, with the unigram model reaching 69% accuracy using a delta of 5.
• The combined model using optimal delta values (5 for unigram, 1 for bigram, 1 for trigram) achieved the best performance at 83% accuracy.
• Performance degradation with increasing N-gram order is attributed to data sparsity in the initial exploratory dataset, suggesting that larger datasets may yield more uniform N-gram distributions.
• The use of pseudo-parallel data from the same speaker enables potential for future semi-supervised training in multilingual speech applications.
• The results indicate that acoustic phone-level N-gram modeling with Naive Bayes is viable for intent recognition in low-resource, multilingual banking scenarios.