[Paper Review] Automated speech audiometry: Can it work using open-source pre-trained Kaldi-NL automatic speech recognition?

Gloria Araiza-Illan, Luke Meyer+2 more|PubMed|Dec 19, 2023

Hearing Loss and Rehabilitation48 references4 citations

TL;DR

This study proposes an automated speech audiometry system using open-source Kaldi-NL for real-time speech recognition in the digits-in-noise (DIN) test, achieving a mean word error rate (WER) of 5.0% across 30 normal-hearing Dutch adults. Simulations show that up to four erroneous triplets cause SRT variations within typical within-subject variability (0.70 dB), indicating clinical feasibility for hearing screening without human supervision.

ABSTRACT

A practical speech audiometry tool is the digits-in-noise (DIN) test for hearing screening of populations of varying ages and hearing status. The test is usually conducted by a human supervisor (e.g., clinician), who scores the responses spoken by the listener, or online, where software scores the responses entered by the listener. The test has 24-digit triplets presented in an adaptive staircase procedure, resulting in a speech reception threshold (SRT). We propose an alternative automated DIN test setup that can evaluate spoken responses whilst conducted without a human supervisor, using the open-source automatic speech recognition toolkit, Kaldi-NL. Thirty self-reported normal-hearing Dutch adults (19-64 years) completed one DIN + Kaldi-NL test. Their spoken responses were recorded and used for evaluating the transcript of decoded responses by Kaldi-NL. Study 1 evaluated the Kaldi-NL performance through its word error rate (WER), percentage of summed decoding errors regarding only digits found in the transcript compared to the total number of digits present in the spoken responses. Average WER across participants was 5.0% (range 0-48%, SD = 8.8%), with average decoding errors in three triplets per participant. Study 2 analyzed the effect that triplets with decoding errors from Kaldi-NL had on the DIN test output (SRT), using bootstrapping simulations. Previous research indicated 0.70 dB as the typical within-subject SRT variability for normal-hearing adults. Study 2 showed that up to four triplets with decoding errors produce SRT variations within this range, suggesting that our proposed setup could be feasible for clinical applications.

Motivation & Objective

• To develop a low-cost, automated alternative to traditional human-supervised speech audiometry using open-source ASR.
• To evaluate the performance of pre-trained Kaldi-NL in recognizing digit-triplets in noisy conditions for hearing screening.
• To assess the impact of ASR decoding errors on the final speech reception threshold (SRT) in the DIN test.
• To determine whether automated SRT estimation using Kaldi-NL remains clinically reliable despite recognition errors.

Proposed method

• Administered the DIN test to 30 self-reported normal-hearing Dutch adults using recorded responses for ASR evaluation.
• Used open-source Kaldi-NL to decode spoken digit-triplets from recorded responses, focusing on digit recognition accuracy.
• Calculated word error rate (WER) per participant, specifically measuring errors in digits only, to isolate performance on target speech items.
• Simulated SRT estimation using bootstrapping to assess variability when up to four triplets contained decoding errors.
• Compared SRT variability under error conditions to the known within-subject SRT variability of 0.70 dB for normal-hearing adults.
• Used statistical simulations to evaluate the robustness of the automated SRT output under realistic error distributions.

Experimental results

Research questions

• RQ1 Can open-source Kaldi-NL achieve sufficiently low word error rates for reliable digit recognition in the DIN test?
• RQ2 How do decoding errors in digit-triplets affect the final speech reception threshold (SRT) in automated audiometry?
• RQ3 To what extent do ASR errors impact SRT variability compared to natural within-subject variability?
• RQ4 Is the SRT output from Kaldi-NL-based automated testing stable enough for clinical use?

Key findings

• The average word error rate (WER) across participants was 5.0%, with a standard deviation of 8.8% and a range from 0% to 48%.
• On average, each participant had three digit-triplets with decoding errors, indicating moderate but manageable error rates.
• Simulations showed that up to four erroneous triplets produced SRT variations within the typical within-subject SRT variability of 0.70 dB.
• The results suggest that Kaldi-NL-based automated DIN testing can produce SRT estimates with clinical reliability despite recognition errors.
• The system demonstrates feasibility for deployment in low-resource or remote hearing screening settings without human supervision.