[Paper Review] Denoising convolutional autoencoder based B-mode ultrasound tongue image feature extraction

Bo Li, Kele Xu+4 more|arXiv (Cornell University)|Mar 3, 2019

Speech Recognition and Synthesis21 references4 citations

TL;DR

This paper proposes a denoising convolutional autoencoder (DCAE) for unsupervised feature extraction from B-mode ultrasound tongue images, leveraging convolutional architecture to enhance robustness against speckle noise and image distortions. DCAE achieves a Word Error Rate of 6.17% on the 2010 Silent Speech Interface Challenge, outperforming state-of-the-art DCT-based methods (6.45%) and other autoencoder variants.

ABSTRACT

B-mode ultrasound tongue imaging is widely used in the speech production field. However, efficient interpretation is in a great need for the tongue image sequences. Inspired by the recent success of unsupervised deep learning approach, we explore unsupervised convolutional network architecture for the feature extraction in the ultrasound tongue image, which can be helpful for the clinical linguist and phonetics. By quantitative comparison between different unsupervised feature extraction approaches, the denoising convolutional autoencoder (DCAE)-based method outperforms the other feature extraction methods on the reconstruction task and the 2010 silent speech interface challenge. A Word Error Rate of 6.17% is obtained with DCAE, compared to the state-of-the-art value of 6.45% using Discrete cosine transform as the feature extractor. Our codes are available at https://github.com/DeePBluE666/Source-code1.

Motivation & Objective

• To address the challenge of extracting robust, speech-relevant features from noisy, low-SNR B-mode ultrasound tongue images.
• To explore unsupervised deep learning methods for dimensionality reduction and feature learning in ultrasound-based silent speech interfaces (SSI).
• To improve reconstruction fidelity and recognition performance by leveraging the invariance properties of convolutional architectures.
• To evaluate the DCAE model against established methods like DCT, PCA, and standard autoencoders in both reconstruction and speech recognition tasks.

Proposed method

• Employ a denoising convolutional autoencoder (DCAE) to learn compact, noise-robust representations of ultrasound tongue images in an unsupervised manner.
• Train the DCAE by corrupting input images with noise and optimizing for reconstruction of the original image, enhancing robustness to speckle and artifacts.
• Use a symmetric encoder-decoder architecture with strided convolutions and transposed convolutions to learn hierarchical spatial features.
• Extract features from the bottleneck layer of the DCAE, which are then used as input to a DNN-HMM acoustic model for speech recognition.
• Compare DCAE performance against DCT, deep autoencoder (DAE), denoising autoencoder (DAE), and standard convolutional autoencoder (CAE) using MSE and CW-SSIM metrics.
• Apply the feature extraction pipeline to the 2010 Silent Speech Interface Challenge dataset using Kaldi for recognition system implementation.

Experimental results

Research questions

• RQ1 Can a denoising convolutional autoencoder outperform traditional feature extraction methods like DCT and PCA in reconstructing ultrasound tongue images?
• RQ2 How does the DCAE model compare to standard autoencoders and convolutional autoencoders in terms of reconstruction accuracy and robustness to noise?
• RQ3 Does the DCAE-based feature representation lead to improved word error rates in silent speech recognition compared to state-of-the-art methods?
• RQ4 To what extent do the spatial invariance and noise-robustness of the DCAE architecture preserve speech-related articulatory dynamics in ultrasound sequences?

Key findings

• The DCAE model achieved the lowest Word Error Rate (WER) of 6.17% on the 2010 Silent Speech Interface Challenge, surpassing the state-of-the-art DCT-based method (6.45%).
• On the reconstruction task, DCAE achieved a test MSE of 71.28 and a CW-SSIM of 0.6577 with a 60-dimensional feature vector, outperforming DCT, DAE, and CAE models.
• The DCAE model demonstrated superior noise robustness, with a 1.5% reduction in MSE and a 0.012 increase in CW-SSIM compared to standard CAE under identical settings.
• DCAE-based features yielded a 6.24% WER with 30-dimensional features, indicating strong performance even with reduced dimensionality.
• Visual comparisons confirmed that DCAE reconstructed images with better preservation of tongue shape and reduced speckle artifacts compared to other models.
• The DCAE model showed consistent improvement over non-convolutional autoencoders, suggesting that convolutional inductive biases enhance feature quality in ultrasound imaging.