[Paper Review] MFCC-based Recurrent Neural Network for Automatic Clinical Depression Recognition and Assessment from Speech

Emna Rejaibi, Ali Komaty+3 more|arXiv (Cornell University)|Sep 16, 2019

Emotion and Mood Recognition38 references246 citations

TL;DR

A MFCC-based LSTM framework detects depression from speech and estimates PHQ-8 severity, using data augmentation and transfer learning to overcome small datasets, achieving 76.27% validation accuracy for depression detection and RMSE 0.405 on binary tasks.

ABSTRACT

Clinical depression or Major Depressive Disorder (MDD) is a common and serious medical illness. In this paper, a deep recurrent neural network-based framework is presented to detect depression and to predict its severity level from speech. Low-level and high-level audio features are extracted from audio recordings to predict the 24 scores of the Patient Health Questionnaire and the binary class of depression diagnosis. To overcome the problem of the small size of Speech Depression Recognition (SDR) datasets, expanding training labels and transferred features are considered. The proposed approach outperforms the state-of-art approaches on the DAIC-WOZ database with an overall accuracy of 76.27% and a root mean square error of 0.4 in assessing depression, while a root mean square error of 0.168 is achieved in predicting the depression severity levels. The proposed framework has several advantages (fastness, non-invasiveness, and non-intrusion), which makes it convenient for real-time applications. The performances of the proposed approach are evaluated under a multi-modal and a multi-features experiments. MFCC based high-level features hold relevant information related to depression. Yet, adding visual action units and different other acoustic features further boosts the classification results by 20% and 10% to reach an accuracy of 95.6% and 86%, respectively. Considering visual-facial modality needs to be carefully studied as it sparks patient privacy concerns while adding more acoustic features increases the computation time.

Motivation & Objective

• Motivate objective depression detection and severity assessment from speech using MFCC features and RNNs.
• Address small dataset challenges with data augmentation and transfer learning from related emotion tasks.
• Evaluate performance on the DAIC-WOZ corpus and analyze gender, noise robustness, and generalization.”

Proposed method

• Extract MFCC features (60 coefficients) with first and second derivatives from preprocessed speech segments.
• Use a 3-layer LSTM network followed by two dense layers for binary depression classification (sigmoid) and multi-class PHQ-8 severity estimation (softmax).
• Normalize MFCC features using global z-score normalization across coefficients.
• Apply data augmentation (noise, pitch, shift, speed) to increase data diversity.
• Pretrain the model on an emotions recognition task (RAVDESS) and fine-tune on depression (transfer learning).
• Compare baseline, data augmentation, and transfer learning setups; evaluate both binary depression and 24-level severity predictions.

Experimental results

Research questions

• RQ1 Can MFCC-based RNNs accurately detect depression from speech alone?
• RQ2 How well can the model predict PHQ-8 severity levels from speech (binary vs. multi-class)?
• RQ3 Do data augmentation and transfer learning improve depression detection and severity estimation on DAIC-WOZ?
• RQ4 What is the impact of gender on model performance in depression recognition from speech?
• RQ5 How robust is the proposed system to noise and to generalization across datasets?

Key findings

• Baseline MFCC-based RNN achieves 67.61% validation accuracy for depression (binary) with RMSE 0.5057.
• Severity prediction (24 PHQ-8 classes) yields RMSE 0.168, outperforming the binary task by a substantial margin.
• Data augmentation raises validation accuracy to 74.0% and reduces RMSE to 0.4206 (binary task).
• Transfer learning pretrained on RAVDESS emotions and fine-tuned on DAIC-WOZ increases validation accuracy to 76.27% and RMSE to 0.4055 (binary task).
• F1 score for depression improves from 38% to 46% with transfer learning; gender analysis shows higher validation accuracy for females (85%) than males (83%).
• Model demonstrates noise robustness with 10% Gaussian noise causing an 8.3% drop in binary accuracy and stable performance up to 20% noise.