[Paper Review] Joint gender and age estimation based on speech signals using x-vectors and transfer learning

Damian Kwaśny, Daria Hemmerling|arXiv (Cornell University)|Dec 2, 2020

Speech Recognition and Synthesis17 references9 citations

TL;DR

This paper proposes a joint gender and age estimation system using x-vectors and transfer learning, replacing the standard TDNN with the deeper QuartzNet architecture. Pretraining on VoxCeleb and Common Voice datasets significantly improves performance, achieving state-of-the-art results on TIMIT with a mean absolute error (MAE) of 5.12 years (male) and 5.29 years (female), and a gender classification accuracy of 99.6%.

ABSTRACT

In this paper we extend the x-vector framework for the task of speaker's age estimation and gender classification. In particular, we replace the baseline multilayer-TDNN architecture with QuartzNet, a convolutional architecture that has gained success in the field of speech recognition. We further propose a two-staged transfer learning scheme, utilizing large scale speech datasets: VoxCeleb and Common Voice, and usage of multitask learning to allow for joint age estimation and gender classification with a single system. We train and evaluate the performance on the TIMIT dataset. The proposed transfer learning scheme yields consecutive performance improvements in terms of both age estimation error and gender classification accuracy and the best performing system achieves new state-of-the-art results on the task of age estimation on the TIMIT TEST dataset with MAE of 5.12 and 5.29 years and RMSE of 7.24 and 8.12 years for male and female speakers respectively while maintaining a gender classification accuracy of 99.6%.

Motivation & Objective

• To improve the accuracy of joint speaker age estimation and gender classification using deep learning on speech signals.
• To replace the standard TDNN-based x-vector architecture with a deeper QuartzNet architecture for better feature representation.
• To investigate the effectiveness of transfer learning using large-scale speech datasets (VoxCeleb and Common Voice) for low-resource age and gender estimation.
• To achieve state-of-the-art performance on the TIMIT dataset for both age estimation and gender classification.
• To demonstrate that multitask learning with joint regression and classification improves model generalization and performance.

Proposed method

• The system uses a QuartzNet-based neural network as the x-vector embedder, replacing the traditional TDNN architecture.
• The embedder extracts fixed-size embeddings from variable-length speech utterances using convolutional blocks, residual connections, and statistics pooling (mean and standard deviation).
• A two-stage transfer learning scheme is applied: first pretraining on VoxCeleb for speaker recognition, then fine-tuning on TIMIT with joint age regression and gender classification.
• Additional pretraining on Common Voice further improves performance, especially for MFCC-based features.
• The front-end head consists of two fully connected branches: one for age regression and one for gender classification, trained jointly via multitask learning.
• The model is trained and evaluated using the NVIDIA NeMo framework on the TIMIT dataset with standard train-test splits.

Experimental results

Research questions

• RQ1 Can a deeper QuartzNet-based x-vector architecture outperform the standard TDNN-based x-vector in joint age and gender estimation?
• RQ2 How effective is transfer learning from large-scale speech datasets like VoxCeleb and Common Voice for improving low-resource age and gender estimation on TIMIT?
• RQ3 Does joint multitask learning for age regression and gender classification lead to performance gains compared to single-task approaches?
• RQ4 What is the impact of different input features (MFCCs vs. mel-spectrograms) on the final performance of the joint estimation system?
• RQ5 Can the proposed two-stage transfer learning scheme achieve state-of-the-art results on the TIMIT dataset for both age estimation and gender classification?

Key findings

• The proposed system achieves a new state-of-the-art mean absolute error (MAE) of 5.12 years for male speakers and 5.29 years for female speakers on the TIMIT test set.
• The root mean square error (RMSE) is reduced to 7.24 years for males and 8.12 years for females, representing a significant improvement over prior work.
• Gender classification accuracy reaches 99.6%, demonstrating robust performance even with joint training on age and gender.
• Pretraining on VoxCeleb alone improves performance over the baseline, with further gains from additional pretraining on Common Voice.
• The MFCC-based system with dual pretraining (VoxCeleb + Common Voice) achieves the best results, outperforming previous DNN-based and feature-engineering-based approaches.
• The QuartzNet-based embedder architecture consistently outperforms the standard TDNN-based x-vector in both age estimation and gender classification tasks.