[Paper Review] Multilayer bootstrap network for unsupervised speaker recognition

Xiao-Lei Zhang|arXiv (Cornell University)|Sep 21, 2015

Speech and Audio Processing15 references3 citations

TL;DR

This paper proposes a multilayer bootstrap network (MBN)-based unsupervised speaker recognition method that reduces high-dimensional supervectors from a universal background model (UBM) to a low-dimensional space via nonlinear dimensionality reduction, followed by clustering. The approach outperforms unsupervised PCA and k-means methods and approaches supervised LDA performance, demonstrating robustness to UBM and MBN hyperparameter settings.

ABSTRACT

We apply multilayer bootstrap network (MBN), a recent proposed unsupervised learning method, to unsupervised speaker recognition. The proposed method first extracts supervectors from an unsupervised universal background model, then reduces the dimension of the high-dimensional supervectors by multilayer bootstrap network, and finally conducts unsupervised speaker recognition by clustering the low-dimensional data. The comparison results with 2 unsupervised and 1 supervised speaker recognition techniques demonstrate the effectiveness and robustness of the proposed method.

Motivation & Objective

• To develop an unsupervised speaker recognition method that does not require manually labeled training data.
• To address the challenge of unsupervised speaker recognition under strict conditions: no manual labeling, minimal hyperparameter tuning, and robustness across varying data and modeling conditions.
• To improve performance over existing unsupervised methods like PCA and k-means clustering in speaker recognition tasks.
• To evaluate the effectiveness and robustness of the multilayer bootstrap network (MBN) in reducing high-dimensional supervectors for speaker clustering.

Proposed method

• The method first extracts d-dimensional supervectors from an unsupervised universal background model (UBM) trained on MFCC features.
• It applies multilayer bootstrap network (MBN) to reduce the supervector dimension from d to a lower dimension d̄ (d̄ ≪ d) through a layer-by-layer nonlinear dimensionality reduction process.
• Each MBN hidden layer performs k-centers clustering on randomly selected, sparse subsets of features, with outputs concatenated across clusterings to form the next layer’s input.
• The MBN training involves random feature selection (controlled by a = d̂/d), random center initialization (k), random reconstruction via cyclic shift (r = d′/d̂), and sparse representation via nearest-neighbor assignment.
• Finally, k-means or agglomerative clustering is applied to the low-dimensional MBN output to identify speakers, depending on whether the number of speakers is known.
• The method is evaluated using normalized mutual information (NMI) and classification accuracy, comparing against PCA, k-means, and LDA-based systems.

Experimental results

Research questions

• RQ1 Can the MBN-based method achieve high speaker recognition performance without requiring manually labeled training data?
• RQ2 How does the MBN-based method compare to unsupervised baselines (PCA and k-means) in terms of clustering accuracy and robustness?
• RQ3 To what extent is the MBN-based method robust to variations in UBM hyperparameters such as mixture number and EM iteration count?
• RQ4 How does the number of MBN hidden layers affect the final speaker recognition performance?

Key findings

• The MBN-based method outperforms both unsupervised PCA and k-means clustering methods across all tested UBM configurations, achieving higher normalized mutual information (NMI) scores.
• The method approaches the performance of the supervised LDA-based system, particularly under optimal settings, demonstrating strong effectiveness in unsupervised speaker recognition.
• The MBN-based method is significantly less sensitive to variations in UBM mixture number and EM iteration count than PCA or k-means, indicating superior robustness.
• Increasing the number of MBN hidden layers consistently improves recognition accuracy, with performance rising steadily up to five layers.
• The method maintains high performance across different output dimensions and is robust to hyperparameter settings, especially when compared to PCA and k-means, which show strong sensitivity to both UBM and dimensionality parameters.
• Visualizations using PCA and MBN show that MBN produces more compact and well-separated speaker clusters, especially under poor UBM conditions (e.g., 0 EM iterations).