[Paper Review] MUSAN: A Music, Speech, and Noise Corpus

David Snyder, Guoguo Chen+1 more|arXiv (Cornell University)|Oct 28, 2015

Speech Recognition and SynthesisComputer Science5 references922 citations

TL;DR

This paper introduces MUSAN, a freely redistributable audio corpus of 109 hours of music, speech, and noise from public domain and Creative Commons sources, designed for training voice activity detection (VAD) and music/speech discrimination systems. Using GMM-based models on the corpus, the authors achieve competitive performance—especially in VAD, where a GMM+energy hybrid improves speaker recognition EER by up to 23.16% under low-resource conditions.

ABSTRACT

This report introduces a new corpus of music, speech, and noise. This dataset is suitable for training models for voice activity detection (VAD) and music/speech discrimination. Our corpus is released under a flexible Creative Commons license. The dataset consists of music from several genres, speech from twelve languages, and a wide assortment of technical and non-technical noises. We demonstrate use of this corpus for music/speech discrimination on Broadcast news and VAD for speaker identification.

Motivation & Objective

• To create a publicly available, legally redistributable audio corpus for training voice activity detection (VAD) and music/speech discrimination systems.
• To address the lack of legally redistributable raw audio datasets for music and speech, particularly those with clear licensing for commercial use.
• To provide a diverse, multi-lingual, and multi-genre dataset covering speech, music, and environmental noises to improve robustness in speech processing applications.
• To demonstrate the utility of the corpus through baseline experiments using GMM-based models on standard benchmarks like Broadcast News and NIST SRE 2010.
• To evaluate the impact of VAD quality on downstream speaker recognition performance under limited speech availability.

Proposed method

• The corpus was compiled from US Public Domain and Creative Commons-licensed sources, including Librivox (read speech), Jamendo, Free Music Archive, Incompetech, and HD Classical Music (music), and Free Sound and Sound Bible (noise).
• Audio is formatted as 16kHz mono WAV files, with metadata and licensing information stored in LICENSE and ANNOTATIONS files for each audio segment.
• For music/speech discrimination, GMMs were trained on 20 MFCCs with delta and acceleration features, using 4–128 components, and evaluated at equal error rate (EER) on Broadcast News data.
• For VAD, a hybrid system combines an energy-based VAD with a GMM-based VAD trained on speech, music, and noise subsets, using priors tuned on out-of-domain data.
• Speaker recognition experiments used an i-vector system with a GMM universal background model (UBM) and PLDA backend, with VAD decisions used to filter unvoiced frames.
• Performance was evaluated on the NIST SRE 2010 core set, with test utterances truncated to 1–60 seconds to simulate real-time processing constraints.

Experimental results

Research questions

• RQ1 Can a large-scale, legally redistributable audio corpus of music, speech, and noise be constructed from public domain and Creative Commons sources?
• RQ2 How does model performance on music/speech discrimination compare when trained on MUSAN versus the widely used but non-redistributable GTZAN dataset?
• RQ3 To what extent does a GMM-based VAD trained on MUSAN improve speaker recognition performance under limited speech availability?
• RQ4 Does the inclusion of diverse noise types and non-English speech enhance the robustness of VAD and discrimination systems?
• RQ5 How does the performance of a hybrid GMM+energy VAD compare to a baseline energy-only VAD in low-resource speaker recognition scenarios?

Key findings

• The MUSAN corpus contains 109 hours of audio, including 60 hours of speech (from Librivox and US government recordings), 42.5 hours of music (from multiple genres and sources), and 6 hours of diverse noise samples.
• GMM-based music/speech discrimination trained on MUSAN achieved an EER of 3.75% with 16 components, comparable to the 3.85% EER achieved by a model trained on the GTZAN dataset.
• The addition of a GMM-based VAD reduced speaker recognition EER by up to 23.16% when only 1 second of speech was available, compared to the baseline energy VAD.
• The GMM+energy VAD outperformed the energy-only VAD across all test durations, with the largest relative improvement (23.16%) observed at 1 second of speech.
• The corpus is fully redistributable under a flexible Creative Commons license, with all audio files properly attributed and licensed, enabling commercial use.
• The results demonstrate that MUSAN is a viable, legally sound alternative to existing datasets for training robust VAD and music/speech discrimination systems.