[论文解读] Deep clustering: Discriminative embeddings for segmentation and separation

John R. Hershey, Zhuo Chen+2 more|arXiv (Cornell University)|Aug 18, 2015

Speech and Audio Processing参考文献 19被引用 7

一句话总结

本文提出深度聚类，一种与类别无关的深度学习框架，可为语音源分离学习判别性频谱图嵌入。通过训练深度网络生成能够实现理想亲和矩阵低秩逼近的嵌入，该方法使简单的聚类即可解码源分割，在仅使用双说话人数据训练的情况下，对未见说话人混合源实现了6 dB的SDR提升。

ABSTRACT

We address the problem of acoustic source separation in a deep learning framework we call "deep clustering." Rather than directly estimating signals or masking functions, we train a deep network to produce spectrogram embeddings that are discriminative for partition labels given in training data. Previous deep network approaches provide great advantages in terms of learning power and speed, but previously it has been unclear how to use them to separate signals in a class-independent way. In contrast, spectral clustering approaches are flexible with respect to the classes and number of items to be segmented, but it has been unclear how to leverage the learning power and speed of deep networks. To obtain the best of both worlds, we use an objective function that to train embeddings that yield a low-rank approximation to an ideal pairwise affinity matrix, in a class-independent way. This avoids the high cost of spectral factorization and instead produces compact clusters that are amenable to simple clustering methods. The segmentations are therefore implicitly encoded in the embeddings, and can be "decoded" by clustering. Preliminary experiments show that the proposed method can separate speech: when trained on spectrogram features containing mixtures of two speakers, and tested on mixtures of a held-out set of speakers, it can infer masking functions that improve signal quality by around 6dB. We show that the model can generalize to three-speaker mixtures despite training only on two-speaker mixtures. The framework can be used without class labels, and therefore has the potential to be trained on a diverse set of sound types, and to generalize to novel sources. We hope that future work will lead to segmentation of arbitrary sounds, with extensions to microphone array methods as well as image segmentation and other domains.

研究动机与目标

• 为解决基于类别的深度学习模型在分离未知或新型声音源方面的局限性，实现与类别无关的源分离。
• 结合深度网络的学习能力与谱聚类的灵活性，实现基于分割的分段。
• 开发一种可泛化至未见说话人及任意数量源的框架，且无需显式类别标签或重新训练。
• 实现端到端的嵌入学习，隐式编码分割信息，允许通过简单聚类进行解码。

提出的方法

• 训练一个深度双向LSTM网络，生成最小化理想成对亲和矩阵重构误差的频谱图嵌入。
• 采用对比损失目标，促使同一源的嵌入聚类更紧密，而不同源的嵌入彼此分离。
• 对理想亲和矩阵应用低秩逼近，以避免计算成本高昂的谱分解。
• 通过在学习到的嵌入上应用k-means聚类进行源分离解码，并对多个源使用排列对齐。
• 通过深度展开实现可微聚类目标，以实现嵌入与聚类的联合优化。
• 采用基于核的相似性度量（外积）而非局部核（如高斯核），以降低计算成本。

实验结果

研究问题

• RQ1 深度神经网络能否在无需显式类别标签的情况下学习到有效的判别性嵌入，以实现有效的源分离？
• RQ2 在仅使用双说话人混合数据训练的模型，能否在不重新训练的情况下泛化到三说话人混合？
• RQ3 所提出的方法在信噪比（SDR）方面是否优于传统基线模型（如最优NMF）？
• RQ4 该模型在训练数据中未出现的未见说话人上的泛化能力如何？
• RQ5 该框架能否扩展至其他领域，如图像分割或多麦克风系统？

主要发现

• 所提出的深度聚类方法在未见说话人混合上实现了6 dB的信噪比（SDR）提升，优于最优NMF基线模型。
• 尽管仅在双说话人数据上训练，该模型仍能泛化至三说话人混合，表明其可扩展至未知数量的源。
• 开放集性能（未见说话人）与封闭集性能（保留的训练说话人）相当，表明泛化能力出色。
• 即使在全语音段聚类下性能依然良好，表明嵌入捕捉到了如音高和音色等全局特征。
• 激活函数的选择（logistic与tanh）具有可测量的影响，其中logistic网络在测试配置中表现略优于tanh。
• 在女性-女性和男性-男性混合中性能显著优于同性别混合，可能由于谱对比度更高。