[论文解读] Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation

Yi Luo, Zhuo Chen+1 more|arXiv (Cornell University)|Oct 14, 2019

Speech and Audio Processing参考文献 36被引用 50

一句话总结

引入 Dual-path RNN (DPRNN) 来通过交替的块内RNN和块间RNN 来建模极长序列，替代 TasNet 中的一维 CNN 以实现在更小模型上达到 WSJ0-2mix 的 state-of-the-art。

ABSTRACT

Recent studies in deep learning-based speech separation have proven the superiority of time-domain approaches to conventional time-frequency-based methods. Unlike the time-frequency domain approaches, the time-domain separation systems often receive input sequences consisting of a huge number of time steps, which introduces challenges for modeling extremely long sequences. Conventional recurrent neural networks (RNNs) are not effective for modeling such long sequences due to optimization difficulties, while one-dimensional convolutional neural networks (1-D CNNs) cannot perform utterance-level sequence modeling when its receptive field is smaller than the sequence length. In this paper, we propose dual-path recurrent neural network (DPRNN), a simple yet effective method for organizing RNN layers in a deep structure to model extremely long sequences. DPRNN splits the long sequential input into smaller chunks and applies intra- and inter-chunk operations iteratively, where the input length can be made proportional to the square root of the original sequence length in each operation. Experiments show that by replacing 1-D CNN with DPRNN and apply sample-level modeling in the time-domain audio separation network (TasNet), a new state-of-the-art performance on WSJ0-2mix is achieved with a 20 times smaller model than the previous best system.

研究动机与目标

• 激发在时域语音分离中对有效长序列建模的需求。
• 提出一种简单架构，将长序列分割成块，并交替应用局部和全局 RNN。
• 证明输入长度的亚线性增长（O(sqrt(L)）在保持性能的同时提升了优化效果。
• 展示在 TasNet 中用 DPRNN 替换 CNN 能以更小的模型获得最新的最优结果。

提出的方法

• 将长输入分割成重叠的块以形成一个3-D张量。
• 对每个块独立应用块内双向 RNN（局部建模）。
• 跨块应用块间 RNN 以捕捉全局依赖（话语级建模）。
• 在每个 DPRNN 块之后使用残差连接和层归一化。
• 将多个 DPRNN 块堆叠并采用重叠-相加来重建序列。

实验结果

研究问题

• RQ1 双路径 RNN 架构是否能有效地对极长序列进行时域语音分离建模？
• RQ2 在固定感受野的 CNN 基线之上，交替应用块内和块间处理是否能带来提升？
• RQ3 DPRNN 在模型规模和在 WSJ0-2mix 以及嘈杂混响条件下的性能影响如何？

主要发现

• DPRNN-TasNet 在 WSJ0-2mix 上相比 TCN-TasNet 将 SI-SNRi 提升 4.6%，且模型尺寸小 49%。
• 用 DPRNN 替代 1-D CNN 使更小的模型也能达到可比或更好的性能，当编码器窗口为 2 个样本且帧数超过 30k 时可达到最佳 SI-SNRi。
• DPRNN-TasNet 在 WSJ0-2mix 上以比 FurcaNeXt 小 20 倍的模型达到新的 SI-SNRi/SDRi state-of-the-art。
• 在嘈杂混响条件下，DPRNN 展现出显著的性能提升，SI-SNRi 更高、WER 更低，相较于基于 TCN 的模型。