[论文解读] Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency

Xiang Zhang, Ziyuan Zhao+2 more|ArXiv.org|Jun 17, 2022

EEG and Brain-Computer Interfaces被引用 127

一句话总结

引入 Time-Frequency Consistency (TF-C)，用于时间序列的自监督预训练，通过对齐基于时间的和基于频率的嵌入，提升对未见目标数据集的迁移能力。

ABSTRACT

Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short-cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need examples directly from the target domain, making them suboptimal for pre-training. To address this challenge, methods need to accommodate target domains with different temporal dynamics and be capable of doing so without seeing any target examples during pre-training. Relative to other modalities, in time series, we expect that time-based and frequency-based representations of the same example are located close together in the time-frequency space. To this end, we posit that time-frequency consistency (TF-C) -- embedding a time-based neighborhood of an example close to its frequency-based neighborhood -- is desirable for pre-training. Motivated by TF-C, we define a decomposable pre-training model, where the self-supervised signal is provided by the distance between time and frequency components, each individually trained by contrastive estimation. We evaluate the new method on eight datasets, including electrodiagnostic testing, human activity recognition, mechanical fault detection, and physical status monitoring. Experiments against eight state-of-the-art methods show that TF-C outperforms baselines by 15.4% (F1 score) on average in one-to-one settings (e.g., fine-tuning an EEG-pretrained model on EMG data) and by 8.4% (precision) in challenging one-to-many settings (e.g., fine-tuning an EEG-pretrained model for either hand-gesture recognition or mechanical fault prediction), reflecting the breadth of scenarios that arise in real-world applications. Code and datasets: https://github.com/mims-harvard/TFC-pretraining.

研究动机与目标

• 在预训练数据与目标数据之间存在领域差异的情况下，推动时间序列的稳健预训练。
• 提出 TF-C 作为一种可泛化的预训练原则，在预训练阶段不需要目标域数据。
• 开发一个可分解的模型，包含时间编码器和频率编码器以及跨空间投射器，以强化 TF-C。
• 引入基于时间和基于频率的对比损失，以及一种一致性（类三元组）损失，以融合表示。
• 证明相对于最先进基线，在多样数据集和任务上实现的迁移提升。

提出的方法

• 为时间域和频率域定义两个并行编码器，并引入跨空间投射到共享的时–频空间。
• 应用一组基于时间的增强，以及扰动谱的频率增强策略。
• 使用 NT-Xent 风格的对比损失分别对齐基于时间和基于频率的表示。
• 引入受三元组启发的一致性损失，促使跨域的时–频表示更加接近。
• 将损失合成为一个 TF-C 目标，平衡对比项和一致性项（L_TF-C = λ(L_T + L_F) + (1−λ)L_C）。

实验结果

研究问题

• RQ1 在没有目标域数据的情况下，是否可以在共享潜在空间中对齐同一时间序列的时间基表示和频率基表示？
• RQ2 频域增强和 TF-C 一致性目标是否能提升对未见目标数据集的迁移？
• RQ3 在多样化的时间序列任务上，TF-C 相对于最先进的自监督基线的表现如何？
• RQ4 TF-C 预训练在一对一和一对多迁移设置中是否都有益？
• RQ5 所提出的频率扰动对表示鲁棒性有何影响？

主要发现

• 在一对一迁移设置中，TF-C 在平均 F1 分数上比所有基线高出 15.4%。
• 在具有挑战性的一对多迁移设置中，TF-C 将精准度提升了 8.4%。
• 该方法在包括 EEG、EMG、ECG、步态和振动信号在内的八个数据集上显示出强大的迁移能力。
• 该模型采用四组件架构（时间编码器、频率编码器，以及两个跨空间投射器）以嵌入到共享的时–频空间。
• 频域增强在时序对比学习中是有效且新颖的。