[论文解读] Large-scale Training of Foundation Models for Wearable Biosignals

Salar Abbaspourazad, Oussama Elachqar+4 more|arXiv (Cornell University)|Dec 8, 2023

Mobile Health and mHealth Applications被引用 20

一句话总结

原作者在 Apple Heart and Movement Study 的大量无标签 PPG 与 ECG 数据上进行自监督预训练，使用参与者级正样本，得到嵌入能够编码人口统计与健康信息，并且在 PPG 与 ECG 的效用上存在差异。

ABSTRACT

Tracking biosignals is crucial for monitoring wellness and preempting the development of severe medical conditions. Today, wearable devices can conveniently record various biosignals, creating the opportunity to monitor health status without disruption to one's daily routine. Despite widespread use of wearable devices and existing digital biomarkers, the absence of curated data with annotated medical labels hinders the development of new biomarkers to measure common health conditions. In fact, medical datasets are usually small in comparison to other domains, which is an obstacle for developing neural network models for biosignals. To address this challenge, we have employed self-supervised learning using the unlabeled sensor data collected under informed consent from the large longitudinal Apple Heart and Movement Study (AHMS) to train foundation models for two common biosignals: photoplethysmography (PPG) and electrocardiogram (ECG) recorded on Apple Watch. We curated PPG and ECG datasets from AHMS that include data from ~141K participants spanning ~3 years. Our self-supervised learning framework includes participant level positive pair selection, stochastic augmentation module and a regularized contrastive loss optimized with momentum training, and generalizes well to both PPG and ECG modalities. We show that the pre-trained foundation models readily encode information regarding participants' demographics and health conditions. To the best of our knowledge, this is the first study that builds foundation models using large-scale PPG and ECG data collected via wearable consumer devices $\unicode{x2013}$ prior works have commonly used smaller-size datasets collected in clinical and experimental settings. We believe PPG and ECG foundation models can enhance future wearable devices by reducing the reliance on labeled data and hold the potential to help the users improve their health.

研究动机与目标

• 解决缺乏用于鲁棒模型开发的大规模、带注释的生物信号数据集的问题。
• 在来自可穿戴设备的大规模无标签 PPG 与 ECG 数据上训练基础模型。
• 研究参与者级嵌入揭示的人口统计和健康状况信息。
• 评估自监督学习技术和增强策略对生物信号表征的影响。

提出的方法

• 使用带 InfoNCE 的对比自监督学习与 KoLeo 正则化来学习嵌入。
• 使用随机的、参与者级的增强模块，从同一参与者的不同片段生成正样本对。
• 将动量训练应用于编码器和投影头以稳定表征。
• 使用基于 EfficientNet 的一维编码器（256 维嵌入）和 128 维投影头用于 SSL 损失。
• 在通过 Apple Watch 收集的约 2000 万个 PPG 片段（来自 141k 参与者）以及约 375 万个 ECG 片段（来自 106k 参与者）上进行预训练。
• 在下游人口统计和健康相关目标上评估线性探针，以评估嵌入中编码的信息。

实验结果

研究问题

• RQ1 可穿戴生物信号的大规模自监督预训练在没有标签的情况下是否能产生有信息量的嵌入？
• RQ2 PPG 与 ECG 的嵌入是否编码除了基本心率信息之外的参与者人口统计和健康状况？
• RQ3 在本框架下，PPG 与 ECG 在信息内容和预训练动态方面有何差异？
• RQ4 正样本对选择与增强选项对下游性能有何影响？
• RQ5 不同的编码器架构如何影响学习到的表征的质量与效率？

主要发现

• 在线性探针中，PPG 嵌入几乎总是优于基线人口统计特征，在预测年龄、BMI 和生物性别方面。
• ECG 嵌入编码健康信息，但对下游健康目标的预测能力相对低于 PPG 嵌入。
• 预训练的嵌入包含可预测大量自报健康状况和药物信息的信息。
• 参与者级正样本对选择显著提升下游预测性能，相比片段级配对。
• KoLeo 正则化与对比/自监督框架提升表征质量；移除 KoLeo 会恶化下游指标。
• 不同的编码器架构也能达到相似性能，其中 1D-EfficientNet 在参数效率方面具有优势。