[论文解读] An Interpretable Machine Vision Approach to Human Activity Recognition using Photoplethysmograph Sensor Data

Eoin Brophy, José Juan Dominguez Veiga+3 more|arXiv (Cornell University)|Dec 3, 2018

Non-Invasive Vital Sign Monitoring被引用 7

一句话总结

该论文提出了一种新颖的、可解释的人体活动识别（HAR）系统，仅使用腕戴光电容积脉搏波（PPG）传感器数据，通过将时间序列信号转换为缩放图像，利用卷积神经网络（CNN）进行分类。采用迁移学习结合Inception-v3与SVM对2048维特征进行处理，该方法实现了92.3%的平均准确率——超过结合PPG与惯性传感器的性能——并通过类别激活图（CAMs）和t-SNE可视化实现了可解释的人工智能。

ABSTRACT

The current gold standard for human activity recognition (HAR) is based on the use of cameras. However, the poor scalability of camera systems renders them impractical in pursuit of the goal of wider adoption of HAR in mobile computing contexts. Consequently, researchers instead rely on wearable sensors and in particular inertial sensors. A particularly prevalent wearable is the smart watch which due to its integrated inertial and optical sensing capabilities holds great potential for realising better HAR in a non-obtrusive way. This paper seeks to simplify the wearable approach to HAR through determining if the wrist-mounted optical sensor alone typically found in a smartwatch or similar device can be used as a useful source of data for activity recognition. The approach has the potential to eliminate the need for the inertial sensing element which would in turn reduce the cost of and complexity of smartwatches and fitness trackers. This could potentially commoditise the hardware requirements for HAR while retaining the functionality of both heart rate monitoring and activity capture all from a single optical sensor. Our approach relies on the adoption of machine vision for activity recognition based on suitably scaled plots of the optical signals. We take this approach so as to produce classifications that are easily explainable and interpretable by non-technical users. More specifically, images of photoplethysmography signal time series are used to retrain the penultimate layer of a convolutional neural network which has initially been trained on the ImageNet database. We then use the 2048 dimensional features from the penultimate layer as input to a support vector machine. Results from the experiment yielded an average classification accuracy of 92.3%. This result outperforms that of an optical and inertial sensor combined (78%) and illustrates the capability of HAR systems using...

研究动机与目标

• 探究仅依靠腕戴PPG传感器是否足以区分不同的人体活动。
• 通过消除对复杂惯性传感硬件的需求，简化可穿戴HAR系统。
• 开发一种可通过可视化可解释性技术让非技术用户理解的机器学习方法。
• 证明PPG信号中包含足够的运动相关伪影，可用于区分不同类型的体力活动。
• 验证迁移学习与混合深度学习-SVM分类方法在PPG生成图像表示上的有效性。

提出的方法

• 将原始PPG时间序列信号转换为缩放的二维图像，以实现深度学习的可视化表示。
• 通过微调预训练Inception-v3模型的倒数第二层，在PPG图像数据集上应用迁移学习。
• 从微调后的Inception-v3模型的全局平均池化层中提取2048维特征向量。
• 使用支持向量机（SVM）分类器对提取的特征进行最终活动预测。
• 生成类别激活图（CAMs），以可视化对分类决策贡献最大的PPG图像区域。
• 应用t-SNE降维技术，可视化不同活动类别间2048维特征向量的聚类情况。

实验结果

研究问题

• RQ1 仅依靠腕戴PPG传感器是否能提供足够信息以区分不同的人体活动？
• RQ2 基于PPG信号图的迁移学习图像分类方法是否优于结合PPG与惯性传感器的传统方法？
• RQ3 所得到的深度学习模型是否能通过CAMs等可视化技术实现可解释的预测？
• RQ4 激活图中不同类别间的差异如何反映不同类型活动下信号形态的变化？
• RQ5 该方法是否能通过可视化可解释性，使非专家用户理解并信任HAR系统的决策？

主要发现

• 所提出方法在四种活动类别（高强度、低强度、步行、跑步）上实现了92.3%的平均分类准确率。
• 该性能优于结合PPG与惯性传感器的系统，后者仅达到78%的准确率。
• 类别激活图（CAMs）清晰地定位了相关信号成分，如振幅与频率模式，证明了模型的可解释性。
• t-SNE可视化显示了各活动类别特征向量的明显聚类，表明在学习的特征空间中实现了有效分离。
• 该方法仅使用单一光学传感器即可实现心率监测与活动识别，降低了硬件复杂度。
• 结合XAI技术（如CAMs与t-SNE）提供了透明且用户友好的界面，用于解释模型决策。