[论文解读] DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing

Shuochao Yao, Shaohan Hu+3 more|arXiv (Cornell University)|Nov 7, 2016

Context-Aware Activity Recognition Systems参考文献 39被引用 105

一句话总结

DeepSense 将 CNNs 和 GRUs 集成，以处理嘈杂时间序列的移动传感数据上的回归和分类，在车载跟踪、异质活动识别以及生物识别用户识别方面取得了最先进的结果，同时在设备端部署仍具实用性。

ABSTRACT

Mobile sensing applications usually require time-series inputs from sensors. Some applications, such as tracking, can use sensed acceleration and rate of rotation to calculate displacement based on physical system models. Other applications, such as activity recognition, extract manually designed features from sensor inputs for classification. Such applications face two challenges. On one hand, on-device sensor measurements are noisy. For many mobile applications, it is hard to find a distribution that exactly describes the noise in practice. Unfortunately, calculating target quantities based on physical system and noise models is only as accurate as the noise assumptions. Similarly, in classification applications, although manually designed features have proven to be effective, it is not always straightforward to find the most robust features to accommodate diverse sensor noise patterns and user behaviors. To this end, we propose DeepSense, a deep learning framework that directly addresses the aforementioned noise and feature customization challenges in a unified manner. DeepSense integrates convolutional and recurrent neural networks to exploit local interactions among similar mobile sensors, merge local interactions of different sensory modalities into global interactions, and extract temporal relationships to model signal dynamics. DeepSense thus provides a general signal estimation and classification framework that accommodates a wide range of applications. We demonstrate the effectiveness of DeepSense using three representative and challenging tasks: car tracking with motion sensors, heterogeneous human activity recognition, and user identification with biometric motion analysis. DeepSense significantly outperforms the state-of-the-art methods for all three tasks. In addition, DeepSense is feasible to implement on smartphones due to its moderate energy consumption and low latency

研究动机与目标

• 推动在移动感知与计算中更好地利用嘈杂的时间序列传感数据。
• 提出一个统一的深度学习框架，能够同时处理回归和分类任务。
• 利用局部传感器交互、多模态融合和时间建模来学习物理/噪声表征以及鲁棒特征。
• 在能耗与延迟方面展示设备端可行性。

提出的方法

• 将输入传感测量分割成时间区间，并对每个区间转换到频域。
• 应用每个传感器的卷积子网以捕捉区间内局部交互和传感器之间的全局交互。
• 用合并卷积子网将传感器输出融合，以学习跨传感器关系。
• 使用两层堆叠的 GRU 来学习区间间的时序依赖，结合 dropout 和循环批量归一化。
• 对于回归，使用共享线性层解码输出，以估计每个区间的物理量。
• 对于分类，聚合区间特征（如平均）并应用 softmax 得到类别概率。

实验结果

研究问题

• RQ1 一个统一的 CNN-RNN 框架是否可以从嘈杂的移动传感数据中同时学习回归的物理信号/噪声组成以及分类的鲁棒特征表示？
• RQ2 带有每传感器 CNN、多模态融合和时序建模的分层感知是否在回归和分类任务上都优于针对具体任务的基线？
• RQ3 在能耗和延迟方面，该框架是否适合设备端部署？
• RQ4 DeepSense 在如车载跟踪、异质人体活动识别以及生物识别用户识别等挑战性任务上的表现如何？

主要发现

MAE（米）	地图辅助准确度
40.43±5.24	93.8%
44.97±5.80	90.2%
52.15±6.24	88.3%
53.06±6.59	87.5%
606.59±56.57
	6.7%

• 与基线相比，DeepSense 在车载跟踪（CarTrack）中显著降低跟踪误差，达到 40.43 ± 5.24 m 的 MAE 和 93.8% 的地图辅助准确率。
• 与变体相比，包含所有组件（独立卷积网、合并卷积网、堆叠 GRU）的 DeepSense 具有最佳性能，移除组件则会降级结果。
• 在 CarTrack 中，若干基线（DS-SingleGRU、DS-noIndvConv、DS-noMergeConv）显示出逐步恶化的 MAE 和地图辅助准确率，证实了对各传感器的处理以及跨传感器融合的价值。
• 在 HHAR（异质性人类活动识别）与 UserID（基于生物识别的运动识别）中，DeepSense 与其变体在很大程度上超越了最先进的基线，至少超出10%。
• 证明该框架可以在移动设备上实现，能耗中等且延迟低（设备端处理）。