[Paper Review] Exploring Contrastive Learning in Human Activity Recognition for Healthcare

Chi Ian Tang, Ignacio Perez-Pozuelo+2 more|arXiv (Cornell University)|Nov 23, 2020

Context-Aware Activity Recognition Systems24 references56 citations

TL;DR

The paper adapts SimCLR-style contrastive learning to sensor-based HAR and systematically studies how time-series augmentations affect performance, finding that certain augmentations, especially rotation, can yield strong gains over supervised training when fine-tuning.

ABSTRACT

Human Activity Recognition (HAR) constitutes one of the most important tasks for wearable and mobile sensing given its implications in human well-being and health monitoring. Motivated by the limitations of labeled datasets in HAR, particularly when employed in healthcare-related applications, this work explores the adoption and adaptation of SimCLR, a contrastive learning technique for visual representations, to HAR. The use of contrastive learning objectives causes the representations of corresponding views to be more similar, and those of non-corresponding views to be more different. After an extensive evaluation exploring 64 combinations of different signal transformations for augmenting the data, we observed significant performance differences owing to the order and the function thereof. In particular, preliminary results indicated an improvement over supervised and unsupervised learning methods when using fine-tuning and random rotation for augmentation, however, future work should explore under which conditions SimCLR is beneficial for HAR systems and other healthcare-related applications.

Motivation & Objective

• Motivate HAR as a health-critical task with limited labeled data.
• Investigate whether contrastive learning can improve representations for HAR over supervised methods.
• Adapt SimCLR to wearable sensor data with appropriate time-series augmentations.
• Quantify how different augmentation combinations affect HAR performance on MotionSense.

Proposed method

• Adapt SimCLR framework to HAR by using time-series transformations as positive pairs.
• Employ a lightweight base encoder (TPN) and a three-layer MLP projection head with NT-Xent loss.
• Use eight time-series transformations to generate augmented views for contrastive learning.
• Evaluate linear and fine-tuned downstream classifiers on MotionSense dataset.
• Pre-train with 200 epochs, batch size 512, using SGD with cosine decay, then assess with linear and fine-tuned protocols.

Experimental results

Research questions

• RQ1 Can contrastive learning improve HAR representations from unlabeled sensor data compared to fully supervised learning?
• RQ2 Which time-series augmentations and their combinations maximize HAR performance in linear and fine-tuned evaluations?
• RQ3 How does SimCLR-pretraining with sensor data compare to previous self-supervised and supervised HAR methods on MotionSense?

Key findings

• In linear evaluation, combining channel shuffling and permutation achieved an average F1 of 0.872.
• In fine-tuned evaluation, random rotation alone achieved the highest F1 of 0.942.
• Some augmentation combinations caused large performance drops, up to 0.278 F1 points in linear evaluation.
• Adapted SimCLR yielded up to a 0.020 gain over fully supervised models.
• Pure supervised and prior self-supervised HAR baselines yielded F1 scores around 0.922–0.923 in this setup.