[Paper Review] Personalizing Smartwatch Based Activity Recognition Using Transfer Learning

Karanpreet Singh, Rajen B. Bhatt|arXiv (Cornell University)|Sep 3, 2019

Context-Aware Activity Recognition Systems12 references4 citations

TL;DR

This paper proposes a transfer learning approach that fine-tunes pre-trained Gradient Boosting Machines (GBM) on-device using minimal user-specific data, improving smartwatch-based activity recognition accuracy without retraining estimators. The method achieves up to 50% error reduction and F1-score increases from 0.9456 to 0.9758 on one dataset, significantly enhancing personalization for users with low baseline performance.

ABSTRACT

Smartwatches are increasingly being used to recognize human daily life activities. These devices may employ different kind of machine learning (ML) solutions. One of such ML models is Gradient Boosting Machine (GBM) which has shown an excellent performance in the literature. The GBM can be trained on available data set before it is deployed on any device. However, this data set may not represent every kind of human behavior in real life. For example, a ML model to detect elder and young persons running activity may give different results because of differences in their activity patterns. This may result in decrease in the accuracy of activity recognition. Therefore, a transfer learning based method is proposed in which user-specific performance can be improved significantly by doing on-device calibration of GBM by just tuning its parameters without retraining its estimators. Results show that this method can significantly improve the user-based accuracy for activity recognition.

Motivation & Objective

• Address the challenge of low accuracy in smartwatch-based activity recognition for specific users, especially elders, children, and patients with atypical movement patterns.
• Overcome the limitations of generic models trained on diverse datasets that fail to capture individual behavioral differences.
• Develop a lightweight, on-device personalization method that avoids retraining and reduces data collection burden for sensitive or hard-to-sample populations.
• Improve user-specific recognition accuracy using only a small amount of labeled user data through parameter tuning of a pre-trained GBM.

Proposed method

• Train a general-purpose GBM model on a large, diverse dataset of smartwatch sensor data (accelerometer and gyroscope) to establish a baseline model.
• Use on-device transfer learning to fine-tune only the GBM’s final prediction weights (not estimators) using a small amount of user-specific labeled data.
• Apply a hyperparameter tuning strategy that optimizes the GBM’s output scores by adjusting the contribution of each weak learner to the final prediction.
• Utilize a one-user-out cross-validation framework to evaluate personalization performance across subjects, ensuring robustness and generalization.
• Leverage the GBM’s sequential boosting mechanism to iteratively correct errors on user-specific data, improving class-specific accuracy.
• Implement the method on-device to preserve privacy and reduce computational overhead, enabling real-time personalization.

Experimental results

Research questions

• RQ1 Can on-device hyperparameter tuning of a pre-trained GBM significantly improve activity recognition accuracy for individual users without retraining?
• RQ2 To what extent does the proposed method reduce error rates compared to a baseline GBM trained on generic data?
• RQ3 How effective is the method in improving recognition accuracy for users with low baseline performance, particularly for challenging activity classes like 'Bike' or 'Walk'?
• RQ4 Can the method achieve meaningful performance gains using only a small amount of user-specific data, especially for populations with limited data availability?
• RQ5 Does the approach maintain or improve F1-score and AUC across multiple subjects and activity classes?

Key findings

• The tuned GBM achieved a 50.2% relative reduction in error rate for the 'Walk' class in subject #7, increasing accuracy from 73.65% to 94.80%.
• The overall F1-score improved from 0.9456 (baseline) to 0.9758 (tuned) on Data Set I, representing a 3.2% absolute increase.
• For Data Set II, the overall F1-score rose from 0.9307 to 0.9619, with subject #7 showing a dramatic F1 improvement from 0.8117 to 0.9671.
• The Area Under Curve (AUC) for subject #7’s 'Bike' class increased from 0.886 to 0.982 after tuning, indicating significantly better discrimination.
• The method achieved a 90.79% accuracy for 'Bike' in subject #8, up from 83.00% in the baseline model, demonstrating consistent gains across users.
• The approach achieved a 96.30% accuracy for the 'Bike' class and 97.57% for 'Walk' after tuning, compared to 90.14% and 94.20% in the baseline model.