[Paper Review] Lightweight Transformer in Federated Setting for Human Activity Recognition

Ali Raza, Kim Phuc Tran+4 more|arXiv (Cornell University)|Oct 1, 2021

Context-Aware Activity Recognition Systems4 citations

TL;DR

This paper proposes a lightweight one-patch Transformer model, TransFed, for human activity recognition (HAR) in home healthcare using federated learning to enhance privacy. It achieves 99.14% accuracy in centralized and 98.74% in federated settings on a newly collected dataset, outperforming state-of-the-art CNN and RNN methods while reducing computational cost.

ABSTRACT

Human activity recognition (HAR) is a machine learning task with important applications in healthcare especially in the context of home care of patients and older adults. HAR is often based on data collected from smart sensors, particularly smart home IoT devices such as smartphones, wearables and other body sensors. Deep learning techniques like convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been used for HAR, both in centralized and federated settings. However, these techniques have certain limitations: RNNs cannot be easily parallelized, CNNs have the limitation of sequence length, and both are computationally expensive. Moreover, in home healthcare applications the centralized approach can raise serious privacy concerns since the sensors used by a HAR classifier collect a lot of highly personal and sensitive data about people in the home. In this paper, to address some of such challenges facing HAR, we propose a novel lightweight (one-patch) transformer, which can combine the advantages of RNNs and CNNs without their major limitations, and also TransFed, a more privacy-friendly, federated learning-based HAR classifier using our proposed lightweight transformer. We designed a testbed to construct a new HAR dataset from five recruited human participants, and used the new dataset to evaluate the performance of the proposed HAR classifier in both federated and centralized settings. Additionally, we use another public dataset to evaluate the performance of the proposed HAR classifier in centralized setting to compare it with existing HAR classifiers. The experimental results showed that our proposed new solution outperformed state-of-the-art HAR classifiers based on CNNs and RNNs, whiling being more computationally efficient.

Motivation & Objective

• Address privacy concerns in home healthcare by replacing centralized data collection with federated learning.
• Overcome limitations of RNNs (poor parallelization) and CNNs (sequence length constraints) in HAR using a novel lightweight Transformer architecture.
• Develop a computationally efficient HAR model that maintains high accuracy without relying on large-scale centralized data.
• Construct a new, publicly released HAR dataset from five participants performing 15 activities in a simulated home environment.
• Evaluate the model’s robustness under non-IID data distributions typical in real-world federated settings.

Proposed method

• Propose a one-patch lightweight Transformer architecture that reduces parameters and computation by processing only a single patch of input sequence.
• Design TransFed, a federated learning framework integrating the lightweight Transformer for HAR, enabling model training across decentralized devices without sharing raw sensor data.
• Implement a testbed using accelerometers, gyroscopes, and magnetometers placed at various body locations to identify points of maximum impulse (PMIs) for optimal data collection.
• Use a non-IID data distribution in federated training to simulate realistic client data skew and assess model generalization.
• Apply standard federated averaging (FedAvg) with FedProx for convergence stability under non-IID conditions.
• Train and evaluate the model on both a newly collected dataset and the public WISDM dataset for cross-validation of performance.

Experimental results

Research questions

• RQ1 Can a lightweight one-patch Transformer achieve higher HAR accuracy than state-of-the-art CNN and RNN models while being computationally efficient?
• RQ2 How does the proposed TransFed framework perform in a federated learning setting with non-IID data distributions?
• RQ3 What is the impact of sensor placement (e.g., wrist, chest, thigh) on HAR performance, and which locations yield the most informative signals?
• RQ4 Can the proposed method maintain high accuracy and privacy in real-world home healthcare scenarios with limited participants and diverse data distributions?
• RQ5 How does the model’s performance compare to existing HAR methods when evaluated on both public and newly collected datasets?

Key findings

• The proposed lightweight Transformer achieved 99.14% accuracy in centralized training on the newly collected dataset, surpassing existing CNN and RNN-based methods.
• In federated learning, the model reached 98.74% accuracy on the same dataset, demonstrating strong performance under decentralized, non-IID data conditions.
• On the WISDM public dataset, the model achieved 98.89% accuracy in centralized evaluation, outperforming prior state-of-the-art methods.
• The model demonstrated superior computational efficiency due to its one-patch architecture, reducing parameter count and inference cost compared to standard Transformers.
• Sensor placement significantly influenced performance, with optimal locations (PMIs) identified for each sensor type, improving data quality and classification accuracy.
• The framework showed robustness to non-IID data, though performance varied depending on client data distribution, highlighting the need for adaptive aggregation strategies in real deployments.