[Paper Review] Towards data-driven stroke rehabilitation via wearable sensors and deep learning

Aakash Kaku, Avinash Parnandi+4 more|arXiv (Cornell University)|Apr 14, 2020

Stroke Rehabilitation and Recovery23 references20 citations

TL;DR

This study proposes a data-driven approach to stroke rehabilitation dose measurement using wearable inertial measurement units (IMUs) and a deep learning model to automatically identify functional movement primitives—basic building blocks of rehabilitation activities. The method employs a convolutional neural network with physical-quantity-specific embeddings and instance normalization, achieving 70% average classification accuracy, enabling objective, granular tracking of therapeutic repetitions.

ABSTRACT

Recovery after stroke is often incomplete, but rehabilitation training may potentiate recovery by engaging endogenous neuroplasticity. In preclinical models of stroke, high doses of rehabilitation training are required to restore functional movement to the affected limbs of animals. In humans, however, the necessary dose of training to potentiate recovery is not known. This ignorance stems from the lack of objective, pragmatic approaches for measuring training doses in rehabilitation activities. Here, to develop a measurement approach, we took the critical first step of automatically identifying functional primitives, the basic building block of activities. Forty-eight individuals with chronic stroke performed a variety of rehabilitation activities while wearing inertial measurement units (IMUs) to capture upper body motion. Primitives were identified by human labelers, who labeled and segmented the associated IMU data. We performed automatic classification of these primitives using machine learning. We designed a convolutional neural network model that outperformed existing methods. The model includes an initial module to compute separate embeddings of different physical quantities in the sensor data. In addition, it replaces batch normalization (which performs normalization based on statistics computed from the training data) with instance normalization (which uses statistics computed from the test data). This increases robustness to possible distributional shifts when applying the method to new patients. With this approach, we attained an average classification accuracy of 70%. Thus, using a combination of IMU-based motion capture and deep learning, we were able to identify primitives automatically. This approach builds towards objectively-measured rehabilitation training, enabling the identification and counting of functional primitives that accrues to a training dose.

Motivation & Objective

• To develop an objective, scalable method for measuring training dose in stroke rehabilitation, which is currently imprecisely quantified by time-in-therapy or manual observation.
• To identify and classify functional movement primitives—minimal, purpose-specific motions—using kinematic data from wearable IMUs.
• To design a deep learning model that is robust to distributional shifts across patients and capable of processing multimodal sensor data (e.g., acceleration, angular velocity).
• To enable future dose-response research by providing a granular, automated count of therapeutic repetitions.
• To overcome limitations of prior methods that only classify motion as functional or nonfunctional, without identifying motion content.

Proposed method

• The study uses inertial measurement units (IMUs) worn on the upper body to capture high-resolution kinematic data (acceleration and angular velocity) during rehabilitation activities.
• Human labelers segmented and annotated the IMU data into functional primitives, creating a supervised dataset of 48 chronic stroke patients.
• A custom convolutional neural network (CNN) is designed with a separate embedding module for different physical quantities (e.g., acceleration, angular velocity) to preserve sensor-specific features.
• The model replaces batch normalization with instance normalization to improve robustness to distributional shifts during inference on new patients.
• The network is trained end-to-end to classify 5 predefined functional primitives: reach, transport, grasp, release, and idle.
• Classification is performed at a 10 ms temporal resolution, enabling fine-grained motion analysis.

Experimental results

Research questions

• RQ1 Can deep learning models accurately identify functional movement primitives from IMU data in chronic stroke patients?
• RQ2 Does using separate embeddings for different physical sensor quantities improve classification performance?
• RQ3 Does replacing batch normalization with instance normalization enhance model robustness across diverse patient populations?
• RQ4 Can this method provide a more granular and objective measure of rehabilitation training dose than time-in-therapy or functional/nonfunctional motion dichotomization?
• RQ5 What are the primary challenges in distinguishing similar primitives (e.g., reach vs. transport) and how can they be addressed?

Key findings

• The proposed deep learning model achieved an average classification accuracy of 70% across five functional primitives, significantly exceeding random chance (20%).
• The use of separate embeddings for different physical quantities in the sensor data improved model performance by preserving modality-specific features.
• Instance normalization enhanced model robustness to distributional shifts, particularly when applied to new patients not seen during training.
• The model struggled most with distinguishing reach from transport and idle from stabilization, suggesting limitations in capturing subtle grasp timing and hand posture from wrist-mounted IMUs.
• The model's performance declined for severely impaired patients, indicating a need for larger, more diverse datasets and improved generalization techniques.
• The study demonstrates the feasibility of using IMU data and deep learning to extract and count functional primitives, forming a foundation for objective, data-driven rehabilitation dosing.