[Paper Review] sEMG-based Hand Gesture Recognition with Deep Learning

Marcello Zanghieri|arXiv (Cornell University)|Jun 19, 2023

Muscle activation and electromyography studies4 citations

TL;DR

This master's thesis presents the first deep learning application on the Unibo-INAIL sEMG dataset, using a 1D-CNN in PyTorch to classify hand gestures from surface EMG signals. Despite achieving 81.2% inter-posture accuracy and 75.9% inter-day accuracy—comparable to the RBF-SVM baseline—the study demonstrates that multi-posture and recent-data-prioritized training strategies significantly improve generalization, highlighting user adaptation and positioning 1D-CNNs as viable candidates for future HMI research.

ABSTRACT

Hand gesture recognition based on surface electromyographic (sEMG) signals is a promising approach for developing Human-Machine Interfaces (HMIs) with a natural control, such as intuitive robot interfaces or poly-articulated prostheses. However, real-world applications are limited by reliability problems due to motion artefacts, postural and temporal variability, and sensor re-positioning. This master thesis is the first application of deep learning on the Unibo-INAIL dataset, the first public sEMG dataset exploring the variability between subjects, sessions and arm postures by collecting data over 8 sessions of each of 7 able-bodied subjects executing 6 hand gestures in 4 arm postures. Recent studies address variability with strategies based on training set composition, which improve inter-posture and inter-day generalization of non-deep machine learning classifiers, among which the RBF-kernel SVM yields the highest accuracy. The deep architecture realized in this work is a 1d-CNN inspired by a 2d-CNN reported to perform well on other public benchmark databases. On this 1d-CNN, various training strategies based on training set composition were implemented and tested. Multi-session training proves to yield higher inter-session validation accuracies than single-session training. Two-posture training proves the best postural training (proving the benefit of training on more than one posture) and yields 81.2% inter-posture test accuracy. Five-day training proves the best multi-day training, yielding 75.9% inter-day test accuracy. All results are close to the baseline. Moreover, the results of multi-day training highlight the phenomenon of user adaptation, indicating that training should also prioritize recent data. Though not better than the baseline, the achieved classification accuracies rightfully place the 1d-CNN among the candidates for further research.

Motivation & Objective

• To evaluate deep learning, specifically 1D-CNNs, on the Unibo-INAIL sEMG dataset, the first public dataset capturing inter-subject, inter-session, and inter-posture variability.
• To address reliability issues in sEMG-based Human-Machine Interfaces (HMIs), such as motion artifacts, postural variability, and sensor repositioning.
• To investigate whether training set composition strategies—such as multi-posture and multi-day training—improve generalization in deep learning models compared to classical machine learning baselines.
• To determine if 1D-CNNs can outperform or match the performance of state-of-the-art classical models like RBF-kernel SVM on this challenging, real-world sEMG data.

Proposed method

• A 1D-CNN architecture was implemented in PyTorch, inspired by a 2D-CNN that performed well on other public benchmarks.
• Data preprocessing involved time-windowing of sEMG signals and a three-way split into training, validation, and test sets.
• Training strategies were based on composition of the training set: single-session, two-posture, and multi-day (5-day) training.
• The model used stochastic gradient descent with mini-batches, cross-entropy loss, L2 regularization, batch normalization, ReLU activation, and dropout for regularization.
• Inter-session and inter-day generalization was evaluated using held-out test sets from D6–D8, with performance measured via mean accuracy and standard error.
• User adaptation was analyzed by comparing multi-day training results, revealing decreasing inter-day variability over time.

Experimental results

Research questions

• RQ1 Can a 1D-CNN achieve competitive performance on the Unibo-INAIL sEMG dataset, which captures high variability across subjects, sessions, and arm postures?
• RQ2 Does multi-posture training improve inter-posture generalization compared to single-posture or single-session training?
• RQ3 Does multi-day training, particularly with recent data, improve inter-day generalization and reflect user adaptation over time?
• RQ4 How does the performance of the 1D-CNN compare to the RBF-kernel SVM baseline, the current state-of-the-art on this dataset?
• RQ5 Is the observed performance gap between the 1D-CNN and the baseline due to preprocessing limitations or inherent data constraints in the Unibo-INAIL dataset?

Key findings

• Multi-session training yielded higher inter-session validation accuracy than single-session training, indicating improved generalization across sessions.
• Two-posture training proved optimal for postural generalization, achieving 81.2% inter-posture test accuracy with a standard deviation of 7.3%.
• Five-day multi-day training was the best temporal generalization strategy, yielding 75.9% inter-day test accuracy with a standard deviation of 8.6%.
• The results were comparable to the RBF-SVM baseline, with validation accuracies of 83% (inter-posture) and 77% (inter-day), suggesting the 1D-CNN is competitive despite not surpassing the baseline.
• Multi-day training results revealed user adaptation, where inter-day differences in gesture execution decreased over time, indicating that recent data should be prioritized in training.
• The 1D-CNN achieved performance on par with classical models, positioning it as a strong candidate for future research, especially with improved preprocessing such as time-frequency analysis for 2D-CNNs.