[Paper Review] TEMGNet: Deep Transformer-based Decoding of Upperlimb sEMG for Hand Gestures Recognition

Elahe Rahimian, Soheil Zabihi+4 more|arXiv (Cornell University)|Sep 25, 2021

Muscle activation and electromyography studiesEngineering28 references19 citations

TL;DR

This paper proposes TEMGNet, a Vision Transformer-based deep learning model that achieves state-of-the-art accuracy in upper-limb hand gesture recognition from surface EMG signals using only 65,713 trainable parameters—seven times fewer than prior models—while attaining 82.93% accuracy on the NinaPro DB2 dataset with a 300ms window, demonstrating superior performance and reduced structural complexity without pre-training or fine-tuning.

ABSTRACT

There has been a surge of recent interest in Machine Learning (ML), particularly Deep Neural Network (DNN)-based models, to decode muscle activities from surface Electromyography (sEMG) signals for myoelectric control of neurorobotic systems. DNN-based models, however, require large training sets and, typically, have high structural complexity, i.e., they depend on a large number of trainable parameters. To address these issues, we developed a framework based on the Transformer architecture for processing sEMG signals. We propose a novel Vision Transformer (ViT)-based neural network architecture (referred to as the TEMGNet) to classify and recognize upperlimb hand gestures from sEMG to be used for myocontrol of prostheses. The proposed TEMGNet architecture is trained with a small dataset without the need for pre-training or fine-tuning. To evaluate the efficacy, following the-recent literature, the second subset (exercise B) of the NinaPro DB2 dataset was utilized, where the proposed TEMGNet framework achieved a recognition accuracy of 82.93% and 82.05% for window sizes of 300ms and 200ms, respectively, outperforming its state-of-the-art counterparts. Moreover, the proposed TEMGNet framework is superior in terms of structural capacity while having seven times fewer trainable parameters. These characteristics and the high performance make DNN-based models promising approaches for myoelectric control of neurorobots.

Motivation & Objective

• To address the high parameter count and data hunger of existing DNN models for sEMG-based myoelectric control.
• To improve gesture recognition accuracy using attention mechanisms that capture long-range temporal dependencies in sEMG signals.
• To reduce model complexity and training cost by leveraging the parallelizable self-attention mechanism of Transformers.
• To demonstrate that Transformers can be effectively trained on small sEMG datasets without pre-training or fine-tuning.
• To develop a compact, high-performance model suitable for real-time prosthetic control applications.

Proposed method

• The proposed TEMGNet architecture is based on the Vision Transformer (ViT) framework, transforming sEMG signals into image-like patches for input.
• Each sEMG signal window is divided into non-overlapping patches of size 12×12, with learnable class and position embeddings added to the patch embeddings.
• The model uses multi-head self-attention mechanisms to capture long-range dependencies across temporal and spatial features in the sEMG data.
• Positional embeddings are learned during training to encode the sequential order of patches, enabling the model to understand temporal structure.
• The model is trained end-to-end on the NinaPro DB2 dataset without pre-training or fine-tuning, using cross-entropy loss for classification.
• Hyperparameter ablation studies were conducted on model depth (number of layers), model dimension (d), and patch size to optimize performance.

Experimental results

Research questions

• RQ1 Can a Vision Transformer-based architecture achieve high accuracy in sEMG-based hand gesture recognition with minimal trainable parameters?
• RQ2 Does the self-attention mechanism in Transformers outperform RNNs and CNNs in capturing temporal and spatial features from sEMG signals?
• RQ3 Can a Transformer model be effectively trained on small sEMG datasets without pre-training or fine-tuning?
• RQ4 How does model depth and dimension affect recognition accuracy and parameter efficiency in sEMG classification?
• RQ5 To what extent does the learned position embedding encode sequential relationships in sEMG signals?

Key findings

• TEMGNet achieved 82.93% recognition accuracy on the NinaPro DB2 dataset with a 300ms window, outperforming state-of-the-art models.
• For a 200ms window, the model achieved 82.05% accuracy, surpassing the 79.0% reported in prior work.
• The model with 65,713 trainable parameters reduced structural complexity by sevenfold compared to the best-performing baseline with 466,944 parameters.
• Increasing model depth beyond two layers did not improve accuracy, indicating diminishing returns beyond a certain depth.
• The Wilcoxon signed-rank test confirmed that the performance difference between Model 1 and Model 4 was statistically significant (p ≤ 1.00e-4).
• Visualization of position embeddings confirmed that the model learns to encode sequential relationships, with neighboring patches showing higher similarity in position embedding vectors.