[论文解读] ViT-HGR: Vision Transformer-based Hand Gesture Recognition from High Density Surface EMG Signals

Mansooreh Montazerin, Soheil Zabihi+3 more|arXiv (Cornell University)|Jan 25, 2022

Muscle activation and electromyography studies被引用 4

一句话总结

本文提出了一种基于视觉变换器（ViT）的框架ViT-HGR，用于高密度表面肌电信号（HD-sEMG）的手势识别，将HD-sEMG信号转换为与ViT架构兼容的分块输入格式。该方法在仅使用78,210个参数、无需数据增强或迁移学习的情况下从零开始训练，实现了65种手势的平均测试准确率84.62%，在准确率和训练效率方面均优于线性判别分析（LDA）。

ABSTRACT

Recently, there has been a surge of significant interest on application of Deep Learning (DL) models to autonomously perform hand gesture recognition using surface Electromyogram (sEMG) signals. DL models are, however, mainly designed to be applied on sparse sEMG signals. Furthermore, due to their complex structure, typically, we are faced with memory constraints; require large training times and a large number of training samples, and; there is the need to resort to data augmentation and/or transfer learning. In this paper, for the first time (to the best of our knowledge), we investigate and design a Vision Transformer (ViT) based architecture to perform hand gesture recognition from High Density (HD-sEMG) signals. Intuitively speaking, we capitalize on the recent breakthrough role of the transformer architecture in tackling different complex problems together with its potential for employing more input parallelization via its attention mechanism. The proposed Vision Transformer-based Hand Gesture Recognition (ViT-HGR) framework can overcome the aforementioned training time problems and can accurately classify a large number of hand gestures from scratch without any need for data augmentation and/or transfer learning. The efficiency of the proposed ViT-HGR framework is evaluated using a recently-released HD-sEMG dataset consisting of 65 isometric hand gestures. Our experiments with 64-sample (31.25 ms) window size yield average test accuracy of 84.62 +/- 3.07%, where only 78, 210 number of parameters is utilized. The compact structure of the proposed ViT-based ViT-HGR framework (i.e., having significantly reduced number of trainable parameters) shows great potentials for its practical application for prosthetic control.

研究动机与目标

• 为解决传统深度学习模型在处理高密度sEMG信号时面临的内存限制和长时间训练的问题。
• 探索视觉变换器（ViT）在分类复杂多通道HD-sEMG信号用于手势识别方面的可行性和有效性。
• 开发一种紧凑的端到端深度学习框架，直接在原始HD-sEMG数据上运行，无需复杂的特征工程或数据增强。
• 在包含19名参与者、65种等长手势的大规模HD-sEMG数据集上评估ViT-HGR的性能，重点关注准确率、参数效率和训练速度。

提出的方法

• 将HD-sEMG信号分割为64样本（31.25 ms）的窗口，并转换为适合视觉变换器输入的分块格式。
• 所提出的ViT-HGR框架采用标准视觉变换器架构，包含可学习的分类标记和位置嵌入，针对HD-sEMG信号的时空结构进行适配。
• 使用交叉熵损失从零开始端到端训练模型，不使用迁移学习或数据增强，利用自注意力机制建模电极间及时间维度上的长程依赖关系。
• 评估了三种不同深度和宽度的ViT-HGR模型变体，以研究模型复杂度与性能之间的权衡。
• 对每位受试者采用5折交叉验证策略，以确保在19名参与者的数据集上评估结果的稳健性。
• 将该框架与标准的线性判别分析（LDA）分类器进行对比，使用相同的5个关键sEMG特征（MAV、ZC、WL、RMS、SSC）和AR系数，以实现公平基准比较。

Figure 1: A graphical representation of the Vision Transformer: (a) A cropped portion of the original signal is fed to the ViT and converted to small patches. These patches go through a patch + position embedding layer and a class token is prepended to them. They are then inputted to the transformer

实验结果

研究问题

• RQ1 视觉变换器能否在无需数据增强或迁移学习的情况下，有效适配高密度表面肌电信号用于手势识别？
• RQ2 与传统的机器学习方法（如LDA）相比，基于ViT的模型在HD-sEMG数据上的准确率和训练效率表现如何？
• RQ3 在基于ViT的HD-sEMG框架中，模型复杂度（参数数量）与分类准确率之间的最优权衡是什么？
• RQ4 紧凑的ViT架构能否仅使用少量参数，在65种手势的大量类别上实现高准确率？

主要发现

• ViT-HGR框架在19名参与者的65种等长手势上实现了84.62% ± 3.07%的平均测试准确率，仅使用78,210个可训练参数。
• 该模型在平均准确率上比LDA基线高出约5%，并将信号处理和训练时间减少50%以上，从所有受试者的9.6小时降至4.4小时。
• 参数更少的模型II（78,210个参数）实现了与参数更多（约300,000个参数）的模型I相当的准确率（84.62%），表明高性能并不依赖于大模型复杂度。
• 与LDA相比，ViT-HGR框架在不同受试者间的准确率标准差更低，表明其在个体间具有更一致的性能表现。
• ViT-HGR框架在无需任何数据增强或迁移学习的情况下成功对HD-sEMG信号中的手势进行分类，证明了其鲁棒性和泛化能力。
• 结果表明，ViT非常适合用于HD-sEMG任务，因其能通过自注意力机制高效建模长程空间和时间依赖关系，支持并行计算并减少训练时间。

Figure 2: Accuracy boxplots of 3 different models of the ViT-HGR framework. Each boxplot represents the Interquartile Range for 19 subjects. The accuracy for each subject is the average accuracy after performing 5-fold cross validation.