[论文解读] Hand Gesture Recognition with Two Stage Approach Using Transfer Learning and Deep Ensemble Learning

Serkan Savaş, Atilla Ergüzen|arXiv (Cornell University)|Sep 20, 2023

Hand Gesture Recognition SystemsComputer Science被引用 3

一句话总结

该论文提出了一种两阶段的手势识别框架，结合了迁移学习与深度集成学习，在HG14数据集上实现了98.88%的准确率。通过微调预训练的VGG和MobileNet模型，并采用狄利克雷技术对预测结果进行集成，显著优于单个模型，证明了集成方法在人机交互应用中的有效性。

ABSTRACT

Human-Computer Interaction (HCI) has been the subject of research for many years, and recent studies have focused on improving its performance through various techniques. In the past decade, deep learning studies have shown high performance in various research areas, leading researchers to explore their application to HCI. Convolutional neural networks can be used to recognize hand gestures from images using deep architectures. In this study, we evaluated pre-trained high-performance deep architectures on the HG14 dataset, which consists of 14 different hand gesture classes. Among 22 different models, versions of the VGGNet and MobileNet models attained the highest accuracy rates. Specifically, the VGG16 and VGG19 models achieved accuracy rates of 94.64% and 94.36%, respectively, while the MobileNet and MobileNetV2 models achieved accuracy rates of 96.79% and 94.43%, respectively. We performed hand gesture recognition on the dataset using an ensemble learning technique, which combined the four most successful models. By utilizing these models as base learners and applying the Dirichlet ensemble technique, we achieved an accuracy rate of 98.88%. These results demonstrate the effectiveness of the deep ensemble learning technique for HCI and its potential applications in areas such as augmented reality, virtual reality, and game technologies.

研究动机与目标

• 通过深度学习提升人机交互（HCI）中手势识别的准确率。
• 评估预训练深度神经网络在包含14种手势类别的HG14数据集上的性能。
• 探索集成学习在结合多个高性能模型以实现手势识别方面的有效性。
• 展示所提方法在增强现实与虚拟现实等应用中的实际潜力。

提出的方法

• 在HG14数据集上使用迁移学习对22个预训练深度学习模型（包括VGGNet和MobileNet变体）进行微调。
• 选取表现最佳的四个模型——VGG16、VGG19、MobileNet和MobileNetV2——进行集成。
• 应用狄利克雷集成技术，融合基学习器的预测结果，优化融合过程。
• 采用两阶段方法：首先通过迁移学习训练模型，然后进行基于集成的预测融合。
• 使用HG14数据集测试集的标准准确率指标评估性能。

实验结果

研究问题

• RQ1 使用预训练模型进行迁移学习是否能在HG14数据集上实现高精度的手势识别？
• RQ2 哪些深度学习架构在HG14数据集上对手势分类表现最佳？
• RQ3 集成学习是否能显著提升识别准确率，超越单个模型？
• RQ4 基于狄利克雷的集成技术是否能有效融合多个高性能模型的预测结果？

主要发现

• 在所评估模型中，MobileNet实现了最高的单模型准确率，达到96.79%。
• VGG16和VGG19的准确率分别为94.64%和94.36%，表现出色。
• 结合四个表现最佳网络的深度集成模型达到98.88%的准确率，显著优于任一单个模型。
• 狄利克雷集成技术有效利用了多样化预测，降低了误差并提升了鲁棒性。