[论文解读] Using Deep Convolutional Networks for Gesture Recognition in American Sign Language

Vivek Bheda, Dianna Radpour|arXiv (Cornell University)|Oct 18, 2017

Hand Gesture Recognition Systems被引用 75

一句话总结

本文提出一种方法，该方法使用 deep convolutional networks 对 American Sign Language 的字母和数字图像进行分类。它展示了将 CNNs 应用于 ASL 手势识别。

ABSTRACT

In the realm of multimodal communication, sign language is, and continues to be, one of the most understudied areas. In line with recent advances in the field of deep learning, there are far reaching implications and applications that neural networks can have for sign language interpretation. In this paper, we present a method for using deep convolutional networks to classify images of both the the letters and digits in American Sign Language.

研究动机与目标

• Motivate the study of sign language interpretation with deep learning.
• Explore CNNs for image-based recognition of ASL gestures.
• Showcase a method to classify ASL letters and digits using CNNs.

提出的方法

• Apply deep convolutional networks to ASL gesture images.
• Train a CNN to classify images into ASL letters and digits.
• Discuss the potential of CNN-based gesture recognition for sign language interpretation.

实验结果

研究问题

• RQ1 Can deep convolutional networks accurately classify ASL hand gestures corresponding to letters and digits from images?
• RQ2 What is the viability of using CNNs for ASL gesture recognition in terms of recognition capability?
• RQ3 Do CNN-based approaches offer advantages for interpreting ASL gestures compared to traditional methods?

主要发现

• The authors present a method for using deep convolutional networks to classify images of ASL letters and digits.
• The work demonstrates the applicability of CNNs to ASL gesture recognition on image data.
• The paper discusses potential implications and applications of CNN-based ASL interpretation.