[Paper Review] Using Deep Convolutional Networks for Gesture Recognition in American Sign Language

Vivek Bheda, Dianna Radpour|arXiv (Cornell University)|Oct 18, 2017

Hand Gesture Recognition Systems75 citations

TL;DR

This paper presents a method that uses deep convolutional networks to classify images of letters and digits in American Sign Language. It demonstrates applying CNNs to ASL gesture recognition.

ABSTRACT

In the realm of multimodal communication, sign language is, and continues to be, one of the most understudied areas. In line with recent advances in the field of deep learning, there are far reaching implications and applications that neural networks can have for sign language interpretation. In this paper, we present a method for using deep convolutional networks to classify images of both the the letters and digits in American Sign Language.

Motivation & Objective

• Motivate the study of sign language interpretation with deep learning.
• Explore CNNs for image-based recognition of ASL gestures.
• Showcase a method to classify ASL letters and digits using CNNs.

Proposed method

• Apply deep convolutional networks to ASL gesture images.
• Train a CNN to classify images into ASL letters and digits.
• Discuss the potential of CNN-based gesture recognition for sign language interpretation.

Experimental results

Research questions

• RQ1 Can deep convolutional networks accurately classify ASL hand gestures corresponding to letters and digits from images?
• RQ2 What is the viability of using CNNs for ASL gesture recognition in terms of recognition capability?
• RQ3 Do CNN-based approaches offer advantages for interpreting ASL gestures compared to traditional methods?

Key findings

• The authors present a method for using deep convolutional networks to classify images of ASL letters and digits.
• The work demonstrates the applicability of CNNs to ASL gesture recognition on image data.
• The paper discusses potential implications and applications of CNN-based ASL interpretation.