[Paper Review] Wisture: RNN-based Learning of Wireless Signals for Gesture Recognition in Unmodified Smartphones

Mohamed Abudulaziz Ali Haseeb, Ramviyas Parasuraman|arXiv (Cornell University)|Jul 26, 2017

Indoor and Outdoor Localization Technologies20 references19 citations

TL;DR

Wisture proposes an LSTM-based RNN model that enables contactless gesture recognition on unmodified smartphones using only standard Wi-Fi RSSI measurements, leveraging artificially induced data traffic to enhance signal resolution. It achieves up to 94% accuracy in classifying three dynamic hand gestures without hardware or OS modifications, demonstrating feasibility for real-world smartphone deployment.

ABSTRACT

This paper introduces Wisture, a new online machine learning solution for recognizing touch-less dynamic hand gestures on a smartphone. Wisture relies on the standard Wi-Fi Received Signal Strength (RSS) using a Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN), thresholding filters and traffic induction. Unlike other Wi-Fi based gesture recognition methods, the proposed method does not require a modification of the smartphone hardware or the operating system, and performs the gesture recognition without interfering with the normal operation of other smartphone applications. We discuss the characteristics of Wisture, and conduct extensive experiments to compare its performance against state-of-the-art machine learning solutions in terms of both accuracy and time efficiency. The experiments include a set of different scenarios in terms of both spatial setup and traffic between the smartphone and Wi-Fi access points (AP). The results show that Wisture achieves an online recognition accuracy of up to 94% (average 78%) in detecting and classifying three hand gestures.

Motivation & Objective

• To develop a machine learning solution for contactless gesture recognition on standard smartphones without requiring hardware or OS modifications.
• To enable real-time gesture classification using only Wi-Fi Received Signal Strength (RSS) measurements from off-the-shelf devices.
• To overcome limitations of existing methods that require root access, custom firmware, or specialized hardware like CSI-enabled Wi-Fi chips.
• To validate the method across diverse spatial configurations and network traffic conditions to ensure robustness and generalization.
• To release a public dataset and source code to support reproducibility and future benchmarking in Wi-Fi-based gesture recognition.

Proposed method

• Utilizes Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNNs) to model temporal patterns in Wi-Fi RSSI signals for gesture classification.
• Employs artificial data traffic induction between the smartphone and Wi-Fi access point to increase signal sampling frequency and improve resolution.
• Applies thresholding filters and signal processing techniques to extract meaningful features from raw RSSI traces.
• Designs a real-time inference pipeline that runs directly on the smartphone without interfering with normal Wi-Fi operations.
• Trains the LSTM model on labeled RSSI sequences collected under various spatial and traffic conditions to generalize across scenarios.
• Uses a window-based approach to segment continuous RSSI streams into gesture-specific intervals for classification.

Experimental results

Research questions

• RQ1 Can Wi-Fi RSSI signals alone, without CSI or hardware modifications, support accurate and real-time contactless gesture recognition on standard smartphones?
• RQ2 How does artificial data traffic induction improve the temporal resolution and classification performance of gesture recognition using standard RSSI?
• RQ3 What is the performance of the LSTM-based model across diverse spatial layouts and varying network traffic conditions?
• RQ4 How does the system generalize to new gestures or different smartphone models without retraining?
• RQ5 Can a low-power preamble gesture mode be designed to reduce false positives and battery consumption during idle periods?

Key findings

• Wisture achieves an average online recognition accuracy of 78% across multiple scenarios, with a peak accuracy of 94% on specific datasets.
• The system maintains high performance even when tested on configurations different from the training data, demonstrating strong generalization.
• The Push and Pull gestures can be recognized with 97% accuracy even without induced traffic, indicating their suitability as low-power preamble gestures.
• Reducing the training dataset size to 10% of the original (100+ samples per gesture) results in a modest accuracy drop from 94% to 83%, showing potential for data-efficient learning.
• The method operates entirely on unmodified smartphones, without requiring root access, firmware changes, or disabling other Wi-Fi applications.
• The release of a public dataset (1000+ labeled gestures) and source code enables reproducibility and future benchmarking in Wi-Fi-based gesture recognition research.