[Paper Review] Memory-Efficient, Limb Position-Aware Hand Gesture Recognition using Hyperdimensional Computing

Andy Zhou, Rikky Muller+1 more|arXiv (Cornell University)|Mar 9, 2021

Ferroelectric and Negative Capacitance Devices29 references18 citations

TL;DR

This paper proposes a memory-efficient, limb position-aware hand gesture recognition system using hyperdimensional (HD) computing that fuses EMG and accelerometer signals to emulate dual-stage classification without increasing memory footprint. By encoding accelerometer data into orthogonal context vectors, the method achieves 93.34% classification accuracy—17.79% higher than EMG-only models—while using 8× less memory than traditional dual-stage architectures.

ABSTRACT

Electromyogram (EMG) pattern recognition can be used to classify hand gestures and movements for human-machine interface and prosthetics applications, but it often faces reliability issues resulting from limb position change. One method to address this is dual-stage classification, in which the limb position is first determined using additional sensors to select between multiple position-specific gesture classifiers. While improving performance, this also increases model complexity and memory footprint, making a dual-stage classifier difficult to implement in a wearable device with limited resources. In this paper, we present sensor fusion of accelerometer and EMG signals using a hyperdimensional computing model to emulate dual-stage classification in a memory-efficient way. We demonstrate two methods of encoding accelerometer features to act as keys for retrieval of position-specific parameters from multiple models stored in superposition. Through validation on a dataset of 13 gestures in 8 limb positions, we obtain a classification accuracy of up to 93.34%, an improvement of 17.79% over using a model trained solely on EMG. We achieve this while only marginally increasing memory footprint over a single limb position model, requiring $8\ imes$ less memory than a traditional dual-stage classification architecture.

Motivation & Objective

• To address the reliability degradation in EMG-based gesture recognition caused by limb position changes.
• To reduce the memory overhead of dual-stage classification architectures that use separate models for each limb position.
• To enable context-aware gesture recognition in wearable devices with limited computational and memory resources.
• To develop a method for superposition of position-specific gesture classifiers using accelerometer-derived context vectors in hyperdimensional space.
• To evaluate the performance of two accelerometer-based context encoding methods for multi-context HD classification.

Proposed method

• Hyperdimensional (HD) computing is used to represent EMG features and gesture classes as high-dimensional hypervectors (HVs), enabling efficient nearest-neighbor classification.
• Accelerometer signals are encoded into context vectors using a continuous item memory (CIM) or orthogonal encoding to represent distinct limb positions.
• Context vectors are used as keys to retrieve position-specific gesture classifier prototypes stored in superposition within a single HD model.
• The method employs context-based orthogonalization, where orthogonal HVs for different limb positions minimize interference during superposition of multiple gesture models.
• Two encoding strategies are evaluated: one using predefined limb positions with orthogonal context vectors, and another using a genetic algorithm to optimize continuous accelerometer encoding for context vectors.
• The system integrates EMG and accelerometer data at inference time, using context vectors to dynamically select the appropriate gesture classifier from superposed prototypes.

Experimental results

Research questions

• RQ1 Can HD computing be used to achieve dual-stage classification performance in a memory-efficient manner for multi-limb-position gesture recognition?
• RQ2 How does context-based orthogonalization improve classification margin and accuracy in multi-context HD classification?
• RQ3 What is the performance trade-off between using predefined orthogonal context vectors and continuous accelerometer-based context encoding?
• RQ4 To what extent can accelerometer signals alone generate context vectors that reduce interference in superposed gesture models?
• RQ5 Can the proposed method maintain high accuracy across diverse limb positions while minimizing memory overhead compared to traditional dual-stage models?

Key findings

• The proposed method achieved a classification accuracy of 93.34% across 13 gestures in 8 limb positions, representing a 17.79% improvement over EMG-only models.
• The memory footprint increased by only a marginal amount compared to a single-limb-position model, requiring 8× less memory than a traditional dual-stage classification architecture.
• Accelerometer-based context encoding using a continuous item memory (CIM) improved accuracy by 13.61% on average compared to direct superposition of prototypes.
• The orthogonal context encoding method provided better performance than continuous encoding, but both outperformed baseline models trained without limb position awareness.
• The method demonstrated robustness to limb position variation, with consistent accuracy improvements across all 8 positions, especially in overlapping or similar positions like 0 and 4.
• The integration of accelerometer data into HD computing enabled dynamic, context-aware inference without separate limb position classification, reducing model complexity and implementation overhead.