[Paper Review] QubitHD: A Stochastic Acceleration Method for HD Computing-Based Machine Learning

Samuel Bosch, Alexander Sanchez de la Cerda+3 more|arXiv (Cornell University)|Nov 27, 2019

Ferroelectric and Negative Capacitance Devices4 citations

TL;DR

QubitHD proposes a stochastic binarization method for Hyperdimensional (HD) computing-based machine learning that maintains high classification accuracy while significantly improving energy efficiency and training speed. Inspired by quantum measurement, it uses randomized binarization to stabilize training and reduce convergence time, achieving 65% better energy efficiency and 95% faster training than state-of-the-art HD methods on FPGA.

ABSTRACT

Machine Learning algorithms based on Brain-inspired Hyperdimensional(HD) computing imitate cognition by exploiting statistical properties of high-dimensional vector spaces. It is a promising solution for achieving high energy efficiency in different machine learning tasks, such as classification, semi-supervised learning, and clustering. A weakness of existing HD computing-based ML algorithms is the fact that they have to be binarized to achieve very high energy efficiency. At the same time, binarized models reach lower classification accuracies. To solve the problem of the trade-off between energy efficiency and classification accuracy, we propose the QubitHD algorithm. It stochastically binarizes HD-based algorithms, while maintaining comparable classification accuracies to their non-binarized counterparts. The FPGA implementation of QubitHD provides a 65% improvement in terms of energy efficiency, and a 95% improvement in terms of training time, as compared to state-of-the-art HD-based ML algorithms. It also outperforms state-of-the-art low-cost classifiers (such as Binarized Neural Networks) in terms of speed and energy efficiency by an order of magnitude during training and inference.

Motivation & Objective

• Address the trade-off between energy efficiency and classification accuracy in binarized Hyperdimensional (HD) computing-based machine learning.
• Overcome the limitations of existing binarization techniques that reduce accuracy and increase training time.
• Improve training stability and convergence speed in HD-based models without sacrificing classification performance.
• Enable practical deployment of HD computing on low-power, edge devices by enhancing energy efficiency and inference speed.
• Demonstrate superiority over both HD-based and non-HD lightweight classifiers (e.g., BNNs, MLPs) in energy efficiency and speed.

Proposed method

• Introduce stochastic binarization inspired by quantum measurement: floating-point hypervectors are stochastically mapped to binary values before training.
• Replace computationally expensive cosine similarity with Hamming distance for similarity checks, reducing computational cost.
• Use randomized binarization during training to prevent convergence to local minima and accelerate movement toward optimal solutions.
• Implement the algorithm on FPGA to enable high-speed training and inference with low energy consumption.
• Maintain high classification accuracy by leveraging the statistical properties of high-dimensional vectors even after binarization.
• Optimize the training process by reducing operations through binary-only computation after stochastic binarization, improving energy efficiency.

Experimental results

Research questions

• RQ1 Can stochastic binarization in HD computing preserve classification accuracy comparable to non-binarized models while improving energy efficiency?
• RQ2 Does introducing randomness during binarization improve training convergence speed and stability compared to deterministic binarization methods?
• RQ3 To what extent can QubitHD outperform state-of-the-art HD-based algorithms in terms of energy efficiency and training speed on FPGA?
• RQ4 How does QubitHD compare to non-HD lightweight classifiers like BNNs and MLPs in terms of energy efficiency and inference speed?
• RQ5 Can Hamming distance replace cosine similarity in HD computing without degrading classification performance?

Key findings

• QubitHD reduces the classification accuracy gap between binarized and non-binarized HD models by 38.8% on average.
• The algorithm achieves 30–50% faster convergence than QuantHD, due to stochastic binarization preventing local minima.
• FPGA implementation of QubitHD delivers a 65% improvement in energy efficiency and 95% reduction in training time compared to state-of-the-art HD algorithms.
• QubitHD provides 56× and 8× improvements in energy efficiency and speedup during training, respectively, over non-binarized HD models.
• Compared to BNNs, QubitHD reduces CPU training time by 101× and is 20× faster during inference on FPGA.
• QubitHD achieves similar classification accuracy to BNNs and MLPs (e.g., 95.3% on ISOLET, 94.1% on UCIHAR) while drastically outperforming them in energy efficiency and speed.