[论文解读] SynergicLearning: Neural Network-Based Feature Extraction for Highly-Accurate Hyperdimensional Learning

Mahdi Nazemi, Amirhossein Esmaili+2 more|arXiv (Cornell University)|Jul 30, 2020

Ferroelectric and Negative Capacitance Devices参考文献 27被引用 4

一句话总结

本文提出 SynergicLearning，一种结合神经网络（NN）用于特征提取与超维（HD）计算分类器的混合机器学习模型，实现高精度（与纯神经网络相当）、相较于独立HD模型的更强鲁棒性，以及高效的片上部署。该框架的能效比优于最先进HD实现1.60倍，延迟降低2.13倍，且配备编译器与参数化硬件映射，支持实时、增量学习。

ABSTRACT

Machine learning models differ in terms of accuracy, computational/memory complexity, training time, and adaptability among other characteristics. For example, neural networks (NNs) are well-known for their high accuracy due to the quality of their automatic feature extraction while brain-inspired hyperdimensional (HD) learning models are famous for their quick training, computational efficiency, and adaptability. This work presents a hybrid, synergic machine learning model that excels at all the said characteristics and is suitable for incremental, on-line learning on a chip. The proposed model comprises an NN and a classifier. The NN acts as a feature extractor and is specifically trained to work well with the classifier that employs the HD computing framework. This work also presents a parameterized hardware implementation of the said feature extraction and classification components while introducing a compiler that maps any arbitrary NN and/or classifier to the aforementioned hardware. The proposed hybrid machine learning model has the same level of accuracy (i.e. $\pm$1%) as NNs while achieving at least 10% improvement in accuracy compared to HD learning models. Additionally, the end-to-end hardware realization of the hybrid model improves power efficiency by 1.60x compared to state-of-the-art, high-performance HD learning implementations while improving latency by 2.13x. These results have profound implications for the application of such synergic models in challenging cognitive tasks.

研究动机与目标

• 弥合高精度神经网络与超维（HD）学习模型计算效率之间的差距。
• 实现适用于增量学习与在线学习的端到端、片上混合模型部署。
• 开发一款编译器与参数化硬件架构，将任意神经网络与分类器映射至协同框架。

提出的方法

• 专门训练神经网络，以生成针对后续分类任务优化的特征，使用超维（HD）计算框架。
• 设计一种混合模型，其中神经网络作为特征提取器，HD分类器执行最终决策。
• 为特征提取与HD分类组件实现参数化硬件设计，以支持高效的片上执行。
• 开发一款编译器，将任意神经网络和/或分类器映射至所提出的硬件感知模型。
• 通过利用HD计算的固有高效性与硬件感知神经网络设计，优化系统以实现低功耗与低延迟。
• 将整个流水线集成至单一、紧凑且可扩展的硬件架构中，以支持实时推理。

实验结果

研究问题

• RQ1 结合神经网络与超维计算的混合模型是否能在保持纯神经网络精度的同时，达到HD模型的效率？
• RQ2 与独立的HD学习模型相比，协同设计如何提升准确率？
• RQ3 通过硬件感知编译与参数化实现，可实现多高的能效比与延迟降低？
• RQ4 所提出的编译器与硬件映射是否能够支持任意神经网络与分类器的灵活部署？
• RQ5 该模型在资源受限环境下的增量式、在线学习中效果如何？

主要发现

• 该混合模型的准确率与纯神经网络相当，误差范围在±1%以内，表明在与优秀训练的特征提取器配合时，基于HD的分类不会损害性能。
• 与独立的超维学习模型相比，该模型准确率至少提升10%，表明存在显著的协同性能增益。
• 端到端硬件实现的能效比优于最先进高性能HD学习实现1.60倍。
• 与现有最佳HD学习系统相比，硬件设计将延迟降低2.13倍。
• 所提出的编译器与参数化硬件可高效地将任意神经网络与分类器映射至目标架构，支持实时与片上部署。
• 由于计算开销低且具备可适应性，该框架非常适合用于增量学习与在线学习应用。