[论文解读] Learning from Hypervectors: A Survey on Hypervector Encoding

Sercan Aygün, Mehran Shoushtari Moghadam+2 more|arXiv (Cornell University)|Aug 1, 2023

Ferroelectric and Negative Capacitance Devices被引用 14

一句话总结

关于 Hyperdimensional Computing 中 hypervectors 如何生成和编码的聚焦综述，考察数据映射、编码方法以及硬件考虑对准确性和效率的影响。

ABSTRACT

Hyperdimensional computing (HDC) is an emerging computing paradigm that imitates the brain's structure to offer a powerful and efficient processing and learning model. In HDC, the data are encoded with long vectors, called hypervectors, typically with a length of 1K to 10K. The literature provides several encoding techniques to generate orthogonal or correlated hypervectors, depending on the intended application. The existing surveys in the literature often focus on the overall aspects of HDC systems, including system inputs, primary computations, and final outputs. However, this study takes a more specific approach. It zeroes in on the HDC system input and the generation of hypervectors, directly influencing the hypervector encoding process. This survey brings together various methods for hypervector generation from different studies and explores the limitations, challenges, and potential benefits they entail. Through a comprehensive exploration of this survey, readers will acquire a profound understanding of various encoding types in HDC and gain insights into the intricate process of hypervector generation for diverse applications.

研究动机与目标

• 澄清如何从各种数据类型（数值、二维、序列等）生成 HV，以及这如何影响模型性能。
• 综述现有 HV 编码技术（基于记录、N-gram、稀疏/密集、基于 LD 的编码）及其权衡。
• 强调 HV 编码的硬件和边缘学习影响。
• 识别在多样应用中的 HV 生成所面临的挑战与开放研究机会。

提出的方法

• 回顾并对以往 HDC 文献中的 HV 映射和编码技术进行分类。
• 比较不同数据表示下 HV 的正交性、密度和稀疏性的考虑。
• 讨论 HV 生成的硬件影响与优化策略（FPGA、CIM 等）。
• 总结不同编码方案如何影响学习任务的准确性和效率。

Figure 1: A general overview of an HDC system targeting classification. (a) From applications to learning and classifying steps: ⓐ Encoding the incoming data. ⓑ Training using the whole bunch of input dataset, ⓒ Similarity check between class-based $\boldsymbol{\mathcal{HV}}$ s ( $C_{class}$ ) and q

实验结果

研究问题

• RQ1 存在于不同数据类型（数值、图像、序列）中的 HV 生成策略是什么，它们如何影响编码质量？
• RQ2 编码选择（正交性、密度、稀疏性、LD/分布）如何影响 HDC 的准确性和硬件效率？
• RQ3 基于记录和基于 N-gram 的 HV 生成方法在各应用中的主要权衡是什么？
• RQ4 当前的硬件考虑因素与支持大规模 HV 编码的平台，尤其是边缘学习方面？
• RQ5 在新兴应用和数据模态下，HV 编码还存在哪些未解决的挑战？

主要发现

• HV 生成质量和向量分布对 HDC 的准确性和效率具有关键影响。
• 多种编码方案（基于记录、N-gram、随机、LD 基）在准确性、硬件成本和可扩展性之间提供权衡。
• 稀疏与密集的 HV 表征对硬件效率有影响，并可针对数据类型（数值与符号）进行定制。
• 面向硬件的优化（FPGA、CIM、片上学习）相较于基线架构可带来显著的面积和吞吐提升。
• 在 DNA 处理和某些数据模态的 HV 编码方面存在显著的研究空白，提示未来工作机会。

Figure 2: The scatter plots of various sequences illustrate the distribution of each sequence in a 2D sample space consisting of 1024 points. The irrational numbers utilized for the Weyl sequence were $\pi$ and the Silver Ratio ( $\sqrt{2}-1$ ). The R2 sequence was generated using the Plastic Consta