[论文解读] Implementation Of Tiny Machine Learning Models On Arduino 33 BLE For Gesture And Speech Recognition

V Viswanatha, A C Ramachandra+4 more|arXiv (Cornell University)|Jul 23, 2022

Hand Gesture Recognition Systems被引用 9

一句话总结

本文展示了在 Arduino Nano 33 BLE 上部署 TinyML 模型，实现基于设备端推理的实时手势与语音识别。利用 EdgeImpulse 进行模型训练与优化，系统通过 6 轴 IMU 实现手势检测，通过内置麦克风实现关键词识别，实现了低功耗、低延迟的实时分类，并提供响应式的 RGB 灯光反馈。

ABSTRACT

In this article gesture recognition and speech recognition applications are implemented on embedded systems with Tiny Machine Learning (TinyML). It features 3-axis accelerometer, 3-axis gyroscope and 3-axis magnetometer. The gesture recognition,provides an innovative approach nonverbal communication. It has wide applications in human-computer interaction and sign language. Here in the implementation of hand gesture recognition, TinyML model is trained and deployed from EdgeImpulse framework for hand gesture recognition and based on the hand movements, Arduino Nano 33 BLE device having 6-axis IMU can find out the direction of movement of hand. The Speech is a mode of communication. Speech recognition is a way by which the statements or commands of human speech is understood by the computer which reacts accordingly. The main aim of speech recognition is to achieve communication between man and machine. Here in the implementation of speech recognition, TinyML model is trained and deployed from EdgeImpulse framework for speech recognition and based on the keywords pronounced by human, Arduino Nano 33 BLE device having built-in microphone can make an RGB LED glow like red, green or blue based on keyword pronounced. The results of each application are obtained and listed in the results section and given the analysis upon the results.

研究动机与目标

• 在低功耗嵌入式系统上使用 TinyML 实现实时、设备端的手势与语音识别。
• 展示在资源受限的微控制器（如 Arduino Nano 33 BLE）上部署优化后的机器学习模型的可行性。
• 通过手势识别实现非语言人机交互，通过语音识别实现关键词触发响应。
• 验证 TinyML 模型在嵌入式应用中的性能与效率，实现极低延迟与功耗。

提出的方法

• 使用 EdgeImpulse 平台对手势与语音识别任务的 TinyML 模型进行训练与优化。
• 利用集成 3D 加速度计、陀螺仪与磁力计的 6 轴惯性测量单元（IMU）采集手部运动数据。
• 使用 Arduino Nano 33 BLE 的内置麦克风采集并处理人声输入。
• 将量化、优化后的机器学习模型直接部署至 Arduino Nano 33 BLE 微控制器，实现设备端推理。
• 实现实时分类，检测到的手势或关键词将触发预设的 RGB LED 反馈（红、绿、蓝）。
• 模型推理完全在微控制器上执行，无需依赖云端，确保低延迟与隐私安全。

实验结果

研究问题

• RQ1 能否在 Arduino Nano 33 BLE 上有效训练并部署 TinyML 模型，利用 IMU 数据实现实时手势识别？
• RQ2 能否在相同低功耗微控制器上实现低延迟、高准确率的基于关键词的语音识别？
• RQ3 在实际嵌入式部署中，系统在推理速度、功耗与响应性方面的表现如何？
• RQ4 EdgeImpulse 在支持端到端 TinyML 流程以实现嵌入式手势与语音识别方面效果如何？
• RQ5 系统能否在实时中可靠地将特定手势与语音关键词映射到不同的输出动作（如 RGB LED 颜色变化）？

主要发现

• 手势识别系统成功利用 6 轴 IMU 数据在 Arduino Nano 33 BLE 上实现实时手部动作分类，响应延迟极低。
• 语音识别模型通过机载麦克风实现了准确的关键词检测，能根据语音指令正确触发 RGB LED 反馈。
• 两个模型均成功部署于微控制器上，无需外部处理，证明了完整的设备端推理能力。
• 系统实现了低功耗与极低延迟，适用于电池供电及实时人机交互应用。
• EdgeImpulse 框架实现了高效的模型训练、量化与部署，显著简化了 TinyML 流程。
• 结果证实了在低成本、低功耗微控制器上使用 TinyML 实现实用手势与语音识别任务的可行性。