[论文解读] Smart speaker design and implementation with biometric authentication and advanced voice interaction capability

Bharath Sudharsan, Peter Corcoran+1 more|arXiv (Cornell University)|Jan 1, 2019

Speech Recognition and Synthesis参考文献 10被引用 16

一句话总结

本文提出了一款原型智能扬声器，集成生物识别面部认证与先进的语音交互功能，采用树莓派、ReSpeaker v2麦克风阵列和摄像头模块。通过结合基于128维深度度量网络的实时面部识别与Snowboy唤醒词检测及Alexa语音服务，该系统仅对经过认证的用户实现安全、免提激活，显著降低了未经授权访问的风险，同时保持低CPU使用率和全双工交互能力。

ABSTRACT

Advancements in semiconductor technology have reduced dimensions and cost while improving the performance and capacity of chipsets. In addition, advancement in the AI frameworks and libraries brings possibilities to accommodate more AI at the resource-constrained edge of consumer IoT devices. Sensors are nowadays an integral part of our environment which provide continuous data streams to build intelligent applications. An example could be a smart home scenario with multiple interconnected devices. In such smart environments, for convenience and quick access to web-based service and personal information such as calendars, notes, emails, reminders, banking, etc, users link third-party skills or skills from the Amazon store to their smart speakers. Also, in current smart home scenarios, several smart home products such as smart security cameras, video doorbells, smart plugs, smart carbon monoxide monitors, and smart door locks, etc. are interlinked to a modern smart speaker via means of custom skill addition. Since smart speakers are linked to such services and devices via the smart speaker user’s account. They can be used by anyone with physical access to the smart speaker via voice commands. If done so, the data privacy, home security and other aspects of the user get compromised. Recently launched, Tensor Cam’s AI Camera, Toshiba’s Symbio, Facebook’s Portal are camera-enabled smart speakers with AI functionalities. Although they are camera-enabled, yet they do not have an authentication scheme in addition to calling out the wake-word. This paper provides an overview of cybersecurity risks faced by smart speaker users due to lack of authentication scheme and discusses the development of a state-of-the-art camera-enabled, microphone arraybased modern Alexa smart speaker prototype to address these risks. Keywords: Alexa Voice Service, Snowboy hotword detection, Smart speaker design, Microphone array, ReSpeaker, Voice algorithms, Open CV, Smart speaker authentication.

研究动机与目标

• 解决由于缺乏强用户认证机制（仅依赖唤醒词激活）而导致的智能扬声器日益增长的网络安全风险。
• 设计并实现一种基于现成组件的低成本边缘智能扬声器原型，以提升安全性和隐私保护。
• 将生物识别面部识别与语音交互相结合，实现对智能扬声器服务的安全、个性化和无缝用户访问。
• 评估在资源受限的边缘设备（如树莓派）上同时集成面部识别、语音唤醒词检测与全双工音频处理的可行性与性能。

提出的方法

• 部署基于树莓派的智能扬声器原型，采用ReSpeaker v2实现4麦克风阵列，内置DSP用于噪声抑制与波束成形。
• 集成树莓派摄像头模块，捕获实时视频帧，利用深度度量网络生成128维面部嵌入，实现面部检测与识别。
• 使用注册用户面部图像数据集训练面部识别模型，将预计算的128维编码存储在.picle文件中，以在推理阶段实现快速比对。
• 采用Snowboy作为唤醒词引擎，实现对“Alexa”的检测，CPU开销低于8%，准确率高，显著减少误报与漏报。
• 连接至亚马逊Alexa语音服务（AVS）C++ SDK，实现生物识别认证成功后的云端语音交互。
• 实现视觉反馈系统，利用ReSpeaker的RGB LED环，在面部识别成功后变为绿色。

实验结果

研究问题

• RQ1 低成本、边缘计算的智能扬声器原型能否有效集成生物识别面部认证以防止未经授权访问？
• RQ2 在资源受限的硬件上，结合面部识别与语音唤醒词检测如何在不降低性能的前提下提升安全性？
• RQ3 背景噪声与声学回声对语音交互质量有何影响？在全双工系统中如何有效缓解？
• RQ4 128维深度度量网络在树莓派上多大程度上实现了准确且高效的面部识别？
• RQ5 自建智能扬声器原型能否在保持低功耗的同时，实现与Alexa的无缝、安全且个性化的交互？

主要发现

• 该原型通过在激活Alexa前强制进行面部识别，成功降低了未经授权访问的风险，消除了如“鹦鹉攻击”和“好奇儿童”误用等威胁。
• Snowboy在唤醒词检测中表现出极低的漏检率与高准确率，CPU使用率低于8%，在测试设置中优于其他唤醒词引擎。
• 通过使用轻量级深度度量网络，面部识别系统在树莓派上实现了实时性能，高效完成128维面部嵌入的计算与比对。
• ReSpeaker v2的DSP波束成形与噪声抑制功能显著提升了语音信号质量，即使在嘈杂环境中也增强了ASR识别准确率。
• 在唤醒词检测后，系统表现出0.5秒的延迟，该延迟可接受，且在音频失真发生时通过重启树莓派得以解决。
• 该原型成功实现了在生物识别认证后与Alexa的全双工交互，RGB LED环的视觉反馈确认了用户识别成功。