[论文解读] Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human EEG.

Robin Tibor Schirrmeister, Jost Tobias Springenberg+7 more|arXiv (Cornell University)|Mar 15, 2017

EEG and Brain-Computer Interfaces被引用 63

一句话总结

本研究提出端到端的深度卷积神经网络（ConvNets），直接从原始人类脑电图（EEG）解码与运动相关的信息，采用具备批量归一化和指数线性单元（ELUs）的先进架构。该方法在性能上与标准的FBCSP算法相当或更优，而新颖的可视化技术揭示，ConvNets能够利用α、β和高频γ波段的谱功率调制，并映射其空间贡献以实现运动解码。

ABSTRACT

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, i.e. learning from the raw data. Now, there is increasing interest in using deep ConvNets for end-to-end EEG analysis. However, little is known about many important aspects of how to design and train ConvNets for end-to-end EEG decoding, and there is still a lack of techniques to visualize the informative EEG features the ConvNets learn. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed movements from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching or surpassing that of the widely-used filter bank common spatial patterns (FBCSP) decoding algorithm. While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta and high gamma frequencies. These methods also proved useful as a technique for spatially mapping the learned features, revealing the topography of the causal contributions of features in different frequency bands to decoding the movement classes. Our study thus shows how to design and train ConvNets to decode movement-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping.

研究动机与目标

• 开发并评估用于从原始EEG中端到端解码想象或执行运动的深度ConvNets，无需手工设计特征。
• 识别在EEG基脑机接口中提升解码性能的架构与训练创新。
• 开发新颖的可视化技术，以解释ConvNets在EEG数据中学习到的有用特征。
• 利用学习到的特征，绘制不同频带特异性贡献在空间上的拓扑分布，以支持运动解码。

提出的方法

• 设计并训练了多种专为从原始时间序列EEG数据中解码而定制的深度ConvNet架构。
• 引入批量归一化和指数线性单元（ELUs）以提升训练稳定性和性能。
• 采用裁剪训练策略，以增强在有限EEG数据上的泛化能力并减少过拟合。
• 开发了可视化方法以解释学习到的特征，识别出如α、β和高频γ调制等频带特异性模式。
• 使用空间映射技术可视化学习到的特征在电极位置上的贡献。
• 与广泛使用的FBCSP算法进行性能对比，该算法依赖手工设计的谱功率特征。

实验结果

研究问题

• RQ1 与传统FBCSP相比，深度ConvNets能否在原始EEG上实现具有竞争力或更优的解码性能，用于与运动相关的任务？
• RQ2 哪些架构与训练创新（如批量归一化、ELUs、裁剪训练）对EEG解码中ConvNet性能的提升最为显著？
• RQ3 ConvNets是否能够学习利用特定频带（如α、β和高频γ）的谱功率调制？
• RQ4 新颖的可视化技术能否有效揭示ConvNets在EEG中学习到的特征所依赖的空间与频谱模式？
• RQ5 对于不同运动类别，学习到的特征在头皮上的因果贡献空间拓扑分布如何？

主要发现

• 所提出的结合批量归一化和指数线性单元的深度ConvNet架构，在解码性能上达到或优于标准FBCSP算法。
• 裁剪训练策略显著提升了模型在EEG解码任务中的泛化能力和性能。
• 可视化技术证实，ConvNets能够学习检测α、β和高频γ频带的谱功率调制。
• 对学习到的特征进行空间映射，揭示了在顶枕区和运动感觉皮层区域具有明显差异的头皮贡献拓扑模式。
• 学习到的特征并非预先固定，表明模型能够自适应地从原始EEG中提取相关的时间-空间-频谱模式。
• 深度学习与先进可视化技术的结合，实现了从原始EEG中可解释的运动相关神经动态脑功能映射。