[论文解读] ConTraNet: A single end-to-end hybrid network for EEG-based and EMG-based human machine interfaces

Omair Ali, Muhammad Saif-ur-Rehman+3 more|arXiv (Cornell University)|Jun 21, 2022

EEG and Brain-Computer Interfaces被引用 4

一句话总结

ConTraNet 是一种新颖的端到端混合深度学习架构，结合卷积神经网络（CNNs）与Transformer模型，用于联合分类原始脑电信号（EEG）和肌电信号（EMG），以实现人机交互（HMI）。该模型在三个公开的EEG-HMI与EMG-HMI数据集上，于多个解码任务（2-、3-、4-和10类）中均优于当前最先进模型，展现出在不同生物信号间强大的泛化能力，且无需针对特定任务进行适应调整。

ABSTRACT

Objective: Electroencephalography (EEG) and electromyography (EMG) are two non-invasive bio-signals, which are widely used in human machine interface (HMI) technologies (EEG-HMI and EMG-HMI paradigm) for the rehabilitation of physically disabled people. Successful decoding of EEG and EMG signals into respective control command is a pivotal step in the rehabilitation process. Recently, several Convolutional neural networks (CNNs) based architectures are proposed that directly map the raw time-series signal into decision space and the process of meaningful features extraction and classification are performed simultaneously. However, these networks are tailored to the learn the expected characteristics of the given bio-signal and are limited to single paradigm. In this work, we addressed the question that can we build a single architecture which is able to learn distinct features from different HMI paradigms and still successfully classify them. Approach: In this work, we introduce a single hybrid model called ConTraNet, which is based on CNN and Transformer architectures that is equally useful for EEG-HMI and EMG-HMI paradigms. ConTraNet uses CNN block to introduce inductive bias in the model and learn local dependencies, whereas the Transformer block uses the self-attention mechanism to learn the long-range dependencies in the signal, which are crucial for the classification of EEG and EMG signals. Main results: We evaluated and compared the ConTraNet with state-of-the-art methods on three publicly available datasets which belong to EEG-HMI and EMG-HMI paradigms. ConTraNet outperformed its counterparts in all the different category tasks (2-class, 3-class, 4-class, and 10-class decoding tasks). Significance: The results suggest that ConTraNet is robust to learn distinct features from different HMI paradigms and generalizes well as compared to the current state of the art algorithms.

研究动机与目标

• 开发一种统一的深度学习架构，能够同时处理基于EEG和基于EMG的人机交互（HMI）任务。
• 解决现有模型仅针对单一生物信号模态设计、缺乏跨范式泛化能力的局限性。
• 同时从原始生物信号中提取局部时空特征（通过CNN）并建模长程依赖关系（通过Transformer中的自注意力机制）。
• 实现从原始时间序列EEG与EMG信号端到端的联合学习，无需手工特征工程。
• 评估模型在多样化HMI解码任务（2类至10类）中的鲁棒性与泛化能力。

提出的方法

• 该模型采用双分支架构，共享CNN与Transformer模块，以并行方式处理原始EEG与EMG信号。
• CNN模块通过学习短信号段中的局部空间相关模式，引入归纳偏置。
• Transformer模块中的自注意力机制可捕捉对分类复杂神经与肌肉信号至关重要的长程时间依赖关系。
• 网络通过交叉熵损失进行端到端训练，直接将原始信号映射到控制指令。
• 来自两种模态的特征表示被融合，并投影到共享决策空间以实现分类。
• 该架构设计为对信号模态具有不变性，从而实现单一模型在EEG-HMI与EMG-HMI任务中的统一部署。

实验结果

研究问题

• RQ1 单一深度学习模型是否能够在无需模态特异性适应的前提下，有效学习并分类EEG与EMG信号的特征？
• RQ2 CNN与Transformer组件的融合在多样化HMI任务中如何提升分类性能？
• RQ3 与模态特异性模型相比，该混合架构在多个类别解码任务中是否展现出更优的泛化能力？
• RQ4 自注意力机制在多大程度上增强了对EEG与EMG信号中长程依赖关系的建模能力？
• RQ5 从原始时间序列信号进行端到端训练，是否能够超越传统依赖手工特征工程的流水线方法？

主要发现

• ConTraNet在所有测试数据集中均实现了卓越的分类准确率，在每种评估设置下均优于当前最先进模型。
• 该模型在EEG-HMI与EMG-HMI范式下的2类、3类、4类和10类解码任务中均表现出一致的性能提升。
• ConTraNet展现出强大的泛化能力，在不同信号模态和任务复杂度下均保持高准确率。
• 消融实验证实，CNN与Transformer组件均对性能有显著贡献，其中Transformer在捕捉长程依赖关系方面起关键作用。
• 端到端训练策略实现了有效的联合特征学习，且无需进行模态特异性微调或架构修改。