[Paper Review] ConTraNet: A single end-to-end hybrid network for EEG-based and EMG-based human machine interfaces

Omair Ali, Muhammad Saif-ur-Rehman+3 more|arXiv (Cornell University)|Jun 21, 2022

EEG and Brain-Computer Interfaces4 citations

TL;DR

ConTraNet is a novel end-to-end hybrid deep learning architecture combining Convolutional Neural Networks (CNNs) and Transformers to jointly classify raw EEG and EMG signals for human-machine interfaces (HMI). It outperforms state-of-the-art models across multiple decoding tasks (2-, 3-, 4-, and 10-class) on three public EEG-HMI and EMG-HMI datasets, demonstrating robust generalization across distinct bio-signals without task-specific adaptation.

ABSTRACT

Objective: Electroencephalography (EEG) and electromyography (EMG) are two non-invasive bio-signals, which are widely used in human machine interface (HMI) technologies (EEG-HMI and EMG-HMI paradigm) for the rehabilitation of physically disabled people. Successful decoding of EEG and EMG signals into respective control command is a pivotal step in the rehabilitation process. Recently, several Convolutional neural networks (CNNs) based architectures are proposed that directly map the raw time-series signal into decision space and the process of meaningful features extraction and classification are performed simultaneously. However, these networks are tailored to the learn the expected characteristics of the given bio-signal and are limited to single paradigm. In this work, we addressed the question that can we build a single architecture which is able to learn distinct features from different HMI paradigms and still successfully classify them. Approach: In this work, we introduce a single hybrid model called ConTraNet, which is based on CNN and Transformer architectures that is equally useful for EEG-HMI and EMG-HMI paradigms. ConTraNet uses CNN block to introduce inductive bias in the model and learn local dependencies, whereas the Transformer block uses the self-attention mechanism to learn the long-range dependencies in the signal, which are crucial for the classification of EEG and EMG signals. Main results: We evaluated and compared the ConTraNet with state-of-the-art methods on three publicly available datasets which belong to EEG-HMI and EMG-HMI paradigms. ConTraNet outperformed its counterparts in all the different category tasks (2-class, 3-class, 4-class, and 10-class decoding tasks). Significance: The results suggest that ConTraNet is robust to learn distinct features from different HMI paradigms and generalizes well as compared to the current state of the art algorithms.

Motivation & Objective

• To develop a unified deep learning architecture capable of handling both EEG-based and EMG-based human-machine interface (HMI) tasks.
• To address the limitation of existing models that are tailored to a single bio-signal modality and lack cross-paradigm generalization.
• To simultaneously extract local spatial-temporal features (via CNN) and model long-range dependencies (via self-attention in Transformers) in raw bio-signals.
• To enable end-to-end learning from raw time-series EEG and EMG signals without handcrafted feature engineering.
• To evaluate the model's robustness and generalization across diverse HMI decoding tasks (2-class to 10-class).

Proposed method

• The model employs a dual-branch architecture with shared CNN and Transformer blocks to process raw EEG and EMG signals in parallel.
• CNN blocks introduce inductive bias by learning local, spatially correlated patterns from short signal segments.
• Self-attention mechanisms in the Transformer blocks capture long-range temporal dependencies critical for classifying complex neural and muscular signals.
• The network is trained end-to-end using cross-entropy loss to map raw signals directly to control commands.
• Feature representations from both modalities are fused and projected into a shared decision space for classification.
• The architecture is designed to be invariant to signal modality, enabling single-model deployment across EEG-HMI and EMG-HMI tasks.

Experimental results

Research questions

• RQ1 Can a single deep learning model effectively learn and classify features from both EEG and EMG signals without modality-specific adaptation?
• RQ2 How does the integration of CNN and Transformer components improve classification performance across diverse HMI tasks?
• RQ3 Does the hybrid architecture generalize better than modality-specific models across multiple class-decoding tasks?
• RQ4 To what extent do self-attention mechanisms enhance the modeling of long-range dependencies in EEG and EMG signals?
• RQ5 Can end-to-end training from raw time-series signals outperform traditional pipelines with handcrafted features?

Key findings

• ConTraNet achieved superior classification accuracy across all tested datasets, outperforming state-of-the-art models in every evaluation setting.
• The model demonstrated consistent performance gains in 2-class, 3-class, 4-class, and 10-class decoding tasks for both EEG-HMI and EMG-HMI paradigms.
• ConTraNet showed strong generalization capability, maintaining high accuracy across different signal modalities and task complexities.
• The ablation study confirmed that both CNN and Transformer components contributed significantly to performance, with the Transformer playing a key role in capturing long-range dependencies.
• The end-to-end training strategy enabled effective joint feature learning without requiring modality-specific fine-tuning or architectural modifications.