[Paper Review] FingerFlex: Inferring Finger Trajectories from ECoG signals

Vladislav Lomtev, A. V. Kovalev+1 more|arXiv (Cornell University)|Oct 23, 2022

EEG and Brain-Computer Interfaces4 citations

TL;DR

FingerFlex proposes a convolutional encoder-decoder neural network with skip connections to decode continuous finger trajectory movements from electrocorticographic (ECoG) signals. It achieves a state-of-the-art correlation coefficient of 0.74 on the BCI Competition IV dataset 4, demonstrating high-precision, real-time capable motor decoding without manual feature engineering or separate models for rest and movement states.

ABSTRACT

Motor brain-computer interface (BCI) development relies critically on neural time series decoding algorithms. Recent advances in deep learning architectures allow for automatic feature selection to approximate higher-order dependencies in data. This article presents the FingerFlex model - a convolutional encoder-decoder architecture adapted for finger movement regression on electrocorticographic (ECoG) brain data. State-of-the-art performance was achieved on a publicly available BCI competition IV dataset 4 with a correlation coefficient between true and predicted trajectories up to 0.74. The presented method provides the opportunity for developing fully-functional high-precision cortical motor brain-computer interfaces.

Motivation & Objective

• To develop a deep learning model that accurately decodes continuous finger flexion trajectories from ECoG signals.
• To eliminate the need for manual feature extraction by enabling end-to-end learning of relevant neural patterns.
• To create a lightweight, real-time capable model that operates on current time windows without future or past data dependency.
• To improve performance over existing methods, especially conventional machine learning and prior deep learning approaches, on the BCI Competition IV dataset 4.
• To enable individualized, high-precision motor decoding with minimal training data (as little as 6.5 minutes per subject).

Proposed method

• A U-Net-like encoder-decoder architecture with strided convolutions and deconvolutions is used to map ECoG time series to predicted finger trajectories.
• Skip connections are incorporated between encoder and decoder blocks to preserve spatial and temporal resolution and improve gradient flow.
• The model processes time-frequency representations of ECoG signals via wavelet-based convolutional filters over physiologically relevant frequency bands (e.g., beta, gamma, high-gamma).
• A novel loss function combining mean squared error (MSE) with a temporal consistency regularization is introduced to enhance regression stability and accuracy.
• The model is trained end-to-end on raw ECoG time series, avoiding handcrafted features, and operates on fixed-length input windows without lookahead or feedback.
• Hyperparameters such as time delay, frequency range, learning rate, and model depth are found to be stable across participants and can be fixed for general use.

Experimental results

Research questions

• RQ1 Can a deep encoder-decoder architecture with skip connections outperform conventional machine learning and prior deep learning models in decoding continuous finger trajectories from ECoG signals?
• RQ2 To what extent can a model trained end-to-end on raw ECoG data achieve high decoding accuracy without manual feature engineering?
• RQ3 Can the proposed model generalize across movement and resting states without separate modeling or switching mechanisms?
• RQ4 How stable are key hyperparameters (e.g., time delay, frequency bands) across different subjects and datasets?
• RQ5 Can the model achieve high performance with minimal training data, enabling practical clinical deployment?

Key findings

• The FingerFlex model achieved a correlation coefficient of up to 0.74 between predicted and true finger trajectories on the BCI Competition IV dataset 4, representing state-of-the-art performance.
• The model outperformed both conventional machine learning methods and prior deep learning approaches, including LSTM-based and fully connected networks.
• The model demonstrated robust performance across subjects with only 6.5 minutes of training data per participant, indicating high data efficiency.
• The use of a hybrid loss function combining MSE with temporal regularization significantly improved prediction stability and accuracy.
• The model is lightweight (approximately 600,000 parameters) and suitable for real-time inference due to its autoregressive, current-window-only prediction mechanism.
• The model’s hyperparameters, including time delay and frequency band ranges, were found to be stable across participants, enabling fixed configuration use.