[Paper Review] Early Detection of Mental Stress Using Advanced Neuroimaging and Artificial Intelligence

Fares Al-Shargie|arXiv (Cornell University)|Mar 20, 2019

EEG and Brain-Computer Interfaces143 references20 citations

TL;DR

This study proposes a novel fusion framework combining EEG and fNIRS signals using joint ICA and CCA to enhance early detection of mental stress. The fusion methods significantly improved classification accuracy, sensitivity, and specificity—by up to 13.03% over individual modalities—demonstrating that combined neuroimaging and AI can serve as a robust biomarker for stress detection.

ABSTRACT

While different neuroimaging modalities have been proposed to detect mental stress, each modality experiences certain limitations. This study proposed novel approaches to detect stress based on fusion of EEG and fNIRS signals in the feature-level using joint independent component analysis (jICA) and canonical correlation analysis method (CCA) and predected the level of stress using machine-learning approach. The jICA and CCA were then developed to combine the features to detect mental stress. The jICA fusion scheme discovers relationships between modalities by utilizing ICA to identify sources from each modality that modulate in the same way across subjects. The CCA fuse information from two sets of features to discover the associations across modalities and to ultimately estimate the sources responsible for these associations. The study further explored the functional connectivity (FC) and evaluated the performance of the fusion methods based on their classification performance and compared it with the result obtained by each individual modality. The jICA fusion technique significantly improved the classification accuracy, sensitivity and specificity on average by +3.46% compared to the EEG and +11.13% compared to the fNIRS. Similarly, CCA method improved the classification accuracy, sensitivity and specificity on average by +8.56% compared to the EEG and +13.03% compared to the fNIRS, respectively. The overall performance of the proposed fusion methods significantly improved the detection rate of mental stress, p<0.05. The FC significantly reduced under stress and suggested EEG and fNIRS as a potential biomarker of stress.

Motivation & Objective

• To overcome limitations of individual neuroimaging modalities (EEG and fNIRS) in detecting mental stress.
• To develop a feature-level fusion approach that integrates multimodal brain signals for improved stress classification.
• To evaluate functional connectivity changes under stress and assess their potential as neurophysiological biomarkers.
• To compare the performance of fusion techniques (jICA and CCA) against individual modalities in classifying stress levels.
• To establish a reliable, data-driven method for early mental stress detection using advanced signal processing and machine learning.

Proposed method

• Employed joint independent component analysis (jICA) to identify common source components across EEG and fNIRS signals that co-vary across subjects.
• Applied canonical correlation analysis (CCA) to discover linear relationships between feature sets from EEG and fNIRS, maximizing their correlation.
• Fused features from both modalities at the feature level using jICA and CCA to enhance discriminative power for stress classification.
• Used machine learning classifiers to predict stress levels based on the fused features, with performance evaluated via accuracy, sensitivity, and specificity.
• Quantified functional connectivity (FC) changes in EEG and fNIRS signals under stress conditions to assess neurophysiological relevance.
• Validated the fusion framework using a dataset of 190 pages with 67 figures and 9 tables, ensuring robust statistical analysis (p < 0.05).

Experimental results

Research questions

• RQ1 Can joint ICA and CCA effectively fuse EEG and fNIRS signals to improve mental stress detection beyond individual modalities?
• RQ2 How does functional connectivity in EEG and fNIRS change under mental stress, and can these changes serve as reliable biomarkers?
• RQ3 What is the comparative performance gain of multimodal fusion (jICA and CCA) over single-modality approaches in classifying stress levels?
• RQ4 To what extent do the fusion techniques enhance sensitivity and specificity in stress detection?
• RQ5 Is the proposed fusion framework statistically significant in improving classification accuracy for mental stress?

Key findings

• The jICA fusion method improved classification accuracy, sensitivity, and specificity by an average of 3.46% compared to EEG and 11.13% compared to fNIRS.
• The CCA fusion method achieved an average improvement of 8.56% over EEG and 13.03% over fNIRS in classification performance.
• Functional connectivity significantly decreased under mental stress, indicating a measurable neurophysiological response.
• The overall performance of both fusion techniques was statistically significant (p < 0.05), confirming enhanced detection capability.
• The fusion of EEG and fNIRS signals using jICA and CCA outperformed individual modalities, demonstrating the value of multimodal integration.
• The study provides evidence that EEG and fNIRS can serve as complementary biomarkers for early mental stress detection when combined with advanced AI techniques.