Explainable schizophrenia detection using fusion of convolutional neural network and vision transformer from electroencephalogram

方法

RP与WSST图像融合，ResNet-18/EfficientNet-B0与Vi

发现

混合模型在Warsaw和ASEEG数据集上分别达82.14%和95.04%准确率，额颞中央电极最具判

机会点

可探索将RP+WSST融合与混合CNN-ViT架构应用于农业EEG或作物胁迫检测，或迁移至其他生物信号分类任务。

Schizophrenia (SZ) is associated with subtle alterations in neural dynamics that are difficult to capture using conventional electroencephalogram (EEG) features. This study introduces a unified deep learning framework that integrates recurrence plots (RP) with wavelet synchrosqueezed transform (WSST) representations into a single fused image modality and leverages attention-enhanced hybrid convolutional–transformer architectures for subject-level classification. Specifically, we propose RP+WSST image fusion combined with convolutional neural network (CNN)–vision transformer (ViT) hybrids (ResNet-18–ViT and EfficientNet-B0–ViT) to jointly model local spatial patterns and global contextual dependencies. Subject-wise 10-fold cross-validation and a strictly isolated hold-out protocol (70/15/15 split) are employed to prevent subject leakage and provide unbiased performance estimates. Compared with single-backbone CNN and ViT models, the proposed hybrid architecture demonstrates competitive generalization. Interpretability is enhanced using appropriate XAI. Gradient-weighted class activation mapping (Grad-CAM) for the CNN branch and Attention Rollout for the ViT branch. The proposed framework is applied to automated SZ detection from resting-state scalp EEG using two independent databases (Warsaw and Atieh schizophrenia EEG (ASEEG)). On the Warsaw dataset, the ResNet-18–ViT hybrid achieved 82.14% accuracy (AUC 83.58%) using Amor-based WSST features. On the ASEEG dataset, performance reached 95.04% accuracy with AUC values above 95%. Channel-wise analysis identified frontal, temporal, and central electrodes as the most discriminative regions, consistent with known SZ-related electrophysiological abnormalities. These findings demonstrate the practical feasibility of deploying the proposed explainable AI (XAI) framework for reliable EEG-based clinical decision support in psychiatric engineering applications.