Hybrid Multimodal Fusion of Raw ECG Signals and Derived Measurements for Stroke Classification

Abstract

Stroke is a leading cause of death and long-term disability, and auxiliary methods for rapid patient identification are needed. Previous studies based on electrocardiography (ECG) have attempted stroke classification using quantitative measurements and future stroke risk prediction using raw signals. However, measurements do not preserve all waveform detail, and models using raw signals alone may not sufficiently learn quantitative characteristics from limited data. We therefore propose a hybrid fusion framework that integrates the raw 12-lead signal and the quantitative measurements of the same ECG record. It combines a signal-only model, a feature-only model, and an interaction early fusion model that couples the two inputs at the raw data and representation levels, averaging their logits for the final prediction. Evaluation used 4147 ECG records from 4020 patients in a single-institution retrospective cohort, with patient-level splits. The proposed model achieved an F1-score of 77.6%, an area under the receiver operating characteristic curve (AUC) of 0.832, and a Brier score of 0.169. Among the evaluated models, the proposed framework achieved the highest AUC and lowest Brier score. The AUC improved by 0.034 over the best unimodal model, and an AUC of 0.820 was maintained under a year-based temporal holdout. These results suggest that integrating complementary representations of the same ECG at multiple levels can improve stroke classification. The framework could potentially serve as an auxiliary decision-support tool for identifying patients who require further evaluation, using ECG characteristics statistically associated with stroke.