Background Many ECG-AI models have been developed to predict a wide range of cardiovascular outcomes. The underrepresentation of women in cardiovascular disease studies has raised concerns if these models are equally predictive in women as compared to men. We tested the effect of sex-imbalance in training datasets on predictive performance of ECG-AI models, investigating imbalance in representation (ratio women-to-men), as well as in outcome prevalence, and percentage of misclassification.

Methods We used a dataset containing raw 12-lead ECGs (n = 474,006) of 181,755 individuals who visited the University Medical Center Utrecht at any of the non-cardiology departments between July 1997 and August 2023 and sampled a sex-balanced dataset (n = 165,156) including only one ECG per individual. Multiple deep convolutional neural networks were trained to predict four outcomes; left bundle branch block, Long QT Syndrome, left ventricular hypertrophy or ECGs classified as ‘abnormal’ by a physician. Using subsampling, we simulated scenarios of sex-imbalance in representation (nscenario=5) for all outcomes and disease prevalence (nscenario=5), both representation and disease prevalence (nscenario=20) and disease misclassification (nscenario=7) for ‘abnormal’. Model performance was evaluated per scenario using area under the receiver operating characteristic curve (AUC) and smooth expected calibration error (smECE) for women and men separately.

Results Across all scenario’s, the AUC remained stable, with small absolute differences between women and men for sex-imbalance in representation (ΔAUC: [0.002-0.025]), in disease prevalence (ΔAUC: [0.01-0.02]), in scenarios of both representation and disease prevalence (ΔAUC: [0.003-0.039]), and in outcome misclassification (ΔAUC: [0.007-0.077]). Only when disease prevalence in train and test data was sex-imbalanced, we observed differences in calibration error between sexes (max ΔsmECE: 0.26), with similar patterns for women and men.

Conclusion The neural networks in this study demonstrated resilience to sex-imbalance in training ECG data.

Graphical abstractGraphical summary of the study methodology and results showing that ECG classification with convolutional neural networks is not sensitive to sex-imbalances in datasets. AUC = Area under the receiver operating curve; smECE = smooth expected calibration error. Created in BioRender. Meijer, I. (2025) https://BioRender.com/nxkwvoi.

The Department of Cardiology at UMC Utrecht may receive royalties in the future from sales of deep learning ECG algorithms developed by Cordys Analytics, a spin-off company. Additionally, RvE is the Chief Scientific Officer (CSO) and RvdL is Chief Medical Officer (CMO) and both are shareholders of Cordys Analytics. These affiliations and potential financial interests have been disclosed and are being managed in accordance with institutional policies.

This project is part of the Dutch Cardiovascular Alliance Consortium IMPRESS (2020B004).

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The University Medical Center Utrecht ethical committee approved this research.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

The data cannot be shared publicly. The code relevant to this work is available on a private GitHub repository (SexImbalanceAI) and can be accessed upon reasonable request.