Abstract
Background Artificial intelligence (AI) techniques have been proposed for automation of cine CMR segmentation for functional quantification. However, in other applications AI models have been shown to have potential for sex and/or racial bias.
Objectives To perform the first analysis of sex/racial bias in AI-based cine CMR segmentation using a large-scale database.
Methods A state-of-the-art deep learning (DL) model was used for automatic segmentation of both ventricles and the myocardium from cine short-axis CMR. The dataset consisted of end-diastole and end-systole short-axis cine CMR images of 5,903 subjects from the UK Biobank database (61.5±7.1 years, 52% male, 81% white). To assess sex and racial bias, we compared Dice scores and errors in measurements of biventricular volumes and function between patients grouped by race and sex. To investigate whether segmentation bias could be explained by potential confounders, a multivariate linear regression and ANCOVA were performed.
Results We found statistically significant differences in Dice scores (white ∼94% vs minority ethnic groups 86-89%) as well as in absolute/relative errors in volumetric and functional measures, showing that the AI model was biased against minority racial groups, even after correction for possible confounders.
Conclusions We have shown that racial bias can exist in DL-based cine CMR segmentation models. We believe that this bias is due to the imbalanced nature of the training data (combined with physiological differences). This is supported by the results which show racial bias but not sex bias when trained using the UK Biobank database, which is sex-balanced but not race-balanced.
Condensed Abstract AI algorithms have the potential to reflect or exacerbate racial/sex disparities in healthcare. We aimed to determine the impact of sex and race on the performance of an AI segmentation model for automatic CMR quantification in a cohort of 5,903 subjects from the UK Biobank database, which is sex-balanced but not race-balanced. We tested the model’s bias in performance using Dice scores and absolute/relative errors in measurements of biventricular volumes and function. Our study demonstrates that the model had a racial bias but no sex bias, and that subject characteristics and co-morbidities could not explain this bias.
Competing Interest Statement
EPA, BR, JMH, SKP, SN, APK and RR has nothing to declare. SEP provides consultancy to and is shareholder of Circle Cardioavascular Imaging, Inc., Calgary, Alberta, Canada.
Funding Statement
EPA was supported by the EPSRC (EP/R005516/1, EP/P001009/1) and by core funding from the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z). EPA, BR, MHR, APK and RR acknowledge support from the Wellcome/EPSRC Centre for Medical Engineering at Kings College London (WT 203148/Z/16/Z), the NIHR Cardiovascular MedTech Co-operative award to the Guy’s and St Thomas’ NHS Foundation Trust and the Department of Health National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ NHS Foundation Trust in partnership with King’s College London ( the views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, EPSRC, or the Department of Health). SEP, SN and SKP acknowledge the British Heart Foundation for funding the manual analysis to create a cardiovascular magnetic resonance imaging reference standard for the UK Biobank imaging resource in 5000 CMR scans (PG/14/89/31194). SEP acknowledges support from the National Institute for Health Research (NIHR) Biomedical Research Centre at Barts. SEP has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825903 (euCanSHare project). SEP acknowledges support from and from the SmartHeart EPSRC programme grant (EP/P001009/1). SEP also acknowledges support from the CAP-AI programme, London’s first AI enabling programme focused on stimulating growth in the capital’s AI Sector. CAP-AI is led by Capital Enterprise in partnership with Barts Health NHS Trust and Digital Catapult and is funded by the European Regional Development Fund and Barts Charity. SEP acknowledge support from the Health Data Research UK, an initiative funded by UK Research and Innovation, Department of Health and Social Care (England) and the devolved administrations, and leading medical research charities. SN and SKP are supported by the Oxford NIHR Biomedical Research Centre and the Oxford British Heart Foundation Centre of Research Excellence.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This study complies with the Declaration of Helsinki; the work was covered by the ethical approval for UK Biobank studies from the NHS National Research Ethics Service on 17th June 2011 (Ref 11/NW/0382) and extended on 18 June 2021 (Ref 21/NW/0157) with written informed consent obtained from all participants.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Disclosures: EPA, BR, JMH, SKP, SN, APK and RR has nothing to declare. SEP provides consultancy to and is shareholder of Circle Cardiovascular Imaging, Inc., Calgary, Alberta, Canada.
Funding: EPA and APK were supported by the EPSRC (EP/R005516/1) and by core funding from the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z). This research was funded in whole, or in part, by the Wellcome Trust WT203148/Z/16/Z. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. SEP, APK and RR acknowledge funding from the EPSRC through the Smart Heart programme grant (EP/P001009/1). EPA, BR, MHR, APK and RR acknowledge support from the Wellcome/EPSRC Centre for Medical Engineering at King’s College London (WT 203148/Z/16/Z), the NIHR Cardiovascular MedTech Co-operative award to the Guy’s and St Thomas’ NHS Foundation Trust and the Department of Health National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ NHS Foundation Trust in partnership with King’s College London (the views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, EPSRC, or the Department of Health). SEP, SN and SKP acknowledge the British Heart Foundation for funding the manual analysis to create a cardiovascular magnetic resonance imaging reference standard for the UK Biobank imaging resource in 5000 CMR scans (www.bhf.org.uk; PG/14/89/31194). SEP acknowledges support from the National Institute for Health Research (NIHR) Biomedical Research Centre at Barts. SEP has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825903 (euCanSHare project). SEP also acknowledges support from the CAP-AI programme, London’s first AI enabling programme focused on stimulating growth in the capital’s AI Sector. CAP-AI is led by Capital Enterprise in partnership with Barts Health NHS Trust and Digital Catapult and is funded by the European Regional Development Fund and Barts Charity. SEP acknowledge support from the Health Data Research UK, an initiative funded by UK Research and Innovation, Department of Health and Social Care (England) and the devolved administrations, and leading medical research charities. SN and SKP are supported by the Oxford NIHR Biomedical Research Centre and the Oxford British Heart Foundation Centre of Research Excellence.
Data Availability
This research has been conducted using the UK Biobank Resource (application numbers 17806 and 2964). The UK Biobank data are available for approved projects from https://www.ukbiobank.ac.uk/
Abbreviations
- AI
- Artificial intelligence
- CMR
- Cardiac magnetic resonance
- CVD
- Cardiovascular diseases
- DL
- Deep learning
- DSC
- Dice similarity coefficient
- ED
- End diastole
- EF
- Ejection fraction
- ES
- End systole
- LV
- Left ventricle
