Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Fairness in Cardiac Magnetic Resonance Imaging: Assessing Sex and Racial Bias in Deep Learning-based Segmentation

Esther Puyol-Antón, Bram Ruijsink, Jorge Mariscal Harana, Stefan K. Piechnik, Stefan Neubauer, Steffen E. Petersen, Reza Razavi, Phil Chowienczyk, Andrew P King
doi: https://doi.org/10.1101/2021.07.19.21260749
Esther Puyol-Antón
aSchool of Biomedical Engineering and Imaging Sciences, King’s College London, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: esther.puyol_anton@kcl.ac.uk
Bram Ruijsink
aSchool of Biomedical Engineering and Imaging Sciences, King’s College London, London, United Kingdom
bDepartment of Adult and Paediatric Cardiology, Guy’s and St Thomas’ NHS Foundation Trust, London, London, United Kingdom
cDept of Cardiology, Division of Heart and Lungs, University Medical Centre Utrecht, The Netherlands
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jorge Mariscal Harana
aSchool of Biomedical Engineering and Imaging Sciences, King’s College London, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stefan K. Piechnik
dDivision of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stefan Neubauer
dDivision of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Steffen E. Petersen
eWilliam Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University London, Charterhouse Square, London, EC1M 6BQ, United Kingdom
fBarts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust, West Smithfield, EC1A 7BE, London, United Kingdom
gHealth Data Research UK, London, United Kingdom
hAlan Turing Institute, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Reza Razavi
aSchool of Biomedical Engineering and Imaging Sciences, King’s College London, London, United Kingdom
bDepartment of Adult and Paediatric Cardiology, Guy’s and St Thomas’ NHS Foundation Trust, London, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Phil Chowienczyk
iBritish Heart Foundation Centre, King’s College London. London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrew P King
aSchool of Biomedical Engineering and Imaging Sciences, King’s College London, London, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Background Artificial intelligence (AI) techniques have been proposed for automation of cine CMR segmentation for functional quantification. However, in other applications AI models have been shown to have potential for sex and/or racial bias.

Objectives To perform the first analysis of sex/racial bias in AI-based cine CMR segmentation using a large-scale database.

Methods A state-of-the-art deep learning (DL) model was used for automatic segmentation of both ventricles and the myocardium from cine short-axis CMR. The dataset consisted of end-diastole and end-systole short-axis cine CMR images of 5,903 subjects from the UK Biobank database (61.5±7.1 years, 52% male, 81% white). To assess sex and racial bias, we compared Dice scores and errors in measurements of biventricular volumes and function between patients grouped by race and sex. To investigate whether segmentation bias could be explained by potential confounders, a multivariate linear regression and ANCOVA were performed.

Results We found statistically significant differences in Dice scores (white ∼94% vs minority ethnic groups 86-89%) as well as in absolute/relative errors in volumetric and functional measures, showing that the AI model was biased against minority racial groups, even after correction for possible confounders.

Conclusions We have shown that racial bias can exist in DL-based cine CMR segmentation models. We believe that this bias is due to the imbalanced nature of the training data (combined with physiological differences). This is supported by the results which show racial bias but not sex bias when trained using the UK Biobank database, which is sex-balanced but not race-balanced.

Condensed Abstract AI algorithms have the potential to reflect or exacerbate racial/sex disparities in healthcare. We aimed to determine the impact of sex and race on the performance of an AI segmentation model for automatic CMR quantification in a cohort of 5,903 subjects from the UK Biobank database, which is sex-balanced but not race-balanced. We tested the model’s bias in performance using Dice scores and absolute/relative errors in measurements of biventricular volumes and function. Our study demonstrates that the model had a racial bias but no sex bias, and that subject characteristics and co-morbidities could not explain this bias.

Competing Interest Statement

EPA, BR, JMH, SKP, SN, APK and RR has nothing to declare. SEP provides consultancy to and is shareholder of Circle Cardioavascular Imaging, Inc., Calgary, Alberta, Canada.

Funding Statement

EPA was supported by the EPSRC (EP/R005516/1, EP/P001009/1) and by core funding from the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z). EPA, BR, MHR, APK and RR acknowledge support from the Wellcome/EPSRC Centre for Medical Engineering at Kings College London (WT 203148/Z/16/Z), the NIHR Cardiovascular MedTech Co-operative award to the Guy’s and St Thomas’ NHS Foundation Trust and the Department of Health National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ NHS Foundation Trust in partnership with King’s College London ( the views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, EPSRC, or the Department of Health). SEP, SN and SKP acknowledge the British Heart Foundation for funding the manual analysis to create a cardiovascular magnetic resonance imaging reference standard for the UK Biobank imaging resource in 5000 CMR scans (PG/14/89/31194). SEP acknowledges support from the National Institute for Health Research (NIHR) Biomedical Research Centre at Barts. SEP has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825903 (euCanSHare project). SEP acknowledges support from and from the SmartHeart EPSRC programme grant (EP/P001009/1). SEP also acknowledges support from the CAP-AI programme, London’s first AI enabling programme focused on stimulating growth in the capital’s AI Sector. CAP-AI is led by Capital Enterprise in partnership with Barts Health NHS Trust and Digital Catapult and is funded by the European Regional Development Fund and Barts Charity. SEP acknowledge support from the Health Data Research UK, an initiative funded by UK Research and Innovation, Department of Health and Social Care (England) and the devolved administrations, and leading medical research charities. SN and SKP are supported by the Oxford NIHR Biomedical Research Centre and the Oxford British Heart Foundation Centre of Research Excellence.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study complies with the Declaration of Helsinki; the work was covered by the ethical approval for UK Biobank studies from the NHS National Research Ethics Service on 17th June 2011 (Ref 11/NW/0382) and extended on 18 June 2021 (Ref 21/NW/0157) with written informed consent obtained from all participants.

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Footnotes

  • Disclosures: EPA, BR, JMH, SKP, SN, APK and RR has nothing to declare. SEP provides consultancy to and is shareholder of Circle Cardiovascular Imaging, Inc., Calgary, Alberta, Canada.

  • Funding: EPA and APK were supported by the EPSRC (EP/R005516/1) and by core funding from the Wellcome/EPSRC Centre for Medical Engineering (WT203148/Z/16/Z). This research was funded in whole, or in part, by the Wellcome Trust WT203148/Z/16/Z. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. SEP, APK and RR acknowledge funding from the EPSRC through the Smart Heart programme grant (EP/P001009/1). EPA, BR, MHR, APK and RR acknowledge support from the Wellcome/EPSRC Centre for Medical Engineering at King’s College London (WT 203148/Z/16/Z), the NIHR Cardiovascular MedTech Co-operative award to the Guy’s and St Thomas’ NHS Foundation Trust and the Department of Health National Institute for Health Research (NIHR) comprehensive Biomedical Research Centre award to Guy’s & St Thomas’ NHS Foundation Trust in partnership with King’s College London (the views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, EPSRC, or the Department of Health). SEP, SN and SKP acknowledge the British Heart Foundation for funding the manual analysis to create a cardiovascular magnetic resonance imaging reference standard for the UK Biobank imaging resource in 5000 CMR scans (www.bhf.org.uk; PG/14/89/31194). SEP acknowledges support from the National Institute for Health Research (NIHR) Biomedical Research Centre at Barts. SEP has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825903 (euCanSHare project). SEP also acknowledges support from the CAP-AI programme, London’s first AI enabling programme focused on stimulating growth in the capital’s AI Sector. CAP-AI is led by Capital Enterprise in partnership with Barts Health NHS Trust and Digital Catapult and is funded by the European Regional Development Fund and Barts Charity. SEP acknowledge support from the Health Data Research UK, an initiative funded by UK Research and Innovation, Department of Health and Social Care (England) and the devolved administrations, and leading medical research charities. SN and SKP are supported by the Oxford NIHR Biomedical Research Centre and the Oxford British Heart Foundation Centre of Research Excellence.

Data Availability

This research has been conducted using the UK Biobank Resource (application numbers 17806 and 2964). The UK Biobank data are available for approved projects from https://www.ukbiobank.ac.uk/

  • Abbreviations

    AI
    Artificial intelligence
    CMR
    Cardiac magnetic resonance
    CVD
    Cardiovascular diseases
    DL
    Deep learning
    DSC
    Dice similarity coefficient
    ED
    End diastole
    EF
    Ejection fraction
    ES
    End systole
    LV
    Left ventricle
  • Copyright 
    The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
    Back to top
    PreviousNext
    Posted September 14, 2021.
    Download PDF

    Supplementary Material

    Data/Code
    Email

    Thank you for your interest in spreading the word about medRxiv.

    NOTE: Your email address is requested solely to identify you as the sender of this article.

    Enter multiple addresses on separate lines or separate them with commas.
    Fairness in Cardiac Magnetic Resonance Imaging: Assessing Sex and Racial Bias in Deep Learning-based Segmentation
    (Your Name) has forwarded a page to you from medRxiv
    (Your Name) thought you would like to see this page from the medRxiv website.
    CAPTCHA
    This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
    Share
    Fairness in Cardiac Magnetic Resonance Imaging: Assessing Sex and Racial Bias in Deep Learning-based Segmentation
    Esther Puyol-Antón, Bram Ruijsink, Jorge Mariscal Harana, Stefan K. Piechnik, Stefan Neubauer, Steffen E. Petersen, Reza Razavi, Phil Chowienczyk, Andrew P King
    medRxiv 2021.07.19.21260749; doi: https://doi.org/10.1101/2021.07.19.21260749
    Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
    Citation Tools
    Fairness in Cardiac Magnetic Resonance Imaging: Assessing Sex and Racial Bias in Deep Learning-based Segmentation
    Esther Puyol-Antón, Bram Ruijsink, Jorge Mariscal Harana, Stefan K. Piechnik, Stefan Neubauer, Steffen E. Petersen, Reza Razavi, Phil Chowienczyk, Andrew P King
    medRxiv 2021.07.19.21260749; doi: https://doi.org/10.1101/2021.07.19.21260749

    Citation Manager Formats

    • BibTeX
    • Bookends
    • EasyBib
    • EndNote (tagged)
    • EndNote 8 (xml)
    • Medlars
    • Mendeley
    • Papers
    • RefWorks Tagged
    • Ref Manager
    • RIS
    • Zotero
    • Tweet Widget
    • Facebook Like
    • Google Plus One

    Subject Area

    • Cardiovascular Medicine
    Subject Areas
    All Articles
    • Addiction Medicine (161)
    • Allergy and Immunology (414)
    • Anesthesia (90)
    • Cardiovascular Medicine (857)
    • Dentistry and Oral Medicine (159)
    • Dermatology (97)
    • Emergency Medicine (248)
    • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (393)
    • Epidemiology (8560)
    • Forensic Medicine (4)
    • Gastroenterology (383)
    • Genetic and Genomic Medicine (1749)
    • Geriatric Medicine (167)
    • Health Economics (372)
    • Health Informatics (1239)
    • Health Policy (620)
    • Health Systems and Quality Improvement (467)
    • Hematology (196)
    • HIV/AIDS (372)
    • Infectious Diseases (except HIV/AIDS) (10294)
    • Intensive Care and Critical Care Medicine (553)
    • Medical Education (192)
    • Medical Ethics (51)
    • Nephrology (211)
    • Neurology (1677)
    • Nursing (97)
    • Nutrition (250)
    • Obstetrics and Gynecology (326)
    • Occupational and Environmental Health (450)
    • Oncology (928)
    • Ophthalmology (263)
    • Orthopedics (101)
    • Otolaryngology (172)
    • Pain Medicine (113)
    • Palliative Medicine (40)
    • Pathology (252)
    • Pediatrics (534)
    • Pharmacology and Therapeutics (250)
    • Primary Care Research (207)
    • Psychiatry and Clinical Psychology (1766)
    • Public and Global Health (3835)
    • Radiology and Imaging (623)
    • Rehabilitation Medicine and Physical Therapy (320)
    • Respiratory Medicine (520)
    • Rheumatology (208)
    • Sexual and Reproductive Health (167)
    • Sports Medicine (158)
    • Surgery (190)
    • Toxicology (36)
    • Transplantation (101)
    • Urology (76)