Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Validation of acute myocardial infarction (AMI) in electronic medical records: the SPEED-EXTRACT Study

Aldo Saavedra, View ORCID ProfileRichard W. Morris, Charmaine S. Tam, Madhura Killedar, Seshika Ratwatte, Ronald Huynh, Christopher Yu, David Z Yuan, Michelle Cretikos, Janice Gullick, Stephen T. Vernon, Gemma A. Figtree, Jonathan Morris, David Brieger
doi: https://doi.org/10.1101/2020.12.08.20245720
Aldo Saavedra
1The University of Sydney, Centre for Translational Data Science, NSW, Australia
2The University of Sydney, Faculty of Health Sciences, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Richard W. Morris
1The University of Sydney, Centre for Translational Data Science, NSW, Australia
3The University of Sydney, School of Medicine, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Richard W. Morris
Charmaine S. Tam
1The University of Sydney, Centre for Translational Data Science, NSW, Australia
3The University of Sydney, School of Medicine, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Madhura Killedar
4The University of Sydney, Sydney Informatics Hub, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Seshika Ratwatte
3The University of Sydney, School of Medicine, NSW, Australia
8Concord Hospital, Department of Cardiology, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ronald Huynh
3The University of Sydney, School of Medicine, NSW, Australia
8Concord Hospital, Department of Cardiology, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christopher Yu
3The University of Sydney, School of Medicine, NSW, Australia
8Concord Hospital, Department of Cardiology, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Z Yuan
3The University of Sydney, School of Medicine, NSW, Australia
8Concord Hospital, Department of Cardiology, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michelle Cretikos
9New South Wales Ministry of Health, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Janice Gullick
5The University of Sydney, Susan Wakil School of Nursing and Midwifery, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephen T. Vernon
6Kolling Institute of Medical Research and Department of Cardiology, Royal North Shore Hospital, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gemma A. Figtree
3The University of Sydney, School of Medicine, NSW, Australia
6Kolling Institute of Medical Research and Department of Cardiology, Royal North Shore Hospital, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jonathan Morris
7Northern Sydney Local Health District, Clinical and Population Perinatal Health, and Department of Obstetrics and Gynaecology, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Brieger
8Concord Hospital, Department of Cardiology, NSW, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: david.brieger@sydney.edu.au
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Objectives To determine whether data captured in electronic medical records (eMR) is sufficient to serve as a clinical data source to make a reliable determination of ST elevation myocardial infarction (STEMI) and non-ST elevation myocardial infarction (NSTEMI) and to use these eMR derived diagnoses to validate ICD-10 codes for STEMI and NSTEMI.

Design Retrospective validation by blind chart review of a purposive sample of patients with a troponin test result, ECG record, and medical note available in the eMR.

Setting Two local health districts containing two tertiary hospitals and six referral hospitals in New South Wales, Australia.

Participants N = 897 adult patients who had a hs-troponin test result indicating suspected AMI.

Primary outcome measures Inter-rater reliability of clinical diagnosis (κ) for ST-elevated myocardial infarction (STEMI) and Non-ST elevated myocardial infarction (NSTEMI); and sensitivity, specificity, and positive predictive value (PPV) of ICD-10 codes for STEMI and NSTEMI.

Results The diagnostic agreement between clinical experts was high for STEMI (κ = 0.786) but lower for NSTEMI (κ = 0.548). ICD-10 STEMI codes had moderate sensitivity (Se = 88±6.7), very high specificity (Sp = 99±0.7) and high positive predictive value (PPV = 91±6). NSTEMI ICD-10 codes were lower in each case (Se = 69±6.4, Sp = 96.0±1.5, PPV = 84±6).

Conclusions The eMR held sufficient clinical data to reliably diagnose STEMI, producing high inter-rater agreement among our expert reviewers as well as allowing reasonably precise estimates of the accuracy of administrative ICD-10 codes. However the clinical detail held in the eMR was less sufficient to diagnose NSTEMI, indicated by a lower inter-rater agreement. Efforts should be directed towards operationalising the clinical definition of NSTEMI and improving clinical record keeping to enable an accurate description of the clinical phenotype in the eMR, and thus improve reliability of the diagnosis of NSTEMI using these data sources.

Strengths and limitations of this study

  • Expert chart review provided a robust evaluation of the reliability and sufficiency of data directly extracted from the EMR for the diagnosis of AMI

  • Computational interrogation and extraction of the eMR (via SPEED-EXTRACT) allowed us to use a wide selection for inclusion in the sample on the basis of clinical data independent of ICD-10 code, enabling the capture of missed cases (i.e., uncoded AMI) and so determine estimates for the false negative rate and sensitivity

  • Results were necessarily based on the subset of patients with sufficient clinical data in the eMR. Inferences from this subset to the wider patient pool will be biased when the availability of records varies with diagnosis

  • At least two sources of uncertainty in the gold reference standard we used are indistinguishable: uncertainty due to poor clinical detail in the eMR, and uncertainty due to a weak operational definition of the diagnosis (e.g., NSTEMI).

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

Funding for the SPEED-EXTRACT study was provided by grants from the Agency for Clinical Innovation, NSW Ministry of Health and Sydney Health Partners. Author CST was supported by a National Health and Medical Research Centre Early Career Fellowship from Australia (#1037275). Author AS was supported by Sydney Health Partners (Harnessing the eMR to improve care of patients with acute chest pain within Sydney Health)

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Northern Sydney Local Health District (NLSHD) Human Research Ethics Committee (HREC), reference: HREC/17/HAWKE/192

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Footnotes

  • ↵* Co-first author

Data Availability

Data is available from https://github.com/datarichard/SPEED-EXTRACT-validation

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted December 09, 2020.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Validation of acute myocardial infarction (AMI) in electronic medical records: the SPEED-EXTRACT Study
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Validation of acute myocardial infarction (AMI) in electronic medical records: the SPEED-EXTRACT Study
Aldo Saavedra, Richard W. Morris, Charmaine S. Tam, Madhura Killedar, Seshika Ratwatte, Ronald Huynh, Christopher Yu, David Z Yuan, Michelle Cretikos, Janice Gullick, Stephen T. Vernon, Gemma A. Figtree, Jonathan Morris, David Brieger
medRxiv 2020.12.08.20245720; doi: https://doi.org/10.1101/2020.12.08.20245720
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Validation of acute myocardial infarction (AMI) in electronic medical records: the SPEED-EXTRACT Study
Aldo Saavedra, Richard W. Morris, Charmaine S. Tam, Madhura Killedar, Seshika Ratwatte, Ronald Huynh, Christopher Yu, David Z Yuan, Michelle Cretikos, Janice Gullick, Stephen T. Vernon, Gemma A. Figtree, Jonathan Morris, David Brieger
medRxiv 2020.12.08.20245720; doi: https://doi.org/10.1101/2020.12.08.20245720

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Cardiovascular Medicine
Subject Areas
All Articles
  • Addiction Medicine (70)
  • Allergy and Immunology (168)
  • Anesthesia (51)
  • Cardiovascular Medicine (452)
  • Dentistry and Oral Medicine (83)
  • Dermatology (55)
  • Emergency Medicine (158)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (191)
  • Epidemiology (5267)
  • Forensic Medicine (3)
  • Gastroenterology (196)
  • Genetic and Genomic Medicine (759)
  • Geriatric Medicine (80)
  • Health Economics (213)
  • Health Informatics (699)
  • Health Policy (360)
  • Health Systems and Quality Improvement (224)
  • Hematology (99)
  • HIV/AIDS (164)
  • Infectious Diseases (except HIV/AIDS) (5888)
  • Intensive Care and Critical Care Medicine (363)
  • Medical Education (105)
  • Medical Ethics (25)
  • Nephrology (83)
  • Neurology (767)
  • Nursing (43)
  • Nutrition (131)
  • Obstetrics and Gynecology (143)
  • Occupational and Environmental Health (234)
  • Oncology (480)
  • Ophthalmology (152)
  • Orthopedics (39)
  • Otolaryngology (95)
  • Pain Medicine (39)
  • Palliative Medicine (20)
  • Pathology (141)
  • Pediatrics (223)
  • Pharmacology and Therapeutics (136)
  • Primary Care Research (98)
  • Psychiatry and Clinical Psychology (864)
  • Public and Global Health (2020)
  • Radiology and Imaging (349)
  • Rehabilitation Medicine and Physical Therapy (158)
  • Respiratory Medicine (287)
  • Rheumatology (94)
  • Sexual and Reproductive Health (74)
  • Sports Medicine (77)
  • Surgery (110)
  • Toxicology (25)
  • Transplantation (29)
  • Urology (39)