Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

User testing of a Diagnostic Decision Support System with Machine-assisted Chart Review to Facilitate Clinical Genomic Diagnosis

View ORCID ProfileAlanna Kulchak Rahm, Nephi A. Walton, Lynn K. Feldman, Conner Jenkins, Troy Jenkins, Thomas N. Person, Joseph Peterson, Jonathon C. Reynolds, View ORCID ProfilePeter N. Robinson, Makenzie A. Woltz, View ORCID ProfileMarc S. Williams, Michael M. Segal
doi: https://doi.org/10.1101/2020.08.21.20179580
Alanna Kulchak Rahm
1Genomic Medicine Institute, Geisinger, Danville, Pennsylvania, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Alanna Kulchak Rahm
  • For correspondence: akrahm@geisinger.edu
Nephi A. Walton
2Intermountain Precision Genomics, Intermountain Healthcare, St George, Utah, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lynn K. Feldman
3SimulConsult Inc., Chestnut Hill MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Conner Jenkins
4University of Utah, Salt Lake City, UT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Troy Jenkins
4University of Utah, Salt Lake City, UT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Thomas N. Person
1Genomic Medicine Institute, Geisinger, Danville, Pennsylvania, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joseph Peterson
4University of Utah, Salt Lake City, UT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jonathon C. Reynolds
1Genomic Medicine Institute, Geisinger, Danville, Pennsylvania, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter N. Robinson
5The Jackson Laboratory for Genomic Medicine and Institute for Systems Genomics, University of Connecticut, Farmington, CT, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Peter N. Robinson
Makenzie A. Woltz
1Genomic Medicine Institute, Geisinger, Danville, Pennsylvania, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marc S. Williams
1Genomic Medicine Institute, Geisinger, Danville, Pennsylvania, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Marc S. Williams
Michael M. Segal
3SimulConsult Inc., Chestnut Hill MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Background There is a need in clinical genomics for systems that assist in clinical diagnosis, analysis of genomic information and periodic re-analysis of results, and can utilize information from the electronic health record to do so. Such systems should be built using the concepts of human-centered design, fit within clinical workflows, and provide solutions to priority problems.

Methods We adapted a commercially available diagnostic decision support system (DDSS) to use extracted findings from a patient record and combine them with genomic variant information in the DDSS interface. Three representative patient cases were created in a simulated clinical environment for user testing. A semi-structured interview guide was created to illuminate factors relevant to human factors in CDS design and organizational implementation.

Results Six individuals completed the user testing process. Tester responses were positive and noted good fit with real-world clinical genetics workflow. Technical issues related to interface, interaction, and design were minor and fixable. Testers suggested solving issues related to terminology and usability through training and infobuttons. Time savings was estimated at 30-50% and additional uses such as in-house clinical variant analysis were suggested for increase fit with workflow and to further address priority problems.

Conclusion This study provides preliminary evidence for usability, workflow fit, acceptability, and implementation potential of a modified DDSS that includes machine-assisted chart review. Continued development and testing using principles from human-centered design and implementation science are necessary to improve technical functionality and acceptability for multiple stakeholders and organizational implementation potential to improve the genomic diagnosis process.

What is already known?

  • There is a need in clinical genomics for tools that assist in analysis of genomic information and can do so using information from the electronic health record.

  • Such tools should be easy to use, fit within clinical workflows, and provide solutions to priority problems as defined by clinician end-users.

  • Natural language processing (NLP) is a useful tool to read patient records and extract findings.

What does this paper add?

  • We demonstrated the use of Human-centered design and implementation science principles in a simulated environment for assessment of a new version of a decision support tool prior to large-scale implementation.

  • This study provides preliminary evidence that a clinical decision support tool with machine-assisted chart review is acceptable to clinical end-users, fits within the clinical workflow, and addresses perceived needs within the differential diagnosis process across all Mendelian genetic disorders.

  • Terminology codes for DDSSs should have levels of granularity tuned to the sensitivity and specificity appropriate to its various functions, e.g., NLP versus chart documentation.

Competing Interest Statement

LKF and MMS have an ownership stake in SimulConsult. MMS is PI of the NIH-funded Small Business Innovation Research (SBIR) grant. LKF and MMS provided the DDSS software to Geisinger for this research, including the modifications to enable the NLP and flagged findings, but were not involved in the user testing evaluation and analysis. All other authors declare no competing interests

Funding Statement

This study was supported by the National Human Genome Research Institute of the National Institutes of Health under the Small Business Innovation Research (SBIR) Program, Award Number 1R43HG010322-01 (principal investigator: MMS). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Geisinger IRB

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

study data are qualitative interview transcripts and are not made publicly available, however, authors may be contacted for information.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted January 09, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
User testing of a Diagnostic Decision Support System with Machine-assisted Chart Review to Facilitate Clinical Genomic Diagnosis
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
User testing of a Diagnostic Decision Support System with Machine-assisted Chart Review to Facilitate Clinical Genomic Diagnosis
Alanna Kulchak Rahm, Nephi A. Walton, Lynn K. Feldman, Conner Jenkins, Troy Jenkins, Thomas N. Person, Joseph Peterson, Jonathon C. Reynolds, Peter N. Robinson, Makenzie A. Woltz, Marc S. Williams, Michael M. Segal
medRxiv 2020.08.21.20179580; doi: https://doi.org/10.1101/2020.08.21.20179580
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
User testing of a Diagnostic Decision Support System with Machine-assisted Chart Review to Facilitate Clinical Genomic Diagnosis
Alanna Kulchak Rahm, Nephi A. Walton, Lynn K. Feldman, Conner Jenkins, Troy Jenkins, Thomas N. Person, Joseph Peterson, Jonathon C. Reynolds, Peter N. Robinson, Makenzie A. Woltz, Marc S. Williams, Michael M. Segal
medRxiv 2020.08.21.20179580; doi: https://doi.org/10.1101/2020.08.21.20179580

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Health Informatics
Subject Areas
All Articles
  • Addiction Medicine (70)
  • Allergy and Immunology (169)
  • Anesthesia (51)
  • Cardiovascular Medicine (456)
  • Dentistry and Oral Medicine (83)
  • Dermatology (55)
  • Emergency Medicine (160)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (192)
  • Epidemiology (5307)
  • Forensic Medicine (3)
  • Gastroenterology (198)
  • Genetic and Genomic Medicine (762)
  • Geriatric Medicine (81)
  • Health Economics (214)
  • Health Informatics (703)
  • Health Policy (363)
  • Health Systems and Quality Improvement (224)
  • Hematology (100)
  • HIV/AIDS (165)
  • Infectious Diseases (except HIV/AIDS) (5953)
  • Intensive Care and Critical Care Medicine (368)
  • Medical Education (105)
  • Medical Ethics (25)
  • Nephrology (83)
  • Neurology (777)
  • Nursing (43)
  • Nutrition (135)
  • Obstetrics and Gynecology (146)
  • Occupational and Environmental Health (236)
  • Oncology (482)
  • Ophthalmology (155)
  • Orthopedics (40)
  • Otolaryngology (98)
  • Pain Medicine (39)
  • Palliative Medicine (20)
  • Pathology (141)
  • Pediatrics (223)
  • Pharmacology and Therapeutics (138)
  • Primary Care Research (99)
  • Psychiatry and Clinical Psychology (868)
  • Public and Global Health (2043)
  • Radiology and Imaging (356)
  • Rehabilitation Medicine and Physical Therapy (160)
  • Respiratory Medicine (287)
  • Rheumatology (95)
  • Sexual and Reproductive Health (74)
  • Sports Medicine (77)
  • Surgery (110)
  • Toxicology (25)
  • Transplantation (30)
  • Urology (40)