PT  - JOURNAL ARTICLE
AU  - Yashoda Sharma
AU  - Katerina V. Djambazova
AU  - Celine D. Marquez
AU  - Kate Lyden
AU  - Jennifer C. Goldsack
AU  - Jessie P. Bakker
TI  - A systematic review assessing the state of analytical validation for connected, mobile, sensor-based digital health technologies
AID  - 10.1101/2023.05.22.23290371
DP  - 2023 Jan 01
TA  - medRxiv
PG  - 2023.05.22.23290371
4099  - http://medrxiv.org/content/early/2023/05/23/2023.05.22.23290371.short
4100  - http://medrxiv.org/content/early/2023/05/23/2023.05.22.23290371.full
AB  - Background Despite the recent proliferation of digital health technologies (DHTs), there is a lack of formal, industry-wide standards to evaluate the performance of the product’s algorithm in terms of its ability to measure, detect, or predict a clinical state. The advancement and successful use of DHTs in medicine requires that all stakeholders – clinicians, patients, payers, regulators, pharmaceutical companies, and the medical products industry – have a common understanding of what it means when a DHT has been analytically validated.Objective We conducted a systematic review to assess the state of the science on analytical validation for DHTs, using the criteria established by the V3 Framework and EVIDENCE checklist.Methods The systematic review was conducted according to the PRISMA guidelines. A multi-tier PubMed search identified studies published between April 15, 2020, and April 15, 2023, on analytical validation of DHTs; thereafter, each paper was assessed against the EVIDENCE checklist items specific to analytical validation. All studies were required to report quantitative data collected from a connected, mobile sensor product for the measurement, diagnosis, and/or treatment of a behavioral or physiological function, and compare the outcome measures to an established reference standard.Results Of the 1201 papers identified in the literature search, we identified 303 reporting the results of a DHT analytical validation study. The most prevalent therapeutic areas of focus were neurological (26%), cardiovascular (18%), and sleep conditions (17%). Health outcome categories most frequently captured by DHTs were gait (15%), heart rate/rhythm (15%), blood pressure and/or arterial stiffness (11%), sleep staging (10%), and mobility (9%). Only 208 papers (69%) reported all components of the EVIDENCE checklist focused on analytical validation, with the exception of software version and race/ethnicity, thereby meeting our definition of high-quality evidence reporting.Conclusion We are encouraged by the emerging literature evaluating whether outcome measures assessed by DHTs adequately reflect the physiological or behavioral parameter of interest; however, the quality of reporting is not yet sufficient to ensure the advancement of digital clinical measures that are fit-for-purpose for all members of a defined population and eliminate the need for redundant studies. We recommend that journals publishing analytical validation studies require the use of the EVIDENCE checklist as a reporting standard for these manuscripts.Competing Interest StatementJPB reports income and/or other financial interests in Apnimed, Koneksa Health, Philips, and Signifier Medical Technologies. Funding StatementThis study did not receive any funding.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesData are available upon request.AVAnalytical ValidationBioMeTsBiometric Monitoring TechnologiesCONSORTConsolidated Standards of Reporting TrialsDATAccDigital Health Measurement Collaborative CommunityEVIDENCEEValuatIng connecteD sENsor teChnologiEsIEEEInstitute of Electrical and Electronics EngineersPRISMAPreferred Reporting Items for Systematic Review and Meta-AnalysisV3Verification, analytical validation and clinical validation