RT Journal Article SR Electronic T1 Heterogeneity impacts biomarker discovery for precision medicine JF medRxiv FD Cold Spring Harbor Laboratory Press SP 2022.02.14.22270972 DO 10.1101/2022.02.14.22270972 A1 Smith, Kenneth A1 Climer, Sharlee YR 2022 UL http://medrxiv.org/content/early/2022/08/01/2022.02.14.22270972.abstract AB Precision medicine is advancing patient care for complex human diseases. Discovery of biomarkers to diagnose specific subtypes within a heterogeneous diseased population is a key step towards realizing the benefits of precision medicine. However, popular statistical methods for evaluating candidate biomarkers – fold change (FC) and area under the receiver operating characteristic curve (AUC) – were designed for homogeneous data. Herein, we evaluate the performance of these metrics in heterogeneous populations. Using simulated biomarkers that are nearly ‘ideal’ for distinguishing subgroups of various proportions of the diseased population, we observe that AUC misses all up to subset size of 50% and FC misses all biomarkers entirely. We introduce a simple new measure to address this shortfall and run a series of trials comprised of simulated and biological data to demonstrate its utility for evaluating biomarkers associated with disease subtypes.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis study was funded the Alzheimers Association (AARG-22-925002), National Institute on Aging (NIA) grants 1RF1AG053303-01 and 3RF1AG053303-01S2, and research grants from the University of Missouri - St. Louis.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The source data are openly available at: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE15222I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll data produced in the present study are available upon reasonable request to the authors