Abstract
Background The popular beta-binomial approach to estimate the reliability of healthcare quality measures (Adams et al. 2010 New England Journal of Medicine) may yield grossly over-estimated reliabilities for providers with event rates equal to 0% or 100%.
Objective Improve the beta-binomial approach to yield more reasonable reliability estimates for providers with event rates equal to 0% or 100%.
Method We revise the beta-binomial approach by substituting Bayesian estimates with various priors for the crude event rates. We evaluate the new reliability estimates using Monte Carlo studies and two real-world measure examples.
Results and conclusion The revised beta-binomial approach based on Jeffreys non-informative prior yields more reasonable reliability estimates for providers with event rates equal to 0% or 100% and statistically outperforms the original beta-binomial approach regarding bias and standard errors.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
this study is supported by contracts with Centers for Medicare & Medicaid Services, an agency of the U.S. Department of Health & Human Services.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
↵* zhenqiu.lin{at}yale.edu
Data Availability
All data produced in the present work are contained in the manuscript