Abstract
Background An increasing number of healthcare quality measures are designed for annual reporting. These measures require an entire year of data to accurately report the percentage of patients who met the measure. Annual measures give providers latitude to prioritize clinical workload and patient needs; however, they do not provide a direct means to monitor performance throughout the reporting year. Although there are many possible methods for measuring annual measures at finer-grained timescales, our applied work showed that the most obvious methods could give a misleading and inaccurate view of progress throughout the year. Neither the definitions of the annual measures, nor the research literature, provided any guidance on the best methods for interim monitoring of annual measures.
Objective Our objective was to evaluate four different methods for monitoring annually reported quality measures monthly to best predict year-end performance throughout the reporting year.
Methods We developed four methods for monitoring annual measures by month: 1) Monthly Proportion: The proportion of patients with one or more encounters in the month who still needed to meet the measure at their first encounter of the month and met the measure by the end of the month; (2) Monthly Lookback Proportion: The proportion of patients seen in the month who met the measure by the end of the month, regardless of whether it was met in that month or previously in the reporting year; (3) Rolling 12 Month: The annual measure reported as if each month was the twelfth month of a twelve-month reporting period; and (4) YTD (Year-to-Date) Cumulative: The proportion of patients with one or more visits from the start of the reporting year through the month who satisfy the measure. We applied each method to two annual dental quality measures using data from two reporting years, and four different dental sites. We used mean squared error (MSE) to evaluate year-end predictive performance.
Results Method 3 (Rolling 12 Month) had the lowest MSE in 11 out of 16 cases (2 measures X 2 years X 4 sites) and lowest total MSE (262.39) across all 16 cases. In 5 of the 16 cases, YTD Cumulative had the lowest MSE.
Conclusions The Rolling 12 Month method was best for predicting the year-end value across both measures and all four sites.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
Research reported in this article was supported by award R01DE024166 from the National Institute of Dental and Craniofacial Research, National Institutes of Health.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This study was approved by the Committee for the Protection of Human Subjects at each institution. UTHealth acted as the main IRB where the study is approved under protocol number HSC-DB-14-1051: Implementing Dental Quality Measures in Practice
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Expanded introduction to clarify problem and importance.
Data Availability
Data will be made available upon reasonable request.
Abbreviations
- YTD
- year-to-date
- MSE
- mean squared error
- EWMA
- exponentially weighted moving average
- NQF
- National Quality Forum
- CUSUM
- cumulative sum
