Abstract
Importance Wide-spread distribution of rapid-antigen tests is integral to the United States’ strategy to address COVID-19; however, it is estimated that few rapid-antigen test results are reported to local departments of health.
Objective To characterize how often individuals in six communities throughout the United States used a digital assistant to log rapid-antigen test results and report them to their local Department of Health.
Design This prospective cohort study is based on anonymously collected data from the beneficiaries of The Say Yes! Covid Test program, which distributed 3,000,000 rapid antigen tests at no cost to residents of six communities between April and October 2021. We provide a descriptive evaluation of beneficiaries’ use of digital assistant for logging and reporting their rapid antigen test results.
Main Outcome and Measures Number and proportion of tests logged and reported to the Department of Health through the digital assistant
Results A total of 178,785 test kits were ordered by the digital assistant, and 14,398 households used the digital assistant to log 41,465 test results. Overall, a small proportion of beneficiaries used the digital assistant (8%), but over 75% of those who used it reported their rapid antigen test results to their state public health department. The reporting behavior varied between communities and was significantly different for communities that were incentivized for reporting test results (p < 0.001). In all communities, positive tests were less reported than negative tests (60.4% vs 75.5%; p<0.001).
Conclusions and Relevance These results indicate that app-based reporting with incentives may be an effective way to increase reporting of rapid tests for COVID-19; however, increasing the adoption of the digital assistant is a critical first step.
Introduction
Rapid antigen home-tests for COVID-19 are an important part of the Federal Government’s strategy for COVID-19 to expand testing access and availability throughout the United States.1 However, the distribution and scale-up of rapid home-tests for COVID-19 has been inconsistently accompanied by standard public health reporting mechanisms, challenging the ability to monitor rates of COVID-19 testing. It is important to understand more about individuals’ reporting choices, to create an optimal system for self-testing and surveillance. This study characterized how often individuals in six communities logged their home-test results through a digital platform and patterns of result reporting to state Departments of Health (DoH).
Methods
Say Yes! Covid Test (SYCT!) Intervention Communities and Procedures
The Say Yes! Covid Test (SYCT!) program, a partnership between the National Institutes of Health (NIH) and the Centers for Disease Control and Prevention (CDC), distributed over 3,000,000 self-tests to six communities across the United States from April to October 2021.2,3 More details about the SYCT! intervention can be found elsewhere.4,5 This analysis included data from six communities that finished test distribution before December 2021 and allowed users to report rapid antigen test results to the state department of health through a digital assistant: Louisville, KY; Indianapolis, IN; Fulton County, GA; O’ahu, HI; Ann Arbor/Ypsilanti, MI; and Chattanooga, TN. Test kits were distributed by online ordering and direct shipment to residents’ homes (direct-to-consumer or “DTC”) or local pick-up at community sites. Each household was restricted to ordering one test kit. Test kits in Kentucky, Indiana, Georgia, and Hawaii contained eight rapid home-tests, while those in Michigan and Tennessee contained 25 tests. A $25 gift card incentive was offered to participants in Indiana and Kentucky if they reported at least one test result to their state DoH through the digital assistant. The incentive was also offered in Georgia and Hawaii starting on October 4th. This study received non-research determination by the University of Massachusetts Chan Medical School Institutional Review Board.
Data collection
An optional online platform and accompanying app, developed by CareEvolution LLC, was launched with the SYCT! intervention as a platform for DTC orders, logging test results, and reporting results to the state DoH. All features of the digital tool were freely available and stored without personal identifiable information. The log feature allowed individuals to document their test dates and results for their records. Individuals were given the option to report each logged test to their state DoH through the digital assistant. For logged tests, test date, result (positive, negative, or invalid), and reporting decision (report or no report) were included in a data feed accessible to CareEvolution. For this analysis, reported tests included those reported with personal identifiable information or anonymously. Tests logged in the digital assistant from April 1, 2021 to January 12, 2022 were included in the analyses. Residents of Tennessee were unable to report tests to the DoH until June 24, 2021, so data before this point was excluded from reporting analyses.
Analyses
Total DTC orders and digital assistant users were calculated by community. The percentage of logged tests reported to the DoH by community was displayed graphically using R 4.1.1.6
Results
Distinct users for Logging Test Results
In total, 178,785 households ordered test kits through the digital assistant, and 14,398 households used the digital assistant to log 41,465 test results (Table 1). In Hawaii and Georgia, 100% and 66.7% of test kits were distributed before the onset of incentivization, respectively. Of the six intervention communities, Michigan had the greatest proportion of digital assistant users (23.5%) out of total online orders. Overall, 8.1% of individuals who used the digital assistant to order tests opted to log one or more test results on the platform. While the median number of tests logged by participants was one test, a small number of participants (3.1%) logged upwards of 15 test results in the digital assistant.
Reporting behaviors
Three-quarters (75.0%) of all tests logged in the digital assistant were reported to the state DoH (Figure 1, Supplemental Table 1). Sites with complete incentivization, Indiana and Kentucky, reported a higher proportion of test results to DoH than unincentivized or partially incentivized sites (p<0.001). The proportion of unreported results ranged from 9.2% (95% CI: 8.4-9.9) in Indiana to 34.9% (95% CI: 34.2-35.6) in Hawaii. In all intervention communities, positive results were less reported than negative results (60.4% vs 75.5%; p<0.001).
Discussion
Digital assistant users who logged test results made up a small proportion, only 8.1%, of households who ordered DTC test kits through the digital assistant. However, of those who logged test results, approximately 75% reported their results to their state’s DoH. The high usage of the digital assistant for ordering test kits demonstrates that the digital assistant was accessible to intervention communities. This implies that low usage of the digital assistant for logging tests may be due to inadequate community education about the importance of tracking and reporting home-test results. Further, there was a difference in reporting of tests based on result, with positive test results significantly less reported than negative results. It is important to understand and address the hesitations behind reporting positive tests.
The proportion of unreported tests was nearly three times higher in Tennessee and Hawaii compared to Indiana and Kentucky. This difference may be due to differences in incentivization structures, as participants in Indiana and Kentucky were incentivized to report tests throughout the intervention. Alternatively, test distribution in Indiana and Kentucky occurred in October 2021, following the Delta surge. Community awareness about the importance of reporting rapid antigen test results may have increased at this time relative to previous sites.
The high proportion of app users reporting their results to the DoH in Indiana and Kentucky suggests that app-based reporting systems may be successful in facilitating the reporting process when paired with incentives. However, the challenge remains drawing people to use the digital assistant, as evidenced by the low uptake of the digital assistant for testing purposes. Symptom-based participatory surveillance through digital applications has been used successfully for monitoring influenza-like-illness, among other infectious diseases, and rapid testing offers great opportunity to build on these technologies to rapidly ascertain changes in community prevalence of infection.7,8 Other means of improving uptake of the digital assistant or other reporting mechanisms should be explored further to maximize the value of these interventions.
Strengths and Limitations
This report offers a unique look into COVID-19 test reporting behaviors of nearly fifteen thousand digital assistant users throughout the United States. However, there are limitations to this data. The number of digital assistant users is quite small compared to all intervention participants, and with the current data, we are unable to assess the demographics or socioeconomic status of digital assistant users, nor how digital assistant users compare to non-users.
Conclusion
Three-quarters of those who used the digital assistant for testing reported their results to the DoH, indicating that app-based reporting may be an effective way to increase reporting of rapid tests for COVID-19. However, the relatively low voluntary uptake of the digital assistant indicates that user-centered strategies may be necessary to maximize digital assistant usage.
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
Funding
This study was funded by the NIH RADx-Tech program under 3U54HL143541-02S2.
Conflicts
VK is principal, and TS, SS, CN, and EH are employees of health care technology company CareEvolution. DDM reports consulting and research grants from Bristol-Myers Squibb and Pfizer, consulting and research support from Fitbit, consulting, and research support from Flexcon, research grant from Boehringer Ingelheim, consulting from Avania, non-financial research support from Apple Computer, consulting/other support from Heart Rhythm Society. LG is on a scientific advisory board for Moderna on projects unrelated to SARS-CoV-2
Acknowledgments
We are deeply grateful first to the local communities of Louisville, KY; Indianapolis, IN; Fulton County, GA; O’ahu, HI; Ann Arbor/Ypsilanti, MI; and Chattanooga, TN, including the local health officials and second to our collaborators from the National Institute of Health (NIBIB and NHLBI) who provided scientific input into the design of this study and interpretation of our results, but could not formally join as co-authors due to institutional policies. We received meaningful contributions from Drs. Bruce Tromberg, Jill Heemskerk, Rachael Fleurence, Andrew Weitz, Krishna Juluru, Felicia Qashu, Dennis Buxton, Jue Chen and Erin Itturriaga.
