What does it cost to expand two-way texting for post-operative follow-up? A cost analysis in routine voluntary medical male circumcision settings in South Africa

Introduction

For more than two decades, studies on adult voluntary medical male circumcision (VMMC) have demonstrated that VMMC is an efficient, safe, and cost-effective method to reduce the continued burden of HIV(1–3). Rates of moderate and severe adverse events (AEs) are less than 2% in VMMC programs at scale but multiple in-person post-operative visits within 14 days of the procedure continue to be recommended for all clients. In South Africa, despite few AEs, national guidelines require two in-person follow-up visits after VMMC, a largely unnecessary burden for most patients and their providers that may deter VMMC uptake(4,5).

Mobile health (mHealth) interventions in this context therefore offer an innovative way of utilizing digital technologies to reduce the burden of in-person visits for clients and healthcare workers. mHealth interventions can help the digitalization of overburdened healthcare systems, as well as assist with reaching remote or underserved areas by providing remote access to health services(6). Two prior randomized clinical trials (RCT) were conducted in routine VMMC clinics in Zimbabwe and South Africa, piloting a novel two-way texting (2wT) approach between eligible males ages 15-years and older and VMMC nurses(5,7). Both studies demonstrated that 2wT is safe, reduced follow-up workload, and is cost-effective compared to routine VMMC post-operative visits. However as both studies were done within the context of an RCT, the results did not explore the true costs of 2wT when implemented as part of routine VMMC care.

To further evaluate the effects of a 2wT in routine VMMC programs operating in South Africa, the International Training and Education Center for Health (ITECH) at the University of Washington, The Aurum Institute, Right to Care, and Medic built upon their RCT evidence by conducting a quasi-experimental, stepped wedge design (SWD) expansion study across three districts: an urban district in Gauteng province and both a peri-urban and a rural district in the North West province. The expansion study was conducted in two phases, intensive and maintenance, and 2wT was implemented as part of routine VMMC service delivery. Eligible males could opt-into 2wT follow-up as an alternative to scheduled, post-operative visits. During the intensive phase (January – October 2023 with follow-up through December 2023), sites continued to receive support from the research team, and the 2wT system continued to undergo revisions to increase usability in the routine setting. During the maintenance phase (January 2024, onward), sites received quality assurance support from the study team, preparing them to fully maintain the 2wT approach independently by the end of 2024. Results on clinical outcomes from the SWD that demonstrated 2wT safety and effectiveness were published separately(8).

Despite cost often being identified as a key outcome, there remains a lack of transparency and comprehensive cost evaluation of mHealth interventions(9,10). Scale-up of previous mHealth interventions in low- and middle-income countries (LMIC) has failed due to a lack of data on the cost of programs at scale(11). While the evidence base has grown in recent years, there remains a need for high-quality economic evaluation to better understand the complexities and implications of scaling up digital health strategies(12). Further evidence of economic evaluation using an implementation science lens is needed to support the scale-up of mHealth interventions in routine practice.

Therefore, to complement the safety and effectiveness evidence from the SWD, we embedded a costing study with three objectives: (1) assess the additive cost of implementing 2wT for follow-up during the intensive phase of the stepped wedge design SWD, (2) measure the costs of augmenting 2wT personnel to reach high 2wT demand during peak VMMC season, and (3) estimate the cost savings of 2wT compared to routine VMMC follow-up at scale from the payer perspective. By looking at costs of 2wT when implemented in routine VMMC clinics without RCT conditions, we aim to provide updated estimates on the costs associated with the use of 2wT in routine practice, during common VMMC campaign approaches, and inform the feasibility of 2wT expansion in routine practice.

Methods

Background

2wT intervention

The open-source 2wT technology and implementation approach for VMMC follow-up has been previously described in detail(7,13,14). Briefly, the 2wT approach provides an option for clients to choose interactive, text-based telehealth with a dedicated 2wT VMMC nurse in lieu of in-person visits the standard of care (SoC), for post-VMMC follow-up. The hybrid 2wT system combines automated education messages along with messages requesting a client response to confirm appropriate healing. 2wT allows for clients to interact with a clinician who can provide reassurance, wound care instructions, and refer them to in-person review for potential complications during the critical 14-days post VMMC procedure. 2wT participants can always seek in-person care when desired. 2wT does not require clients to have a smartphone; clients can use any basic mobile phone with SMS capability and communicate in various local languages.

For clinics, 2wT participants are managed via the open-source app that is downloaded onto smartphones, tablets, or desktop computers. Enrollment of clients in 2wT, education on using the 2wT system, and 2wT-specific post-operative counseling, as well as any needed or requested in-person follow-up visits for 2wT participants all take place at routine VMMC clinics (“spokes”), with care provided by existing VMMC teams. The 2wT dedicated nurse (“hub”), is the first line of communication with 2wT participants from all sites, providing clinical care via messaging, conducting initial phone tracing for clients who failed to respond to texts, and referring clients back to their 2wT enrollment VMMC clinic “spoke” for follow-up care if needed. The 2wT system includes quality assurance prompts. For the hub, if a client does not respond by day 3 after VMMC, the hub nurse receives a “Task” to follow up that client by phone, clearing the Task by documenting whether the client was reached and ok; reached and referred to care; or not reached so referred to the site for tracing (unknown client status). For the sites (spokes), clients referred to care from the hub generate a “Task” for them to complete after client review, noting whether the client was okay (no AE); was confirmed with an AE; or did not return (with options that client was reached and was ok or client not reached).

Stepped wedge design (SWD): implementation and results

The expansion study was implemented across study sites in two phases using a hub-to-spoke model. During the intensive SWD phase, routine VMMC clinics acted as the spokes and received support from the 2wT research team who provided training, quality assurance, and clinical oversight while also maintaining the centralized 2wT hub. Between January and December 2023, in the SWD, 2,856 of 6,842 clients opted into 2wT (37.8%) across three intervention waves and two platforms (SMS or WhatsApp). Among those with post-operative follow-up, the AE ascertainment rate was higher among 2wT participants (0.60%) than SoC clients (0.13%) (p = 0.0018). On average, 2wT participants had 2.1 fewer visits compared to SoC clients (p<0.001). Among 2wT participants, 1832/2586 (71%) responded via 2wT by Day3 and 2,069/2,586 (80%) responded via 2wT over 14 days. Lost to follow-up (LTFU), or no response or visit, was documented for 93 2wT participants (3.6%) and 342 (8.0%) SoC clients(8).

Following the completion of the intensive phase, the study transitioned to a maintenance phase, starting January 2024, during which the study-supported 2wT team was gradually withdrawn, with the aim of fully transitioning the 2wT system and management to local teams over an additional year of implementation.

2wT enrollment threshold: Identifying a 2wT quality gap during VMMC campaigns

In South Africa, similar to other high priority VMMC countries in SSA, VMMC clinics experience seasonal fluctuation: the months of June and July are often high-volume VMMC campaign periods, with smaller campaigns in March and September. A large proportion of annual circumcision productivity takes place in these higher demand periods, often with focus on reaching adolescent boys ages 15 and older who are on school holidays(15,16). During the intensive SWD phase, the 2023 VMMC campaign period saw 2wT enrollments spike from a weekly high of approximately 200 to over 450, resulting in approximately 900 clients simultaneously in the 2wT system in the first 14-days. This high-volume VMMC period and surge in 2wT enrollment created an extraordinary workload for the 2wT hub, resulting in delays in completing the tracing Task for 2wT participants who had not responded to 2wT texts by day 3. Figure 1 illustrates that during the 4-week campaign period, including only clients in June and July 2023, the total number of 2wT participants who did not respond by day 3 after VMMC was 499 of 1,464 (34%) of which 274/1,464 (19%) were not successfully traced by the hub. During the non-campaign months, 294/1,122 (26%) of 2wT participants did not respond by day 3, but only 7% of the Tasks did not result in successfully tracing by the hub. This comparison suggests that there is a 2wT participant to hub threshold at which the quality of 2wT-based care may diminish. A forthcoming paper will examine the 2wT tracing and referral cascade, including this gap in 2wT quality when the threshold is reached.

• Download figure
• Open in new tab

Fig 1. Resolution of tracing tasks by the hub for 2wT participants during campaign periods vs. routine VMMC levels, South Africa, 2023.

Results

During the intensive intervention phase of the SWD, 6,842 males aged 15 years or older underwent VMMC: 2586 (38%) opted to participate in 2wT for follow-up. From the SWD clinical outcomes, we found that on average, 2wT participants had 0.07 visits (SD 0.30) compared to 2.04 visits among routine clients (SD 0.75)(8). Routine clients had a higher rate of loss to follow-up (LTFU) at 8% compared to 2wT participants at 4%.(8)

Objective 3: Estimating cost savings of 2wT implementation in standard (non-research) clinical practice

The parameters used to calculate the cost savings of 2wT compared to routine VMMC follow-up are included in Table 4. For a district performing 10,000 VMMCs annually, enrolling clients in 2wT at the rate observed during the intensive phase of the SWD (40%) would increase costs by $0.06 per client. 2wT becomes cost neutral with 45% enrollment into the intervention. Increasing enrollment to 80% would reduce the mean cost by $0.46/client. Further scaled up to a scenario where 100,000 VMMCs are performed annually, at 40% enrollment in 2wT costs of follow-up would be reduced by $0.88/client, decreasing by $0.93/client at 80% enrollment.

The base-case compared the cost per client of 2wT to routine in-person follow-up, assuming implementation at the level observed during the intensive phase of the SWD. We assumed 10,000 annual VMMCs and an enrollment rate of 40% in 2wT, resulting in an increased cost of $0.06/client in the base-case scenario. In the univariate sensitivity analysis (Fig 2), the cost savings of 2wT for post-VMMC follow-up compared to routine in-person visits was most sensitive to the enrollment rate in 2wT, average number of follow-up visits attended by routine clients, and minutes spent by clinicians at post-operative visits.

View this table:

• View inline
• View popup

Table 4. (A) Cost parameters and (B) estimated cost savings per client of 2wT vs. routine VMMC follow-up for extended implementation of 2wT

• Download figure
• Open in new tab

Fig 2. Univariate sensitivity analysis of input factors comparing 2wT to routine care.

All variables that changed the incremental cost by more than $0.15 are shown. Colors correlate to input ranges of each parameter. (e.g., 32% enrollment rate in 2wT increases costs by $0.26/client while a 75% enrollment rate decreases costs by $0.45/client)

Discussion

When implemented in routine VMMC clinical practice at small scale, 2wT follow-up increased AE ascertainment and reduced LTFU, increasing the quality of post-operative care. At small scale, 2wT also raised costs by approximately $4/client – a modest additional cost to the payer balanced by fewer in-person visits, increased AE ascertainment, and decreased rates of LTFU. However, when operating at scale in a district performing a reasonable 10,000 VMMCs annually, scenario analysis suggests that 2wT becomes cost neutral with 45% 2wT enrollment; saves $0.29/client with 60% 2wT enrollment; and saves $0.46/client with 80% 2wT enrollment without consideration of the cost impact of improved quality, monitoring, and client engagement in care. During 4-week campaign periods, to ensure 2wT quality and adherence to protocols, adding temporary, dedicated 2wT personnel would cost an additional $0.81/client. Across all 12 months, even with additional human resources during peak VMMC operations, it again appears that 2wT becomes cost neutral when implemented on larger scales or with increased enrollment rates. This strongly suggests that 2wT could lead to significant health system savings. These results align with prior costing studies on 2wT for VMMC follow-up in Zimbabwe and South Africa, which found 2wT to be a cost-effective method to increase AE ascertainment and improve the quality of post-operative care at scale. Several discussion points inform 2wT scale-up decisions.

First, while our results differ from what was previously reported in the 2022 2wT RCT in South Africa(18), they provide an updated cost estimate for routine 2wT implementation. The RCT costing analysis found that 2wT could save $3.56 per client. In this SWD, we suggest that enrolling 4,000 2wT participants per year in a district that performs 10,000 VMMCs annually would only add a minimal $0.06 per 2wT client, with increased cost savings when a greater proportion of clients enroll. The difference in cost savings between RCT and more routine SWD conditions is likely due to several key implementation factors. First, the RCT did not include additional costs of maintaining the 2wT system, recurrent monthly technology fees, or equipment needed for systems operation, which were included in this analysis. Second, the RCT estimated costs separately in rural and urban areas, and found more cost savings in rural areas, while in the SWD, the clients were predominantly urban. Lastly, the RCT included time-in-motion observations to allocate time, and therefore cost, more accurately (e.g., client tracing activities, additional clinic costs for visits (reception, clerk), an activity that we could not undertake in this small, embedded costing study.

Second, personnel costs make up the largest proportion of 2wT-specific costs, demonstrating the continued need for adequate staffing for mHealth interventions to ensure quality implementation and monitoring. 2wT does not remove the need for VMMC client follow-up, but rather dramatically improves the efficiency of care. By no longer requiring in-person follow-up visits, 2wT reduces the time clinicians and staff at clinics need to spend on unnecessary clinic reviews. However, clinician oversight is still needed to monitor 2wT client healing and complete reviews for 2wT clients with desire or need for in-person care. In the SWD, during routine VMMC client volume, a dedicated 2wT-specific VMMC nurse spent an estimated 40% of their work time responding to clients with concerns via text message and phone call, if desired, directing them to return to their clinic for in-person follow-up. During high-enrolling campaign periods, a full time 2wT hub clinician was needed with support from an auxiliary intern at each 2wT enrolling site to complete critical activities like confirming 2wT enrollments on participant phones, alerting VMMC teams for tracing or referral tasks, and reconciling paper to digital records. 2wT makes VMMC follow-up more efficient and raises the quality of post-operative care when implemented according to standard operating procedures. 2wT reduces, but does not remove, the need for clinical oversight and quality care assurances. During scale-up, personnel costs could be reduced by using a trained enrolled nurse, a lower category of nurse, instead of a professional nurse, as with additional training they can offer the same quality of follow up care.

Third, by using the activity-based costing method, we provide clarity on the potential cost savings of 2wT while increasing the transparency around the costs of scaling up mHealth interventions. The lack of data on costs, especially in scale-up, remains a key barrier in implementing mHealth interventions, particularly in LMIC(9). By separating the costs of implementation during the intensive phase of the SWD into routine-administration and research-specific costs, we demonstrate the variation of cost during the research-supported phase versus the cost to the funder while transitioning to independent management. Referring to recommendations on the scope and disaggregation of costs, we reported estimates using a micro-costing approach to increase accuracy and separated costs by phase to delineate the long-term costs of scale-up(9,19,20).

Our study had several limitations. First, the cost savings of 2wT compared to routine in-person follow-up was estimated using data collected during the prior RCT, as data on post-operative visits and client tracing was not collected during the SWD. However, RCT tracing rates and time-motion data stemmed from both rural and urban clinics operated by the same implementing partner, reducing bias in tracing and provider time estimates. Second, the technology costs in 2wT changed over the study period; however, these changes are typical in the lifecycle of mHealth technology, including increasing monthly website-hosting or monthly data fees, indicating a challenge in costing mHealth interventions (20). Finally, we likely underestimated the costs associated with client tracing for both 2wT and routine clients. Due to lack of routine tracing data, we calculated tracing costs for clients identified as LTFU, ignoring the maximum amount of effort (3 phone calls and 3 home visits) that could have been undertaken for each. Therefore, our estimate may underestimate 2wT cost savings as routine clients were more likely to be LTFU.

Conclusion

Overall, evidence from this cost analysis of the 2wT approach in routine settings found that implementation would be cost-neutral to the payer with low enrollment and cost savings at scale. Combining with previous evidence that 2wT safeguards client safety while improving the quality of post-operative care, there are clear advantages to adoption of the opt-in 2wT approach for males who want to receive telehealth-based follow-up. Further scale-up would likely increase cost savings and dramatically reduce the burden of in-person reviews on clinics and clients.

Funding

Research reported in this publication was supported by the National Institute of Nursing Research (NINR) of the National Institutes of Health under award number 5R01NR019229, “Expanding and Scaling Two-way Texting to Reduce Unnecessary Follow-Up and Improve Adverse Event Identification Among Voluntary Medical Male Circumcision Clients in the Republic of South Africa.” Statistical support was provided by the University of Washington and its Fred Hutch Center for AIDS Research, a National Institutes of Health (NIH)-funded program under award number AI027757 that is supported by the following NIH Institutes and Centers: National Institute of Allergy and Infectious Diseases, National Cancer Institute, National Institute of Mental Health, National Institute on Drug Abuse, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Heart, Lung and Blood Institute, National Investigation Agency, National Institute of General Medical Sciences, and National Institute of Diabetes and Digestive and Kidney Diseases. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of Interest

None declared.

Supporting information

S1 Table. Costing tool for 2wT implementation. Modifiable Excel spreadsheet for 2wT costing during stepped wedge expansion study and for additional scenario analysis.