“It reminds me and motivates me”: Human-centered design and implementation of an interactive, SMS-based digital intervention to improve early retention on antiretroviral therapy: usability and acceptability among new initiates in a high-volume, public clinic in Malawi

Introduction

Poor retention of people living with HIV (PLHIV) on antiretroviral therapy (ART) negatively affects individual clinical outcomes and viral load suppression(1). In sub-Saharan Africa (SSA), in particular, suboptimal retention results in missed targets for critical UNAIDS 95-95-95 targets by 2030(2). A systematic review of ART retention in SSA estimated 36-month retention at 65% (3). Early retention, within the first 6 months on ART, can be the most challenging, resulting in concentrated treatment disruptions within the first year on ART(4). Factors influencing retention on ART are complex and dynamic, including individual (e.g., poverty, poor access to care, work or familial obligations;), institutional (e.g. healthcare workers (HCW) shortages) and macro-level factors (e.g., COVID-19; global funding priorities)(1,5–7). However, once patients miss visits or have treatment interruptions, early retention efforts are most likely to successfully return patients to care(8).

Digital or mobile health (mHealth) interventions may help reduce retention gaps(9). One-way “blast” educational messages or reminders (pre-defined, no response) modesty improve retention and adherence (10,11). Adding interactive components, such as two-way texting, fosters direct patient to HCW communication and appears more effective at increasing engagement in care (12–14). Still, many retention apps fail (15–18). Across much of SSA, formidable technology, infrastructure, and connectivity challenges continue to stymie mHealth (19,20). Additionally, potentially increased workload or required levels of digital literacy among healthcare workers (HCWs) may limit adoption and potential scale-up (21–23). Policy, governance structures, and changing standards present additional challenges that require additional stakeholder involvement from inception (24). Lastly, the dearth of rigorous mHealth intervention evaluation reduces the evidence base to inform improvements (25). Nonetheless, mHealth shows immense promise to support patients, providers, and programs. Digital health innovations must respond with clearer evidence of success or failure on which to build (26).

To accurately reflect the value and fit of innovation to a specific programmatic need, mHealth programs should prioritize assessment of patient usability and acceptability. According to the ISO/IEC 9126–1, usability involves understandability, learnability, operability and attractiveness– or how an intended user correctly or incorrectly interacts with a digital innovation (27). Usability is often measured quantitatively. Acceptability is more focused on individual beliefs and practices or their attitude towards using the App and may be assessed qualitatively by asking users how they perceive the App to meet their needs and preferences (28). Usability and acceptability are both essential to designing effective mHealth interventions that may scale successfully(29). To further amplify diverse stakeholder engagement and to ensure that the App matches diverse user demands, incorporation of elements of human-centered design (HCD) – such as iterative user feedback – can also benefit mHealth usability and acceptability (30–33).

In 2019, University of Washington’s International Training and Education Center for Health (I-TECH) and technology partner, Medic, pioneered a two-way, text-based (2wT) approach for direct provider-to-patient communication for a randomized control trial (RCT) leveraging the open-source Community Health Toolkit (CHT) (https://communityhealthtoolkit.org/) and an HCD approach. The rigorous RCT demonstrated that interactive, hybrid, 2wT between male circumcision (MC) providers and patients during the 14-day post-operative period was as safe as in-person follow-up while significantly reducing provider workload(34). Patients and providers rated the 2wT system highly for usability and acceptability (35), and 2wT lowered costs while improving post-operative care(36). 2wT for MC success was likely due to adherence to many of the core characteristics of mHealth excellence, including prioritizing patient accessibility and acceptance, low technology costs, locally-led adaptation, strong stakeholder collaboration, and government partnership for sustained impact(37,38). 2wT for MC adapted successfully from RCT to routine settings (39), reaching over 22,000 men in post-MC care by mid-2022.

Building on the strength of the open-source, CHT-based 2wT and its HCD approach, I-TECH and Lighthouse Trust, the largest public provider of antiretroviral therapy (ART) in Lilongwe, Malawi, partnered with Medic to adapt the 2wT system to improve early ART retention. Lighthouse Trust is a World Health Organization (WHO) recognized centre of excellence, a public Ministry of Health (MoH) provider, and a leading organization in Malawi’s HIV/AIDS response (40). Lighthouse has five centers of excellence across the country, including the Martin Preuss Clinic (MPC). MPC is the largest facility with over 25,000 patients on ART. All diagnosed HIV-positive individuals who report to clinic reception are registered in the point of care (POC) electronic medical record system (EMRS). Since 2006, the Lighthouse Trust implemented a Back-to-Care (B2C) program to increase long-term ART retention. The B2C program identifies and traces patients who miss clinic appointment by at least 14 days and supports them to continue ART. Although B2C retention efforts at Lighthouse are robust (41,42) human and financial resource constraints stretch scare resources, leaving B2C service gaps. In public ART clinics in Malawi like Lighthouse, digital health interventions like 2wT could improve early retention while simultaneously reducing patient tracing workload and costs.

Therefore, in 2021, Lighthouse, I-TECH, and Medic developed a 2wT-based intervention to improve ART retention and initiated a quasi-experimental study to determine 12-month retention impact by comparing 500 2wT participants to a historical cohort of 500 new ART initiates at Lighthouse’s MPC Clinic in Lilongwe, Malawi. In this process-oriented paper, we describe the HCD process for the 2wT for early ART retention app. We present mixed-methods results on patient usability and acceptability with 100 early participants, 50 at 3-months and 50 at 6-months post 2wT enrollment, and report observations of participants interacting with the system. We hypothesized that using an HCD approach and intensive local participation in 2wT optimization would result in high usability and acceptability of 2wT from the patient perspective.

Methods

2wT System development

Technology background

Since 2008, Medic equipped more than 41,000 health workers to support more than 80 million care activities across 15 countries (44). Medic is the steward of the CHT open source project that currently supports almost 60 apps across 23 unique workflows (44,45), including 2wT-based apps in Malawi, South Africa, and Zimbabwe. Apps built using the CHT work on basic phones, smartphones, tablets, and computers with or without internet connectivity, in any language (46) and are tailored to meet key digital health characteristics recommended by the WHO(33). The core areas of functionality used by the Malawi 2WT system include peer-to-peer and automated interactive messaging, task management features (unread message; message not delivered), longitudinal patient records, routine syncing, and reports for routine monitoring (e.g. patient response rates, referrals, etc.).

2wT Human-Centered Design

The Medic-led HCD approach for 2wT for early ART retention [30] focused on patient and provider users’ needs, working in collaborative, multidisciplinary teams (Figure 1). Before 2wT system adaptation, system developers, healthcare workers (HCWs), monitoring and evaluation (M&E) teams, B2C staff and patients discussed and specified the context use. A brief formative research was conducted to inform adaptation and implementation. Then, a pilot of 50 ART patients solicited end-user experience in message context, optimal message delivery timing and language preferences (Chichewa or English) with the aim to launch a patient-centered, usable, and acceptable app. Formative message research and pilot testing results also ensured participants’ privacy and confidentiality were protected, making changes to messaging protocols to improve patient protections. Prior to updating the system messages, text messages were reviewed by expert patients (experienced ART patient mentors) for clarity and understandability. After system deployment, weekly technical feedback meetings were conducted to provide a platform to incorporate the voices and experiences of patient users in further refinement processes.

Figure 1:

Medic’s human-centered design approach adopted in the designing, developing and optimizing 2wT for MC post-operative care (30,47) [omitted]

System components and functionalities

The 2wT system (Figure 2) is a hybrid automated and manual texting interaction between patients and healthcare workers. In the 2wT system, weekly motivational messages are sent without a response request. Individually tailored appointment reminders are sent in anticipation of a visit with a response requested. Multiple responses and free text are accepted at any time. Messages are sent either in English or Chichewa according to patients’ preferences. Patients can send a message with any content at any time in any language. Generally, the 2wT officer responds to any SMS during routine clinic hours within 1-2 days of any message. Patients can opt out of motivational and/or visit reminder messages at any time.

• Download figure
• Open in new tab

Figure 2:

2wT flow diagram for ART retention

2wT Intervention: weekly motivational messages

The 2wT system broadcasts weekly motivation messages with a new health-related message sent each week; example are shown in Table 1. The messages are sent from the same Lighthouse aggregator number and contain no information related to HIV or AIDS. Messages repeat after approximately 6 months. For weekly motivational messages, there is no response required nor requested from patients. Patients may respond if they choose.

View this table:

• View inline
• View popup

Table 1: A sample of motivational messages sent by the 2wT for ART retention

2wT Intervention: visit reminders and responses

The 2wT system sends individualized clinic visit reminders with a response requested. Patients are asked to respond to the visit reminder messages with a single number: 1=yes if confirming clinic attendance or 0=no if they will not attend (Figure 2). A “yes” triggers, “See you soon!” and stops visit reminders for that specific visit. A “no” triggers automated, scripted texts to determine if someone has transferred or wants to change their visit date. The system also allows for free text interaction between 2wT staff and the patient to determine next steps. Patients can request an appointment date change, report transfers, or request a call. Patients receive automated missed visit alerts on days 2, 5, and 11 post appointment date when a clinic appointment is missed. Motivational and reminder visit messages are muted if or when specific ART outcomes are ascertained (stopping ART, transferring to other clinic and death). A 2wT officer prompt for B2C tracing is automatically triggered by the system when a patient misses a clinic appointment by 14 days.

Data collection and management

A brief, semi-structured, interviewer-administered questionnaire (Supplement 1) contained 16 closed-ended questions to assess 2wT usefulness, acceptability, perceived benefits, and barriers. In open-ended questions, patients were asked for feedback on the intervention, express concerns and suggest improvements.

Participants were also observed completing key 2wT tasks to assess the effectiveness of 2wT education at enrollment and to help improve participant’s adherence to 2wT prompts. Participants were shown four pictures of 2wT visit reminder messages and asked to indicate, using words or touching their phone, how they would: (a) confirm visit attendance, (b) transfer out to another clinic, (c) change appointment dates; (d) ask for help with something else (Figure 3). Results from assessment methods were combined to understand 2wT usability and acceptability from patient users.

• Download figure
• Open in new tab

Figure 3: Visit reminders, transfer reporting, and requests for help.

Results

Discussion

Likely as a result of the HCD-based adaption of the 2wT model, we found 2wT to be highly usable and acceptable as an early retention intervention for patients at MPC’s routine ART program. Although some patients, including those who are older or illiterate, may prefer reminder calls (14), these results indicate that the vast majority appreciated both the visit and motivational messages and would recommend 2wT for their friends. However, a quarter of respondents had trouble responding correctly to messages and there were some who expressed concerns with confidentiality, namely people seeing their messages. Compared to those in the 1-3 months in 2wT group, slightly more participants in the 4-6 month group correctly completed the tasks, showing that patient understanding and confidence likely increase over time. Although confirming visit attendance was considered simple, almost 30% found it difficult to report a transfer out or reschedule a clinic visit respectively. These findings suggest several strengths and areas of improvement for 2wT.

Iterative, highly-participatory, human-centered design (HCD) was central to App optimization. Routine B2C retention staff helped co-design, develop, test, and optimize 2wT through a series of formative input sessions that also included ART patients, monitoring and evaluation staff. Cycles of optimization strengthened app specifics including: message frequency; motivational message content; timing of delivery; and wording of the automated messages. Pre-testing was also critical: the pilot test conducted among 50 participants prior to the roll-out helped to identify system weaknesses, coding flaws, and bugs which were rectified before study implementation. The hybrid approach, blending automated motivation messages and individually tailored visit reminders with access to direct, interactive, healthcare worker-to-patient messaging, put a human face to the intervention. Although SMS was the primary form of communication, phone calls were made at patient request. This flexibility to respond to patients via SMS or phone also demonstrates prioritization of patient preferences. Patients believed and were assured that a member of the Lighthouse team was supporting them in the manner they preferred, likely contributing to 2wT satisfaction.

Multi-method formative assessment of usability also added rich data on which to improve the system for patient users. Quantitative, observational, and qualitative data each provided key information to ensure that 2wT was both easy to use and highly useful to new ART patients, resulting in a system that helped patients attend visits and stay motivated in care. Other research also used multiple methods to help ensure adaptation and optimization for the local context. In South Africa, an app for HIV self-testing showed that nearly all (98.7%) found the app easy to use and reported that 89.0% of participants could follow all the steps on the app without error(48). However, usability does not always indicate usefulness. In another South African app for HIV self-testing, high usability (87.8%) did not lead to participants to log onto the app: only 22% self-reported results (49). In Malawi, assessment of two electronic community case management Apps to support decision making for childhood illnesses found significant differences between the two apps in ease of use (50).

There are several areas for improvement in the 2wT system. First, participants received training on responding to the 2wT system messages at enrolment only, which likely contributed to observed differences in usability over time. Practice and repetition, complemented by refresher patient education (in-person or via posters) during subsequent clinic visits may be recommended as patients tend to understand the system over time. Second, similar to other studies (51–53), confidentiality concerns or fear of disclosure of HIV status using SMS for ART reminders, were reported by some participants. Several protections were implemented to reduce confidentiality concerns. 2wT message co-creation and pre-testing with patients attempted to ensure that both motivational and visit reminder messages were neutral and benign, reducing potential associations with HIV-related care or services. 2wT was also implemented only as opt-in: patients were consented after 2wT educational counseling that included a presentation with example messages. Those who self-reported phone sharing were also excluded. However, this may not have been enough as 17% of the participants reported that they were afraid someone else would see the messages. Additional protections should be explored. Lastly, 2wT is limited only to new ART enrollees and those with mobile phones and is intended to contribute to, and not replace, the routine Back to Care retention activities. However, reducing the literacy level required for participation or considerations of how to include those with shared phones would improve access to this retention support.

Limitations

As with all interviewer-led questionnaires, it is possible that patients were affected by the interviewer, a bias that could have influenced responses in a more positive or negative direction. Usability could change over time. One hundred patients were included in the usability assessment, but de-identified data linked to study-specific IDs resulted in errors linking usability results to patient demographics, reducing the sample size for demographics from 100 to 98. Lastly, SMS response rates, retention outcomes and cost data is forthcoming. It is expected that these results will shed additional light on whether high usability and acceptability will result in effectiveness of the 2wT approach on retention as expected.

Conclusions

2wT for early ART retention at Lighthouse Trust was purposefully designed with patients at the forefront. As a result, it appears that the App was highly usable and acceptable to those who chose to enroll in the retention support intervention, perhaps setting them up for a successful start to lifelong engagement in care. Although 2wT is not for all patients in all settings, the 2wT intervention approach has several strengths that are worthy of consideration for other digital health collaborators working to improve early ART retention. The local team led this process, ensuring that needs, priorities, and practices from Lighthouse, and not from international collaborators, were centered in the decision-making (54). 2wT was purposefully built using the open-source Community Health Toolkit (CHT), a global digital good. By leveraging an existing set of tools, and adapting from previous 2wT-based evidence (34), delays with App development were reduced while the potential for sustainability increased. Lastly, as 2wT is implemented in a routine, MoH ART services context, the HCD process reflected the realities of routine, and not research, settings, raising the likelihood of longer-term fidelity and potential scale-up of the 2wT intervention in other LMIC programs.