Centering healthcare workers in developing digital health interventions: usability and acceptability of a two-way texting retention intervention in a public HIV clinic in Lilongwe, Malawi

Study Setting

Lighthouse Trust is a WHO-recognized Center of excellence (COE) that operates two clinics in Lilongwe, Malawi [33]. The 2wT intervention was implemented in the largest COE, Martin Preuss Centre clinic (MPC), in Bwaila Hospital located in Lilongwe, Malawi. MPC currently has over 24,000 clients alive in care.

Participants

The overall 2wT intervention was designed with and for new ART initiates (within 6 months of starting ART) and HCWs at MPC. HCW participants in the HCD-focused formative research and usability study were selected by purposive sampling to identify participants based on association with retention efforts, including 2wT. For the formative HCD research using informal feedback sessions, eligible HCW participants included key stakeholders such as: departmental managers, M&E team members (data officers and IT officers), Expert Clients (MPC client-to-client ART mentors), and Back to Care staff, including field tracers and retention officers. For the key informant interviews (KIIs), only HCWs actively engaged in the 2wT implementation were eligible for interviews.

Technology Overview: The Community Health Toolkit (CHT)

2wT system was built on a foundation provided by the open-source Community Health Toolkit (CHT), a free, open-source digital health global good to advance universal health coverage [34]. The CHT Core Framework, stewarded by Medic and released under AGPL-3.0, is a global good [35] that supports community health workflows, thereby decreasing the time and resources required to build full-featured, reliable, interoperable, secure, and ready-to-scale digital health applications (apps) to improve client care rather than code from scratch. The development of the CHT adheres to WHO’s recommended principles for digital development [36]. Apps built on the CHT Core Framework are highly customizable, support multiple languages, run offline, and work with basic feature phones (via SMS), smartphones, tablets, and computers to support integrated care coordination and delivery. While the CHT is designed for community level personas, it is highly extensible through integrations with complementary Android apps, electronic medical records, messaging systems, and national health information systems to meet various needs such as bidirectional messaging, closed-loop referrals, aggregate reporting and any other additional functionality. In 2021, CHT powered applications supported 22.7 million caring activities and more than 41,400 CHWs in 16 countries across Africa and Asia [34].

Adapting 2wT from VMMC to for ART via Human Centered Design

The core components of the target 2wT system were drawn from the VMMC intervention and adapted collaboratively by Malawian ART stakeholders and included: (1) automated workflows that send weekly motivational messages to promote wellbeing; (2) individually-tailored SMS reminders to clients with upcoming visits, with a response requested; and (3) enabling open-ended SMS texts between clients and HCWs to allow clients to reschedule visits, report transfers, or report other logistics concerns. Medic designers led participatory HCD activities, collaborating closely with a multi-disciplinary team of clients, healthcare workers, supervisors and software developers to specify the system requirements for the local context. This approach considers the motivations, concerns and behaviour of the end users and facilitators across the design process, limiting time and resources wasted on creating features that do not benefit, are not relevant, or are unacceptable to the target population [37-41]. At multiple stages, key stakeholders were presented with updated workflow drafts, including features for automatic flags for clients who miss clinic visits, task management and prioritization features, longitudinal client records, data collection forms, routine syncing, and dashboards for routine monitoring. Iterative improvements were informed by analysing users’ system usage patterns and insights gleaned during feedback sessions. Medic’s continuous HCD approach had four dynamic, non-linear phases (Figure 1):

Data collection

Data collection took place over three distinct phases: 1) informal feedback sessions with 2wT stakeholders, 2) a small pilot, and 3) key informant interviews (KIIs).

Phase I: formative HCD co-creation research and informal feedback sessions

We conducted four feedback sessions from October and November, 2020. Sessions included various MPC stakeholders involved in client retention to understand their needs and context, informing the 2wT design. MPC ART clients were also involved in informal feedback. 2wT HCWs conducted two practice educational sessions with 8 clients who volunteered to contribute (3 men and 5 women) to inform development and improvement of 2wT recruitment and educational materials. Documentation of formative feedback sessions was collected via notes and program reports from 2wT staff.

Phase II: Evaluation from a small-scale pilot and subsequent 2wT app improvement

A soft 2wT pilot took place in June 2021 with 50 new ART clients, identifying strengths of the system and key areas of improvement before launch. Results during and after the pilot were discussed with 2wT HCWs, M&E staff and Medic teams via a series of five joint remote design sessions. This series of discussions focused on clients and HCWs feedback and an analysis of system usage trends. Each session built on the previous insights and inputs, focusing on optimizing 2wT for HCWs (system users, M&E and 2wT data users) and clients.

Phase III: Usability perspectives from HCWs post-implementation

2wT launched in June, 2021. Six months later, we conducted ten KIIs with healthcare workers involved in 2wT implementation, supervision, M&E, and IT support over a period of 2 weeks using a facilitator guide. The KII guide was pre-tested and revised by the study team. A trained facilitator from Lighthouse but external to the 2wT study team conducted the interviews. Participants were recruited, briefed on the purpose of the activity and written consent obtained for recording by those who chose to participate. Each interview session took place in English and lasted for approximately 30 mins. All data collection tools and reports were uploaded to a secured folder only accessible by the study researchers.

Data Analysis

For phases one and two, informal program reports were reviewed and synthesized by the 2wT study team. For the KIIs, we generated verbatim transcripts with Otter.ai software; recordings were deleted once transcribed and the transcripts anonymized. We conducted our analysis using NVivo 12 Pro qualitative text analysis software (QSR International. Burlington, Massachusetts). We used a few transcripts to develop and refine a codebook using inductive and deductive approaches and coded all interview scripts. The team collaboratively reviewed and discussed the resultant codes to formulate themes based on the DEPICT method of participatory qualitative analysis [42] that included HCWs in review of initial codes and preliminary results determination.

Ethics

The parent study, including the usability and acceptability components, was reviewed and approved by the University of Washington and the Malawi National Health Sciences Research Committee. Participants in KIIs provided written informed consent. No other identifiers were produced nor utilized for formative data analysis in phases one or two.

2wT system flow

In brief, the 2wT system (Figure 2) is a hybrid automated and manual texting interaction between clients and HCWs that includes both weekly general motivational messages about general wellness sent to all enrolled clients and appointment reminders sent to each client in anticipation of their specific visit with a response requested. Responses yield follow up actions for HCWs such as confirming clinic transfers, visit updates and referrals (Figure 3). If a visit is missed, clients receive up to 3 additional reminders until they either return to the clinic or get referred to B2C for tracing. SMS is free for clients. Messages are sent either in English or Chichewa according to clients’ preferences, but clients can send an SMS at any time in any language. Clients can opt out of motivational and/or visit reminder messages at any time. 2wT officers usually respond within 1-2 days. Additional details on the 2wT intervention were described previously [43].

• Download figure
• Open in new tab

Figure 2.

2wT flow diagram for ART retention [43]

• Download figure
• Open in new tab

Figure 3.

2wT actions completed by HCWs

Phase I Outcomes: Formative HCD co-creation and informal feedback

Several key areas of client-focused improvements were identified across groups and addressed. First, for clients, 2wT is opt-in only, reducing risks of confidentiality or disclosure. To improve inclusion of, and acceptability of 2wT for clients, the level of literacy was reduced to a minimum and available in Chichewa and English, as selected by the client. Expert clients helped write, review, revise, and translate adherence and motivational messages; expert clients were identified as the best group to conduct 2wT education for new ART clients. 2wT educational materials were augmented to include a client flow chart to guide HCWs in explaining to clients how the message flow worked, how to respond, and how to stop messages. HCWs received a flip book and poster to better explain to clients how to interact with the system with illustrations for participants on how to respond with a “1” or “0” and to improve clarity on SMS communication via free text messages. The neutrality of all text messages was reviewed to ensure no reminder nor motivation words would expose the HIV status of the client (e.g., HIV, ART, adherence, clinic, etc.). With the opt-in and clear education for clients before voluntary enrolment, valid concerns were reduced about others/or spouse seeing the text messages where there is risk of disclosure or abuse. HCWs also informed the SMS timing for clients who were potential defaulters (those who missed their visits by >14 days), landing on SMS reminders at 3 times after a missed visit and before 14 days.

For HCWs, additional sensitization confirmed that 2wT would augment, not replace, other early retention interventions. To reduce redundancy between 2wT-focused interventions and other expert client-led or phone reminder systems, SMS reminder timing would be aligned with other B2C interventions for standardization. Client recruitment would take place during routine B2C recruitment and coincide with routine locator and tracing form completion, reducing impact on Lighthouse routine operations. 2wT activity flows and HCW workload were discussed, including discussions of potential workload increases (2wT interaction on the system) and decreases (reduced tracing if clients were maintained in care). As a result, one HCW was designated as the “2wT officer,” and accountable for daily review and responding to client SMS in the morning and afternoon, following up with calls when needed. A set of standard operating procedures (SOPs) for 2wT to EMRs data exchange were created, providing steps to update visit dates, de-identify shared data, increase security, reduce system errors and smooth data flow. Concerns about reminding clients accidentally after they actually attended a visit were diminished due to regular synchronization of data in EMR and 2WT system to prevent or reduce false referral for missed visit reminders or tracing.

Phase II Outcomes: Pilot evaluation and subsequent 2wT app improvement

The pilot revealed challenges in the EMRs to 2wT data exchange that were addressed via joint working sessions focused on data exchange using automated and manual approaches. THe resulting process was a one-way flow of data from the EMR to 2wT allowing only offline EMR data access for 2wT. The team explored the data pipelines and information needs, including how records would be identified and uniquely mapped to the specific clients files. Protections for client data were strengthened, including data export for consenting participants with limited variables relevant to appointment dates and ART outcomes only. Concerns around dealing with EMR outages led to the inclusion of data collection forms which were used to manually update appointment dates. Outcomes captured in 2wT were retrospectively entered into the EMR system. For cost reduction and efficiency, the study team applied for a short code using both Airtel and TNM cellular services for optimization, a decision that allowed all clients to communicate for free via 2wT but delayed the launch by several months. Additional concerns about potential delays or failed SMS resulted in several additional M&E processes including measuring the timeliness of SMS interactions, assessment workload, and message failure rate reviews. System usage analysis of messaging interactions revealed errors in the system such as clients continuing to receive messages after they had transferred clinics and restrictive SMS broadcasting rules that return messages as expired or failed if a phone number is absent on the network or off for over 12 hours.

Phase III Outcomes: Usability perspectives from HCWs post-implementation

Health care workers suggestions on 2wT scale-up

Several HCWs noted the need for additional staff if the 2wT intervention expands to include all clients and not new initiates, recognizing that, “if we are to expand, we will be looking at the whole cohort which is too big, which means everyone has to pass through this [2wT enrolment] room making it to be a lot of work.”[KII9]

For 2wT system improvements, HCWs suggested that 2wT should enrol multiple numbers per client, including numbers from different telecoms.

“On the message, you’ll find they have used another number, on the registration they use another number, when they are coming, they also use another number. Yeah, so we need to identify that this is the very same person but how are we going to identify that? It’s by asking them when they come for enrolment. Do you have other numbers, then we can be able to know.” [KII5].

There were also policy related suggestions. Several hoped that clients without phones could enroll for 2wT via phones of friends or buddies by allowing clients to enroll in reminders via a trusted number:

We would meet some clients who said that “it is my husband who has a phone” or “I live with my mother”, so if it was possible that we could be using phones of those that they feel they could disclose to, this can make it easy to reach a lot of contacts because those who do not have phones have a guardian with a phone.” [KII10]

To improve 2wT acceptability for clients, and to engage more clients who could benefit from 2wT, there will need to be outreach and advocacy to make sure clients know about 2wT. I think when it is being scaled up; there will be that sensitization to make people aware of the programme, both for the clients, and for the providers, that they know what is going on” [KII1]. 2wT messages could be further simplified so that very low literacy clients could enrol. One HCW explained:

My observation is that not everyone can get involved in [2wT], just like another study, not everyone is eligible. But 2wT is restricted to a certain category of people that have the capability of having a phone…Considering our literacy rate in Malawi, and also the economic challenges that we are facing, some clients won’t be able to get into 2wT. Though they would love to be part of the study, but they wouldn’t qualify [be eligible] [KII7]

Other interesting comments outside of key themes from HCWs are worthy of consideration. First, although SMS may be less expensive when compared to the time and costs incurred making phone calls or travelling to trace clients in the community, one HCW expressed that what “has not been really explored more is on the SMS. The cost of SMS for Malawi, they are quite a bit higher as compared to other countries” [KII1]. Another HCW observed that 2wT might reduce opportunities for in-person counselling since when a client “missed an appointment, he/she is supposed to be counselled by the back to care team. Now with texting that means the physical interaction will not be there, so no kind of counselling will be provided to the client for future appointment.” [KII7].

Lastly, HCWs noted that clients may want more information from the 2wT than simply to discuss visits. “Some clients may have [ART] questions that need to be responded to through the 2wT system, but we do not want to discuss clinical things using the system because the personnel in the two-way texting and reply to the texts are not clinical personnel.”[KII9]