Abstract
Client attendance is vital for the success of HIV vertical transmission prevention programs, yet 23.4% of clients missed follow-up appointments after enrolling in a community health worker-led program (n=24,807, Aug-Dec 2022). Predicting which clients are most likely to miss appointments could enable targeted interventions to improve retention. While machine learning appears well-suited for this prediction task, its effectiveness compared to community health worker judgment remains unexplored. We evaluated three machine learning approaches—logistic regression, balanced random forest, and gradient-boosted trees—trained on client enrollment records (n=51,297 training; n=18,577 test) and compared their performance to predictions by community health workers (n=61), who possess direct client interactions and contextual insights. Machine learning models achieved modest predictive performance, with the balanced random forest showing the best accuracy (ROC AUC=0.689). Community health worker predictions similarly showed low accuracy despite their rich contextual knowledge, suggesting attendance is fundamentally difficult to predict. Qualitative insights identified complex and dynamic barriers including stigma, transportation difficulties, and competing personal commitments, often unpredictable at enrollment. These findings highlight fundamental limitations in predicting client attendance and suggest that merely accumulating additional data might not enhance predictive accuracy. Instead, resources might be better allocated to addressing systemic barriers identified by community health workers.
Competing Interest Statement
LM and MM were employed by mothers2mothers, the organization whose program data was used in this study. MO and AL declare no competing interests.
Funding Statement
The research team and mothers2mothers received funding from the Google AI for Social Good Program for 2021-22.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The ethics committee of University of New South Wales (UNSW) Sydney gave ethical approval for this work (HC230085).
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
Authors’ Contact Information: Matthew Olckers, olckersm{at}sun.ac.za, Stellenbosch University, Stellenbosch, South Africa; Alexander Lam, Hong Kong Polytechnic University, Hong Kong, alexander-a.lam{at}polyu.edu.hk; Lazola Makhupula, mothers2mothers, Cape Town, South Africa; Mhlasakululeka Mvubu, mothers2mothers, Cape Town, South Africa.
CCS Concepts:•Applied computing→Health informatics;•Computing methodologies→Classification and regression trees; Supervised learning by classification;•Human-centered computing→Field studies.
D Data Availability Statement
The data that supports the findings of this study are available from mothers2mothers. Restrictions apply to the availability of these data, which were used under license for this study. Data are available from the author(s) with the permission of mothers2mothers. The code used to replicate the results is available in the following repository: https://github.com/matthewolckers/m2m-predict-attendance
