Abstract
Liberia launched its National Community Health Assistant Program in 2016, which seeks to ensure that all people living 5 kilometers or farther from a health facility have access to trained, supplied, supervised, and paid community health workers (CHWs). This study aims to evaluate the impact of the national program following implementation in Grand Bassa County in 2018 using data from population-based surveys.
We measured before-to-after changes in childhood treatment from qualified providers in a portion of the county that implemented in a first phase compared to those which had not yet implemented. We also assessed changes in whether children received oral rehydration therapy for diarrhea and malaria rapid diagnostic tests if they had a fever by a qualified provider (facility based or CHW). For these analyses, we used a difference-in-differences approach and adjusted for potential confounding using inverse probability of treatment weighting. We also assessed changes in the source from which care was received and examined changes by key dimensions of equity (distance from health facilities, maternal education, and household wealth).
We found that treatment of childhood illness by a qualified provider increased by 60.3 percentage points (95%CI 44.7-76.0) more in intervention than comparison areas. Difference-in-differences for oral rehydration therapy and malaria rapid diagnostic tests were 37.6 (95%CI 19.5-55.8) and 38.5 (95%CI 19.9-57.0) percentage points, respectively.
In intervention areas, treatment by a CHW increased from 0 to 81.6% and care from unqualified providers dropped. Increases in treatment by a qualified provider did not vary significantly by household wealth, remoteness, or maternal education.
This evaluation found evidence that the Liberian National Community Health Assistant Program has increased access to effective treatment in rural Grand Bassa County. Improvements were approximately equal across three measured dimensions of marginalization.
Introduction
Liberia has made significant gains in child health since the end of its prolonged civil war. Under-five mortality has reduced by approximately 15% since 2007.(1) Nationwide, these improvements were driven by substantial investments in public sector delivery of health care, strengthening of facility-based services and improved public health services.(2–4) Despite these gains, Liberia continues to suffer one of the highest rates of child death in the world, and Liberian children who live in rural areas are about 20% more likely to die before their fifth birthday than their urban counterparts.(1) This disparity results in part from higher barriers to care for easily treatable childhood illnesses, including higher rates of poverty, insufficient access to transportation, delays from greater travel time, lower health literacy, and, in some instances, lower quality of health services in rural areas.(5–7) Because they provide services directly in rural communities, community health worker (CHW) programs can be well suited to overcoming these barriers.(8–10)
In collaboration with the Liberian Ministry of Health, Last Mile Health, a non-governmental organization, implemented a pilot program of community health workers in Grand Gedeh County in 2012. The aim of the pilot program was to offer proof of principle data on whether trained, well-supplied, and supervised CHWs could increase access to preventive and curative treatment for malaria, pneumonia and diarrhea among children and expand maternal and reproductive health in rural communities. Early evaluation of this pilot project demonstrated before-to-after improvements in management of these childhood illnesses and uptake in facility-based delivery.(11) In a follow-up difference-in-differences analysis, a similar demonstration project in Rivercess County, Liberia, also showed substantial increases in childhood treatment from qualified providers in intervention areas and that care shifted from informal and traditional providers toward CHWs.(12)
Based in part on the early success of these initial pilot programs and lessons from the 2014-2015 Ebola epidemic response, Liberia adopted a National Community Health Assistant Program (NCHAP) in 2016 as part of a comprehensive strategy to extend the reach of the country’s primary health care system into areas more than 5km from a health facility. The program uses an integrated and standardized approach in which CHWs, called community health assistants (CHAs), are trained and deliver a package of interventions in their communities.(13,14) However, the impact of implementation under the full national program on access and uptake of health services has not been assessed. Further, more data are needed to assess whether the program has been implemented in a way that equitably includes the most vulnerable populations.
Our study seeks to evaluate the national CHA program implementation in rural Grand Bassa County, Liberia. The implementation was led by the Liberian Ministry of Health with technical and financial support from Last Mile Health. Because program implementation was phased for operational reasons, we can assess program impact by measuring changes in child health treatment in areas that have already implemented compared to areas that have not implemented yet. We aim to answer three questions. First, did the program achieve greater gains to child health treatment in intervention areas than seen in comparison areas where implementation has not yet begun? Second, if treatment gains are observed, did the source of care shift to CHAs and facility-based providers from less effective informal sector providers? Third, how did changes in child health treatment outcomes in implementation areas vary across three measures of inequity: relative poverty, geographic remoteness, and maternal education?
Methods
Setting, population, and services
Grand Bassa is a predominately rural county in south-central Liberia, with a population of approximately 220,000 people spread over approximately 8000 square kilometers who are served by 31 health facilities.(15) It is divided into eight health districts. The CHA intervention began in Grand Bassa County in June 2018 and provides services to households in communities more than 5 km from the nearest health facility. Because of resource constraints, the program rollout was planned in four phases, with two health districts per phase. In this paper, we report results from the first phase of implementation.
Through the national program, CHAs provide a comprehensive package of health promotion and treatment services, including modules focused on maternal, newborn, and young child health; community registration; surveillance for diseases of epidemic potential; and family planning. These community-based providers are trained on integrated community case management protocols and provided commodities to treat uncomplicated suspected malaria, acute respiratory illness, and diarrhea and to support referral of complicated cases to health facilities. CHAs receive a monthly stipend of $70 USD and are supervised by Community Health Services Supervisors who are mostly nurses, physician assistants, or midwives. The CHSS conducts supervision visits to each community health assistant twice monthly. There were 100 CHAs and 10 Community Health Services Supervisors at the time of the evaluation. All services were delivered following the standard of care under Liberian policy. Further details about CHA recruitment, training, supervision, incentives, and provision have been previously reported.(13,14,16)
Data Collection and Sampling
We conducted two population-based, stratified cluster-sample surveys in February-June 2018 (pre-intervention) and February-April 2019 (5 months after phase 1 implementation). The baseline survey included all eight districts in Grand Bassa but our analytical sample is limited to the four districts sampled at follow-up; the follow-up survey included the two implementation districts and the two non-implementation districts that were most comparable on baseline child health outcomes. At baseline, we mapped the county immediately before data collocation and then we took a simple random sample of communities, stratified by district, and a census of households within each selected community. At follow-up, we sampled communities with probability of selection proportional to size, stratified by implementation phase, and then selected 24 households per selected community by a modified random walk procedure. All women aged 18-49 in selected households were invited to complete the maternal and child health modules of the survey. We provide more details on the sampling approach in the methods supplement (supplement 1). Sample size was determined to estimate trends in programmatic indicators with adequate precision over the planned series of surveys.
Surveys were adapted from the Liberian Demographic and Health Survey and are provided in supplement 2 and administered face-to-face in Liberian Vernacular English and Bassa, the local language in the county. Instruments were translated to Liberian Vernacular English and back-translated to American English to check translation accuracy. Bassa does not have a commonly used written form, but all enumerators were bilingual and trained to administer the survey in both languages. We recruited enumerators who had prior experience administering surveys in this setting, and provided a five-day supplemental training on the questionnaire, its administration, and research ethics with human subjects. Enumerators recorded responses on Android mobile phones and uploaded data regularly to facilitate data quality assurance and enumerator supervision. We discarded data collected by one enumerator because those surveys failed a quality assurance check designed to ensure that enumerators spent enough time surveying each household to obtain accurate information. This resulted in data for 82 children (41 intervention and 41 comparison) being excluded from the follow-up analysis, which was 10.2% of children in that round of surveys. We tested for sensitivity to excluding these observations by re-running our primary analyses including them, and this check is provided in supplement 3.
Variables
Our primary outcome of interest was treatment from a qualified provider if a child has suspected diarrhea, malaria, or acute respiratory infection (ARI) in the two weeks preceding the survey. Suspected malaria and diarrhea were defined by maternal report of fever and diarrhea, respectively. Suspected ARI was defined as maternal report of cough plus fast or difficult breathing.(17) Qualified providers were defined as facility-based providers, community health assistants, or general community health volunteers (GCHVs, a cadre of community health workers that preceded the NCHAP). Other non-qualified provider types included drugstores, black baggers (informal drug dispensers), and traditional providers. For our primary outcome, we assessed care for any of the three conditions; we analyzed each condition separately as secondary outcomes. As additional secondary outcomes, we assessed quality of care using maternal report of oral rehydration therapy for diarrhea and rapid diagnostic testing for malaria. We also assessed what providers were sources of care as secondary outcomes to determine whether care providers changed when the community health assistant program was implemented and to directly assess causal mechanisms.
Our exposure of interest was residence in implementation areas rather than comparison areas in the period after implementation. We treated the following variables as potential confounders: child’s gender, child’s age in months (continuous), maternal age in years (continuous), maternal education (categorized as none, some primary school, or completed at least primary school), whether the mother preferred to complete the survey in English or Bassa, whether the child was reported as having only one illness or more, the number of children under age five residing in the household (continuous), whether the household resides in a community with a primarily agricultural or mining economy, distance to the nearest health facility (log-transformed continuous variables), and quintiles of household wealth. We calculated household wealth using the Filmer-Pritchett approach, which uses the first principal component in a principal component analysis of responses to a household asset index.(18)
Statistical Analyses
We used a difference-in-differences approach to estimate whether before-to-after implementation period changes were greater in the intervention than comparison areas. In the unadjusted analysis, we fit a linear probability model with indicator variables for intervention versus comparison area, year, and their interaction. We accounted for potential changes in the composition of the population over time by using inverse probability of treatment weighting (IPTW) to balance all of the covariates listed above except for residence in a mining community across all four time-by-intervention groups. We assessed balance using standard IPTW diagnostics (see supplement 1) and then fit an IPT-weighted linear probability model that otherwise had the same form as the unadjusted analysis.(12,19–21)
We assessed changes in provider types using survey design-adjusted tests for differences in proportions. We assessed equity in program outcomes by examining before-to-after changes in childhood disease treatment in the intervention areas, stratified by distance from the nearest health facility (dichotomized as 5-9.9km versus 10 or more km based on prior research on distance as a barrier in a similar Liberian setting (22)), household wealth (above or below the median), and maternal education (none versus some).
We conducted several sensitivity analyses (see supplement 3). First, we could not include residence in mining rather than agricultural communities in the IPTW-adjusted models due to the small number of mining communities, but we conducted supplemental analyses restricted only to agricultural communities. Second, we accounted for confounders using regression adjustment rather than IPTW. Finally, we re-ran analysis including observations we excluded for data quality assurance, as described above.
All analyses incorporated sampling weights and accounted for stratification and clustering by Taylor linearized standard errors. The IPTW propensity score model incorporated sampling weights, so the final IPTW difference-in-differences model balances the weighted distribution of covariates and remains population-representative.(23) Analyses used Stata 17.0 We provide more detail on statistical methods in supplement 1, and statistical code is provided in supplement 4.
Participants gave verbal informed consent. Approval for the surveys was provided by the institutional review boards at the University of Liberia (#18-11-140), Liberia Institute for Biomedical Research (#EC/LIBR/012/037), Partners Healthcare (#2013P002480/PHS), and Georgetown University (#2013-1385).
Results
Among households approached, response rates were 90% in 2018 and 83% in 2019. We included 2538 households in 2018 and 1313 households in 2019. Within eligible households, 95% of listed women participated in 2018 and 98% in 2019; information about 90% of listed children was provided in 2018 and 98% in 2019. No variable had more than 0.6% missing data.
In the analysis sample, we included data on a total of 1351 children under the age of five in 2018 and 721 in 2019. Among children under five, the burden of at least one childhood disease in the 2 weeks prior to the survey was 70.0% within intervention areas at baseline, 49.9% within intervention at follow-up, 64.0% and 65.6% within comparison areas at baseline, and follow-up respectively.
Overall, samples were similar across years and areas prior to inverse probability of treatment weighting (Table 1); however more households in the comparison group were in mining communities in both surveys, fewer surveys were completed in English in 2019 for both intervention and control, fewer illnesses were reported for children in 2019 for both intervention and control, and wealth was higher in the control group in 2019. In all groups, IPT weighting produced approximate balance, as measured by standardized differences. We present full IPT weighting balance diagnostics and an IPT-weighted version of table 1 in supplement 1.
Characteristics of Children with Fever, Acute Respiratory Infection, or Diarrhea in the Two Weeks before the Survey (Prior to Inverse Probability of Treatment Weighting)
Changes in Childhood Treatment and Care Source
In IPT-weighted models, treatment for all three illnesses from a qualified provider increased by 56.4 percentage points (95% CI: 45.7, 67.1) in intervention areas and decreased by 3.9 percentage points (95% CI: −15.2, 7.5) in comparison areas, a difference-in-differences of 60.3 percentage points (95% CI: 44.7, 76.0; p < 0.001) (Table 2; Figure 1). The disease-specific difference-in-differences between intervention and control areas were 64.5% (95% CI: 47.6, 81.4; p < 0.001) for fever, 74.3% (95% CI: 58.5, 90.0; p < 0.001) for diarrhea, and 48.2% (95% CI: 17.5, 79.0; p = 0.002) for ARI. The difference-in-differences for treatment of diarrhea with oral rehydration therapy (ORT) and receipt of malaria rapid diagnostic tests (RDT) for fever were 37.6% (95% CI: 19.5, 55.8; p < 0.001) and 38.5% (95% CI: 19.9, 57.0; P < 0.001), respectively. The regression-adjusted and unadjusted models showed similar results. Results did not change meaningfully in any of the sensitivity analyses (supplement 3).
Difference-in-Differences in Childhood Illness Treatment
There was a significant increase in the proportion of children receiving care from community health assistants (0% to 81.6%; p<0.001) from baseline to follow-up in the intervention areas (Figure 2). In the comparison areas, care from a community health assistant also increased from 0% to 2.0% but this was not significant (p=0.154). The increase in care seeking in the intervention area was accompanied by decreases in care seeking to non-qualified providers including drugstores and informal providers, with similar changes not seen in the comparison areas.
Program Equity
At baseline in the intervention areas, treatment from a qualified provider was 14.6 percentage points more common for children living within 10km from a health facility than those farther than 10km. Treatment increased by 55.3 percentage points (95%CI 44.2, 66.3) after implementation in the farther communities and 63.0 percentage points (95%CI 48.9, 77.2) in the closer communities. The 7.7 percentage point widening of the gap was not significant (95%CI −10.2, 25.7, p=0.395) (figure 3). There were no significant differences in treatment from a qualified provider at baseline in intervention areas by maternal education or household wealth. After implementation, treatment by a qualified provider increased by 60.8 percentage points (95%CI 51.2, 70.3) among children whose mothers had no formal education, compared to 44.3 percentage points (95%CI 25.4, 63.3) for children of mothers with some education. The 16.5 percentage point greater improvement was not significant (95%CI −1.9, 34.8, p=0.078). Treatment from a qualified provider improved by 57.1 percentage points (95%CI 44.3, 69.9) after implementation for children in households below median wealth and 59.1 (95%CI 46.8, 71.3) for children in households above median wealth. The 1.9 percentage point difference in improvements was not significant (95CI CI - 14.4, 18.1, p=0.816)
Discussion
This study finds evidence that Liberia’s National CHA Program led to substantial improvements in treatment coverage for childhood fever, diarrhea, and ARI using a research approach designed to infer causality. Care sought from qualified providers increased by 60 percentage points more in intervention areas than comparison areas. Essential elements of care--ORT for diarrhea and RDT-based diagnoses for malaria—also increased by about 40 percentage points more than in comparison areas. In support of the role for the CHA program in causing these gains, we also found that care shifted predominantly to CHAs and sharply away from traditional providers and unregulated drugstores and informal drug dispensers, as well as a shift away from facility-based providers of about 15 percentage points. The substantial increases in childhood treatment improvements were seen irrespective of wealth, distance from health facility, or maternal education.
These findings are broadly consistent with the existing literature on CHW programs. Evaluation of precursor programs to Liberia’s National CHA Program found substantial improvements in childhood illness care seeking from a qualified provider, including consistent improvements in treatment of childhood fever, diarrhea, and acute respiratory illness by qualified providers.(11,12) Further, Liberian community health assistants have been found to have acceptable levels of knowledge about childhood disease treatment.(24) Finally, a recent analysis of Liberian District Health Information System data found that, in areas served by community health assistants after the launch of the NCHAP program, a little over half of malaria diagnoses had shifted from facilities to community health assistants, 95% of diagnoses were confirmed with RDT or microscopy, and diagnosis appeared to be sustained during the first nine months of the covid-19 pandemic.(25) While our evaluation only focuses on one portion of the country, Liberia’s national program now covers over 80% of communities greater than 5km from health facilities, and we would expect similar impact across the national program.(14)
Our findings add to a body of literature indicating that CHWs are an important aspect of rural primary care systems. Systematic reviews, however, have differed about the strength of evidence that community health worker programs produce meaningful improvements in child health outcomes.(10,26) These differences likely result from variation in both the package of interventions and delivery model employed by different community health worker programs and the context in which they are implemented. Perry et al. showed that the most effective CHW programs employ staff who are rooted in the communities they serve while also integrated into the larger health system, with salaries and supervision and supplies and able to successfully refer patients who need care beyond what the CHWs can provide.(27) Further, reasonable concerns are often raised about the difficulty of taking CHW programs to national scale.(28) To that end, our results join evidence from other countries that public sector-led national programs can achieve can meaningfully improve care for childhood illnesses when they are able to induce demand for community-based services.(29–33)
Our results also corroborate existing literature about the equity and reach of CHW programs. We found improvements were similarly large among groups that have been traditionally disadvantaged as those that are relatively advantaged – such as those further from health facilities, with lower wealth, and with lower education. We interpret these results as suggesting equitable reach and acceptability of the program. Prior research has generally found improvements in child health outcomes across maternal education, household wealth, and rural versus urban residence.(34) However, so far as we could identify, prior research has not examined program equity by distance within rural areas, even though distance is a well-known determinant of health outcomes that CHW programs are designed to address.(35) It is particularly important to assess equity by distance because CHW supervision, supply chains, and referral networks all become more difficult in more remote communities.
This study has limitations. We lacked statistical power to directly assess changes in childhood mortality as a result of the community health assistant program, though we would expect that increases in coverage of evidence-based interventions known to reduce amenable under-5 mortality would result in decreases in mortality, based on prior literature.(36) We are executing a subsequent study in 2022 that is powered to assess child mortality. We discarded one enumerator’s data because of concerns about data quality. However, because data were discarded evenly from both intervention and comparison areas, we do not have reason to believe it caused any bias and, in a sensitivity analysis, results did not change meaningfully when that enumerator’s data were included. Finally, our study is subject to the recall, reporting, and recording errors that can occur in survey research. We reduced these risks by using standardized Demographic and Health Survey items, building reporting validation into the data collection program, and through training and supervision of enumerators. Parental report of malaria diagnostic testing, in particular, has previously been found to be less than 70% sensitive, which may explain differences between improved fever care and RDT use.(37) Social desirability may have encouraged respondents to decrease reporting of use of unqualified providers, but this would not explain the magnitude of overall increases in coverage we observed. Data entry errors are inevitable, but in previous assessment of our data systems, we found recording error rates below 2%.(38)
Overall, this study provides evidence that Liberia’s National Community Health Assistant Program has resulted in significant improvements in the treatment of common child health illnesses, which are estimated to cause about half of under-five deaths in Liberia.(39) Not only did treatment coverage increased dramatically in intervention areas while it remained approximately constant in comparison areas, but the quality of care also improved along with a shift from unqualified providers mainly to community health assistants. All of these findings suggest that the Liberian National Community Health Assistant Program as implemented has been successful in strengthening the systems of care and reinforces the value of well-designed public-sector community health worker programs to expand the delivery of equitable, quality care.
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
Acknowledgements
We would like to thank the Last Mile Health Liberia programs and monitoring, evaluation and learning teams for their commitment to supporting the Liberia MOH and Grand Bassa County Health Team in implementing the program and in conduct of the household survey, and our External Advisory Committee for their inputs into study design and interpretation. We also would like to thank the women who responded to the questions, and the CHAs for their remarkable commitment to delivery of quality care to the members of their community.
Footnotes
Funding: This study was supported by the United States Agency for International Development’s Development Innovation Ventures, Johnson & Johnson, and the Margaret A. Cargill Foundation. JDK receives salary support from Last Mile Health. The program being evaluated was supported by the Children’s Investment Fund Foundation; Co-Impact; CRI Foundation; Crown Family Philanthropies; David Weekley Family Foundation; Deerfield Foundation; The ELMA Foundation; Gavi, the Vaccine Alliance; IZUMI Foundation; Jasmine Social Investments; Johnson & Johnson; King Philanthropies; Lucille Foundation; Mulago Foundation; Peery Foundation; Swedish Postcode Foundation; Pfizer Inc.; Planet Wheeler Foundation; Ripple Foundation; Skoll Foundation; Takeda Pharmaceutical Company Limited; UBS Optimus Foundation; USAID Development Innovation Ventures; Valhalla Foundation; and Virgin Unite. Funders played no role in the design, conduct, or reporting of the study.
References
