Comparative functional survival and equivalent annual cost of three long lasting insecticidal net (LLIN) products in Tanzania

: Almost 1.2 billion long-lasting insecticidal nets (LLINs) have been procured for malaria control. Institutional buyers often assume that World Health Organization (WHO) prequalified LLINs are functionally identical with a three-year lifespan. We measured the lifespans of three LLIN products, and calculated their cost-per-year of functional life, through a randomised double-blinded prospective evaluation among 3,420 study households in Tanzania using WHO-recommended methods. Primary outcome was LLIN functional survival (LLINs present in serviceable condition). Secondary outcomes were 1) bioefficacy and chemical content (residual insecticidal activity) and 2) protective efficacy for volunteers sleeping under LLINs (bite reduction and mosquitoes killed). LLIN median functional survival was significantly different: 2 · 0 years for Olyset, 2 · 5 years for PermaNet and 2 · 6 years for NetProtect. Functional survival was affected by accumulation of holes resulting in users discarding nets. Protective efficacy also significantly differed between products as they aged. The longer-lived nets were 20% cheaper than the shorter-lived product. standard cone bioassays out. bioefficacy tests


Results:
A total of 3,393 households were randomised to which 10,571 nets were distributed (3,520 Olyset (33%), 3,513 PermaNet 2.0 (33%) and 3,538 NetProtect (33%)). The three study arms were similar in number of participants, number of nets allocated, household characteristics, house design and socioeconomic characteristics (Table S1). Households lost to follow up was 20% over the three years of the trial.

Functional Survival
There were significant differences between functional survival of the three products (defined as presence of serviceable nets) ( Table 1). Estimated median functional survival was 2·0 years for Olyset, 2·5 years for PermaNet and 2·6 years for NetProtect. There was no significant difference in net use by net product (Table S2).

Economic Analysis
Simulation results show that the expected Equivalent Annual Cost (EAC) in $ USD of the three LLINs in the study varied between $1·2 (1.1-1.4) for PermaNet and NetProtect and $1·5 (1.3-1.7) for Olyset, assuming that each net was priced identically at $3·0 ( Table 1). The longer-lived nets were approximately 20% lower in EAC as compared to the shorter-lived Olyset product.

Components of functional survival
Attrition There were significant differences in attrition between net products. Olyset nets were lost at a faster rate than PermaNet 2.0 and NetProtect ( Table 2, Table S3). After three years, 55% of Olyset nets were no longer present in households compared to 42% of PermaNet 2.0 and 46% of NetProtect (p<0·001; Table 2). Of the 10,571 nets distributed, 4,964 (46%) were lost to follow up over the whole study period (Table S4).  Figure 1: Physical condition of LLINs according to WHO categorisation using proportional Hole Index4 by the three net products and time points. Green shows nets in percentage of nets in good condition (pHI < 65), orange shows % nets in a damaged condition (pHI: 65-364) and red shows % of nets defined as "too torn" (pHI > 364).
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 1, 2019. ; https://doi.org/10.1101/19002212 doi: medRxiv preprint

Physical integrity
The condition of nets that remained in households deteriorated over the course of the study. At each time point, Olyset had the largest proportion and NetProtect had the smallest proportion of 'too torn' nets ( Figure 1). The median hole surface area in Olyset increased from 38 cm 2 at 10 months to 459 cm 2 after 36 months, compared to 6 cm 2 to 295 cm 2 for PermaNet 2.0 and 8 cm 2 and 152 cm 2 for NetProtect (Table S5). Questionnaire data showed that in year 3, 70% of nets no longer in use had been discarded when they were perceived as too damaged to be useful. Others were given away (17%), stolen (3%) or repurposed (3%).

Bioefficacy
At baseline, all products met optimal WHO bioefficacy criteria. After field use, there were significant differences between the bioefficacy of the net products measured using standard WHO cone and tunnel tests over time (Table  3). At 10 months, 100% of NetProtect and PermaNet 2.0 nets met WHO optimal bioefficacy criteria, compared to 73% of Olyset nets (p<0·001). Nets decreased in bioefficacy through time but even after three years, 96% of NetProtect, 85% of PermaNet 2.0 and 75% of Olyset met WHO criteria for bioefficacy (p=0·017; Table 3). When whole nets were tested using IACT, 88% of Olyset, 96% of PermaNet 2.0 and 92% of NetProtect passed WHO optimal criteria of 80% mortality and 90% blood feeding inhibition after 3 years. There were differences between products in 24-hour mortality. Olyset showed lower mortality (p<0·001), but all three products showed similar levels of feeding inhibition ( Figure 2, Table S6). Mosquito mortality was higher for nets defined as "too torn" (OR = 0·65 (0·49, 0·88), p=0·005), but the differences between the net products remained significant after adjusting for physical condition. Similarly, protection from mosquito bites (feeding inhibition) was considerably lower in nets that were "too torn" (OR = 0·12 (0·08, 0·18), p<0·001), but the differences between the net products remained non-significant after adjusting for physical condition.  (Table S7). Deltamethrin content lower than the target dose in NetProtect was explained by a high R-alpha isomer content (0·35 g/kg on average, 26% of the deltamethrin content), a non-relevant impurity of deltamethrin, which may be formed during the manufacturing process.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 1, 2019. ; https://doi.org/10.1101/19002212 doi: medRxiv preprint Figure 2: Ifakara-Ambient Chamber Test (IACT) results on mosquito mortality (top panel) and blood feeding inhibition (bottom panel) by net product (red = Olyset; orange = PermaNet; blue = NetProtect) and time point. Optimal WHO criteria (80% mortality; 90% blood feeding inhibition) are indicated by the dashed line.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 1, 2019. ; https://doi.org/10.1101/19002212 doi: medRxiv preprint

Discussion:
This large and well-powered study clearly disproves the assumption that all pyrethroid treated LLIN products have similar lifespans. Our data also confirms that the median functional life of the LLINs in our study was closer to two years than three years in Tanzania and for the products tested, as previously reported by a systematic review of LLIN retention data in 39 sub-Saharan African countries. 6 The WHO's Guidelines for procuring public health pesticides 14 recommends that the procurement should consider "operational cost" rather than unit price, and an appropriate measure to compare value for money of LLINs would be "cost per median year of net life under local conditions". We measured the relative durability of nets using functional survival estimates, in terms of the equivalent annual cost (EAC). The cost analysis showed approximately 20% lower EAC when a longer-lasting LLIN (PermaNet 2.0 or NetProtect) was chosen over Olyset, assuming prices for the products were identical. The relative increase in price that is acceptable is also much smaller when the lifetime of the standard reference product increases. Thus, the extension of the life of a product is much more valuable if the comparator product is relatively short-lived, as was seen in this study.
LLIN functional life also has important implications for the selection of new products for resistance management that have a higher unit cost. New pyrethroid plus piperonyl butoxide (PBO) nets may not be as durable as standard pyrethroid nets because PBO is lost rapidly from nets during washing, which reduces their efficacy. 15 However, in Tanzania, PBO nets continued to have superior public health benefit, two years after distribution. 7 If the median functional survival of pyrethroid LLINs is two years, then PBO nets may remain cost-competitive.
WHO requires LLIN manufacturers to provide data from three longitudinal field evaluations in different ecologies e.g. West Africa, East Africa and Asia that is used to evaluate LLINs for Prequalification (PQ) listing. While it is recognized that durability is context specific, using the WHO methodology outlined 16,17 , it is possible to routinely generate median functional survival estimates and EAC for at least three locations during PQ evaluation, albeit with a more limited sample size than the present study. The EAC may be a useful metric to compare between products, rather than assessing products based simply on a minimum threshold as is current practice. The role of durability data in LLIN procurement has been side-lined and consideration of its importance in vector control by WHO may re-awaken the LLIN market to reward more durable products that will in turn create incentives for investments in technological advances, research and development by ITN manufacturers. 5,17 Attrition and physical integrity, the two factors that define functional survival of LLINs, 2 differed significantly between the three net products. Olyset demonstrated more rapid accumulation of damage and more rapid attrition. In the current study and in previous work we demonstrated that most LLINs were discarded because they were perceived as too damaged to offer protection against mosquito bites or malaria. 18 Further consideration should be given to developing simple tools to allow countries to assess attrition and fabric integrity during routine surveys to inform planning since it is clear that these two outcomes are both most important in predicting LLIN functional life, are highly variable between contexts and are simpler to collect than bioefficacy or chemical content data.
Of those nets still present after three years, 25-40% were categorized as no longer physically serviceable, depending on the brand. However, even after three years, nets remained highly efficacious when tested by bioassays against insecticide-susceptible malaria vectors. Damage actually increased the mortality of mosquitoes that had entered nets through holes and become trapped, as also observed in other studies. 19 Indeed, torn LLINs continue to provide both individual and community protection from malaria. 20,21 Our IACT experiments, demonstrated that the three brands were all highly protective, although Olyset killed significantly fewer mosquitoes than PermaNet 2.0 and NetProtect. It is of note that most damage to the nets is on the bottom section from where they are tucked under a mat or mattress. The act of tucking makes these holes inaccessible to mosquitoes even though the net appears as badly damaged to the user.
However, it is a limitation of the presented study that only susceptible mosquitoes were used for bioefficacy testing as pyrethroid resistance is widespread and increases feeding success and reduces mortality of mosquitoes. 19 Another limitation is the fact that the study was only conducted in Tanzania and LLIN durability does vary by location. Furthermore, this study was conducted on only three brands of LLINs, all of which are treated with pyrethroids. As new LLINs products come on the market especially those with new insecticides e.g. PBO nets it will be imperative to monitor their comparative durability to ensure that the most cost-effective products are procured for malaria control.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 1, 2019. ; https://doi.org /10.1101/19002212 doi: medRxiv preprint It was demonstrated that nets that are still in households, despite holes, are still protective against mosquito bites and continue to kill mosquitoes, providing personal and community protection. However, if nets are discarded, or no longer used because they are perceived as too damaged, then they have no public health benefit at all. While it is possible to encourage users to retain their damaged but still insecticidal nets through behavioural change communication (BCC) a more effective strategy will be to distribute more physically durable LLINs. 22 LLINs are the largest single item on the global malaria control budget. Most are distributed through mass campaigns. 3 More durable LLINs would reduce the required frequency of campaigns and thus the operational costs of distribution per person-year of coverage. It is technically feasible to manufacture more durable LLINs. However, this will happen only if buyers consider cost-effectiveness for coverage 14 and demonstrate a preference for longer-lasting and better value-for-money products, rather than considering only the unit price.

Methods:
The study has been described in detail previously. 23 It took place in 8 districts in Tanzania, selected to be representative of national environmental, ecological and epidemiological settings (Figure 3). Within each district, 10 villages were randomly selected, and within each village 45 households were recruited to participate in the study. All households were randomised to receive one of three LLIN brands on a 1:1:1 ratio, stratified by village. The three brands were Olyset ® (permethrin incorporated in 150 denier polyethylene; Sumitomo Chemicals, Japan), PermaNet ® 2.0 (deltamethrin coated on 100 denier polyester; Vestergaard Frandsen, Switzerland), or NetProtect ® (deltamethrin incorporated into 110 denier polyethylene; BestNet, Denmark). Distribution of study nets took place between October and December 2013. All nets owned by the participating households were collected and replaced with enough nets to cover all sleeping spaces. Before distribution, a sample of ten nets per product was quality tested. Nets were the same size and colour, labelled by a five-digit serial number so that participants and investigators remained blinded to the LLIN product until data collection was complete. In total, 3,393 households were randomised (1,132 to Olyset, 1,127 to PermaNet 2.0 and 1,134 to NetProtect) to which 10,571 nets were distributed (Trial Characteristics Table S1).
Surveys were conducted among all consenting study households when the LLINs were distributed and at three follow up points: 10 months (August -October 2014), 22 months (August -October 2015) and 36 months (October -December 2016) (Study Flow Table S8). Serial numbers of nets, linked to household identifying codes in a master list, enabled follow-up of each net at each time point. At each follow up visit information on each LLIN was collected, including whether the net was present in the house, reasons why it was not present, and whether the net was in use. Physical integrity of LLINs was measured on a random sample of three nets per household by counting the number, location and size of holes. 16,17 Socio-economic variables and a household member roster were also recorded. Electronic data capture was used for all surveys.
In addition to the data collected as part of the household surveys, at each time point 48 LLINs from each brand were randomly sampled from the master list and returned to the laboratory in Bagamoyo, Tanzania for bioefficacy and chemical analysis using standard WHO methods 16,17 and additional Ifakara Ambient Chamber Test (IACT). 24 Table 4 describes the different components of LLIN durability, the test conducted to obtain the data, the outcome indicators for statistical analysis and the corresponding WHO threshold criteria. 4,16,17 The numbers of LLINs tested for each of the components of LLIN durability are listed in Table S8.
First, the protective efficacy of whole nets returned from the field was evaluated using IACT. 24 Each night, ten male volunteers slept underneath one of the nets (or an untreated control net to monitor the quality of the bioassay) between 21.00 -6.00hrs in a small chamber similar in size to a bedroom, within a screened compartment. At 21.00hrs, 30 laboratory-reared mosquitoes were released into the chamber. The next morning, all mosquitoes within the compartment were recaptured, and scored for 24-hour mortality and blood-feeding inhibition. Each LLIN was tested twice on two consecutive nights. Subsequently, net pieces (25x25cm 2 ) were cut following the WHO sampling pattern and standard WHO cone bioassays were carried out. 16 If nets did not meet WHO optimal bioefficacy criteria for cone tests (Table 4), WHO tunnel tests were conducted. 16   . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)  NetProtect: 1·8 g/kg ± 25% [1·35 -2·25 g/kg] ‡ Nets that were reported as given away, sold or stolen were not in the denominator (lost to follow up). † proportionate Hole Index (pHI): Number of holes of each size category measured in the field are weighted by the approximate surface area of the holes to provide a single measure of damage per net. * previously described as Ifakara Tunnel Test (ITT) 14 Statistical Analysis All statistical analyses were conducted using Stata Statistical Software: Release 13 (StataCorp LP, TX). Attrition and functional survival (Table 4) were calculated using Kaplan-Meier estimators. For both endpoints, nets reported as given away, sold or stolen were treated as lost to follow up. Hazard ratios for the difference in attrition and functional survival were calculated using discrete time survival analysis using a complementary log-log model. 25 Robust standard errors were used to account for the highest level of clustering (district). 26 Of nets that were present, net condition was defined, following WHO recommendations, as either being "good", "damaged" (combined to be "serviceable"), or "too torn/unserviceable" (Table 4). Negative binomial regression was used to compare hole surface area between net products. A Chi-squared test assessed the proportion of nets of each product passing the WHO bioefficacy criteria based on combined cone and tunnel tests, adjusted for control mortality. Logistic regression was used to analyse mortality and blood feeding inhibition from the IACT test; results were adjusted for chamber and experimental night and robust standard errors were used to take account of nets being tested multiple times.

Economic Analysis
The equivalent annual cost (EAC) of an LLIN was calculated according to the standard formula. 27 To assess the value of longer functional survival we used Equation 1 where b is the ratio of the lifespan of the more durable product to the lifespan of reference net n. The variable r is the discount rate. This relationship shows that for any change in net lifespan from n to bn the relative increase in price, a, which would yield an identical EAC for the two products. Other factors being equal a relative price increase less than a would favour the new, longer-lasting LLIN, while relative price increases greater than a would favour the standard reference net.

Equation 1 = #$(#&') )*+ #$(#&') )+
Simulation of EACs for products tested in the study was conducted using Monte Carlo methods, assuming a 3% discount rate, as is standard in health economic analysis. The baseline survival function for LLINs was estimated by regressing the survival proportions of Olyset nets derived from Kaplan-Meier analysis (Table S4) against time. The survival function was converted into a baseline hazard and net failure lifetimes were simulated for a cohort of 500 LLINs assuming a Weibull distribution of times to failure (in terms of functional survival). The results of the cohort were summarised by estimating the median lifetime and this process was repeated 10,000 times for each net type, yielding an estimate of the expected median lifetime and quantiles of its expected distribution. Results were converted into EACs with 95% quantiles. Distributional assumptions for the baseline hazard and the parameters of the Weibull distribution were fitted to the results presented in Table 1. The baseline hazard and proportional hazard were simulated with log normal distributions (Table S9).

Ethics
Ethical approval was granted from ethical review committees at London School of Hygiene & Tropical Medicine (6333/A443), Ifakara Health Institute (IHI/IRB/AMM/ No: 07-2014) and the Tanzanian National Institute for Medical Research (NIMR/HQ/R.8c/Vol. I/285). Community sensitization meetings were held prior to study inception and written informed consent was obtained from the head of the household, or another adult household member of participating households before each survey. Human volunteers for the IACT experiment were all IHI staff members with appropriate training who gave written informed consent.
Role of the funding source The funder of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)         . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 1, 2019. ; https://doi.org/10.1101/19002212 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 1, 2019. ; https://doi.org/10.1101/19002212 doi: medRxiv preprint