Abstract
Introduction Most substandard and falsified medicines are at best not optimally effective, and at worst fatal. While the World Health Organisation and others warn they are a major threat to public health in low and middle income countries, little is known about their true prevalence. Authors of meta-analyses universally warn that survey data are not generalisable, because of unrepresentative study designs and variations in medicines included; tests performed; reference standards and pharmacopeia used; and definitions used when translating multiple quality parameters into a single pass/fail measure.
We hypothesised that weighting for sales volume of different products and brands would increase accuracy of estimates of medicine quality.
Methods and Findings We collected samples of allopurinol, amlodipine, cefixime and dexamethasone, as well as amoxicillin in 2 formulations in seven districts across Indonesia, the world’s fourth most populous country. Outlets, including retail pharmacies, over the counter medicine shops, public and private hospitals, primary health centres, doctors and nurses were randomised. We also sampled from the internet. Retail samples were collected by mystery shoppers, other samples overtly.
We tested 1274 samples for identity and assay, and all relevant samples for dissolution and uniformity of content, using USP reference standards and monographs. Samples that failed any laboratory test were considered out of specification. We calculated prevalence per product and brand, and weighted the results by the sales volume of each product, using sales data from IQVIA and the national public procurement system.
The weighted prevalence of out-of-specification products was 4.9%, 40.3% lower than the raw estimate (8.2%). Antibiotics were more likely to be substandard (weighted prevalence 6.8 vs 3.1; raw prevalence 13.6 vs 4.9, p<0.000). There was no relationship between quality and any of the following: price; branded status; public procurement status; outlet type. Our estimate compared with the Indonesian medicine regulator’s estimate of 4.0%, calculated based on unweighted analysis of 13,539 samples of a wider variety of medicines, collected overtly nation-wide.
Conclusions Where data are available, weighting survey results by sales volume is a cost-effective way of improving the accuracy of estimates of out-of-specification medicines measured in field surveys.
Competing Interest Statement
Yusi Anggriani is a member of the Indonesian Ministry of Health's advisory committee on medicine pricing, and a member of the World Health Organization Technical Advisory Group on Pricing Policies for Medicines. Elizabeth Pisani has worked as a consultant on research commissioned by the WHO Incidents and Substandard/Falsified medical products team. All other authors report no conflict of interest.
Clinical Protocols
https://doi.org/10.7910/DVN/RKYICP
Funding Statement
The study was funded by UK taxpayers through the UK Department of Health and Social Care and the National Institute for Health Research, under NIHR Global Health Policy and Systems Research Commissioned Awards, grant number NIHR131145. Thank you. Thanks also to United States Pharmacopeia, who provided reference standards at discounted prices.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
Additional members of the STARmeds Study Group are listed, by affiliation, at the end of the manuscript
Data availability statement
Additional data are available in three locations, all within the STARmeds repository. Supplementary data for this specific paper (including the product volume data file used for adjusted estimates, the analysis code in Stata format for this paper, the supplementary methods description, supplementary figures and author contributions) are at: https://doi.org/10.7910/DVN/QRKDWG
Data and documentation related to STARmeds fieldwork more generally are in the study archive. This archive is easiest to use in Tree view. It contains the sample level data produced by the STARmeds field study, including raw laboratory data, in csv format. Also included are laboratory protocols and a more detailed description of methods. The archive can be accessed at: https://doi.org/10.7910/DVN/RKYICP.
Finally, we provide a free Toolkit to help researchers and regulators design and implement medicine quality field surveys using mystery shoppers. The toolkit contains downloadable and adaptable versions of data collection software, field control forms, field worker contracts and other potentially useful documentation. The Toolkit can be downloaded from: https://doi.org/10.7910/DVN/OBIDHJ
