Improving the quality of in-patient neonatal routine data as a pre-requisite 2 for monitoring and improving quality of care at scale: A multi-site 3 retrospective cohort study in Kenyan hospitals

^Membership of the Clinical Information Network is provided in the Acknowledgments. Abstract 16 Objectives : The objectives of this study were to (1) determine if membership of a clinical information 17 network (CIN) was associated with an improvement in the quality of documentation of in-patient 18 neonatal care provided over time, and (2) characterise accuracy of prescribing for basic treatments 19 provided to neonatal in-patients if data are adequate. 20 Design and Settings: This was a retrospective cohort study involving all children aged ≤28 days 21 admitted to New-Born Units (NBUs) between January 2018 and December 2021 in 20 government 22 hospitals with an interquartile range of annual NBU inpatient admissions between 550 and 1640 in 23 Kenya. These hospitals participated in routine audit and feedback processes on quality of 24 documentation and care over the study period. | Outcomes: The study’s outcomes were the number of patients as a proportion of all eligible patients 26 with (1) complete domain-specific documentation scores, and (2) accurate domain-specific treatment 27 prescription scores at admission. Findings : 80060 NBU admissions were eligible for inclusion. Upon joining the CIN, documentation 29 scores in the monitoring (vital signs) , other physical examination and bedside testing , discharge 30 information , and maternal history domains demonstrated a statistically significant month-to-month 31 relative improvement in number of patients with complete documentation of 7.6%, 2.9%, 2.4%, and 32 2.0% respectively. There was also statistically significant month-to-month improvement in prescribing 33 accuracy after joining the CIN of 2.8% and 1.4% for feeds and fluids but not for Antibiotic prescriptions. 34 Findings suggest that much of the variation observed is due to hospital-level factors. Conclusions : It is possible to introduce tools that capture important clinical data at least 80% of the time in routine African hospital settings but analyses of such data will need to account for missingness using appropriate statistical techniques. These data allow trends in performance to be explored and could support better impact evaluation, performance benchmarking, exploration of links between health system inputs and outcomes and scrutiny of variation in quality and outcomes of hospital care.


25
Outcomes: The study's outcomes were the number of patients as a proportion of all eligible patients 26 with (1) complete domain-specific documentation scores, and (2) accurate domain-specific treatment 27 prescription scores at admission. 2.0% respectively. There was also statistically significant month-to-month improvement in prescribing 33 accuracy after joining the CIN of 2.8% and 1.4% for feeds and fluids but not for Antibiotic prescriptions.

34
Findings suggest that much of the variation observed is due to hospital-level factors.

35
Conclusions: It is possible to introduce tools that capture important clinical data at least 80% of the 36 time in routine African hospital settings but analyses of such data will need to account for missingness 37 using appropriate statistical techniques. These data allow trends in performance to be explored and 38 could support better impact evaluation, performance benchmarking, exploration of links between LMICs is poor [6][7][8] while poorly functioning information systems mean limited data of questionable 53 quality on the delivery of such interventions in routine hospital settings in LMIC is available [9,10].

54
Availability of high-quality timely, accessible, and easy to use data from routine clinical settings could 55 improve monitoring of intervention adoption and quality of hospital care at scale, and ultimately might 56 help improve clinical outcomes [9][10][11][12]. An integrated approach providing a mechanism to promote 57 continued improvement of clinical information, implementation of effective practices and 58 technologies, and locally relevant research can comprise a 'learning health system', which are posited 59 to be influential in producing the positive change required [13][14][15][16].

60
The objectives of this study were to determine: (1) if good quality data can be generated from 119 developed as part of the Emergency Treatment and Triage plus admission (ETAT+) approach which 120 includes skill training in essential inpatient newborn care [24]. In earlier work they were associated 121 with improved documentation of key patient characteristics during admission [18]. NAR are not 122 provided to hospitals in CIN-N and so their adoption is at the discretion of hospital teams and 123 supported by hospitals' own resources, although CIN-N hospitals are encouraged to use them.

124
Each hospital has a clerk who extracts data from the NAR forms into a Research Electronic Data 125 Capture (REDCap) database [25]. Two sets of data are captured: minimum and full datasets. The 126 minimal dataset -which is unsuitable for this study's analyses -is collected for (1) admissions during 127 major holidays when the data clerk is on leave, and (2)    . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.

230
The proportion of records in which key items are not recorded is illustrated in Table 3.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.

235
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. ; https://doi.org/10.1101/2022.05.31.22275848 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.  Figure 2).

245
For this reason, we do not further examine hospital specific trends for these domains.

246
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. ; https://doi.org/10.1101/2022.05.31.22275848 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.

251
For other domains, performance started lower with a suggestion from all hospitals' data of 252 improvement over time, but also considerable between-hospital variability e.g., maternal history, with static performance over time (e.g., H12) and some with rather erratic performance including . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.  (Table 5).

270
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

275
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. ; https://doi.org/10.1101/2022.05.31.22275848 doi: medRxiv preprint  Figure 3). Hospitals with higher . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

281
baseline performance tended to demonstrate slower rates of improvement than hospitals with lower 282 baseline performance as illustrated in Figure 3 ( Table 5, H13 versus H17 in Figure 3).

283
Objective 2 Findings: Accuracy of essential neonatal intervention prescriptions over time 284 Given the good quality of prescribing data from CIN-N and the reasonable assumptions about the 285 meaning of missing prescribing data, treatment prescribing accuracy was evaluated for the common 286 antibiotics, feeds, and fluids in NBUs. Domain specific treatment accuracy scores revealed an 287 increasing proportion of patients with accurate fluids and feeds prescriptions from approximately 40% 288 to 60%, and 15% to 40% respectively, although feeds prescribing accuracy then regresses to 25% 289 ( Figure 4).

290
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.

294
Antibiotic prescription shows a modest improvement in accuracy from around 65% to 80% within the 295 first 12 months, after which it then fluctuated around 80% over time across all CIN-N admissions.

296
Treatment coverage levels for KMC demonstrated an increase over time from 20% to 40% in neonates 297 with birth weight <2kg (Figure 4). There was a small increase in CPAP coverage levels over time in 298 neonates with a clinical diagnosis of respiratory distress syndrome (RDS) from 4% to 10%. Repeated . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

300
There is evidence of moderate to high hospital variability in both treatment accuracy and coverage 301 scores in CIN-N hospitals (Figure 4).

302
While antibiotic treatment accuracy seems to have a ceiling effect of 80% in pooled hospital data, . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.  Table 2). Where the equidispersion 329 assumption was violated, negative binomial models were used and informed any inference drawn.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022.

332
This study aimed at determining if quality routine clinical data might be generated from CIN-N 333 hospitals and if the quality of data improved over time. As the data quality was reasonable, they were 334 then used to determine whether essential treatments or interventions are being correctly prescribed 335 to newborns and to track intervention adoption. From the time hospitals joined the CIN-N, around 336 80% of newborns had complete documentation in 5/8 documentation domains (Table 4). This 337 relatively good performance at baseline may be a consequence of participation in the paediatric CIN 338 by most of these hospitals prior to formal extension of CIN to NBUs (i.e., CIN-N) with many paediatric 339 practitioners previously exposed to use of the NAR, ETAT+ training and national neonatal guidelines  (Table 6). There is a modest statistically significant 2.8% and 345 1.4% month-to-month relative increase in accurate feeds and fluids prescription after joining the CIN-346 N resulting in an end line performance of around 40% and 60% respectively. Antibiotic prescribing 347 showed no statistically significant month-to-month change. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. demonstrate that data can be collected using a common data platform as part of a learning health 372 system approach from a network of hospitals' NBUs; we further show that these data can be useful 373 for identifying potential gaps in care (e.g., treatment accuracy) with an aim of improving the quality 374 of care provided in facilities and tracking outcomes at scale [13-16, 18, 40]. To our knowledge this is 375 the largest reported long-term neonatal learning health system platform in SSA, serving as an 376 exemplar actionable health information system in line with WHO standards [13,15,16,41].

377
Findings from scoping reviews suggest that having better data can help improve quality of care if 378 coupled with development of local leadership, training, and use of local improvement strategies such 379 as mortality audits or quality improvement cycles; This can help reduce inpatient neonatal mortality 380 in low-income country hospitals [42][43][44][45]. However, the complex intervention strategies required to . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. cases, this may reflect a "ceiling" effect (e.g., Fluids prescribing accuracy). It is evident, however, that . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 407 some hospitals can attain higher accuracy levels consistently (Figures 3 and 5), suggesting 408 improvements in other sites would be possible. Similarly, trends in accuracy in the prescription of 409 feeds and fluids are quiet erratic and vary within and between hospitals ( Figure 6). . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 2, 2022. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 2, 2022.

690
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 2, 2022. ; https://doi.org/10. 1101