Abstract
Objective To evaluate the implementation of an early warning system in obstetric patients (MEWC) during the first two hours after delivery in a single tertiary-care hospital.
Methods The MEWC system implementation was carried out from 15th March to 15th September 2018, over 1166 patients. The parameters collected were systolic and diastolic blood pressure, heart rate, oxygen saturation, diuresis, uterine involution, and bleeding. If a parameter was not within defined limits, an obstetrician first examined the patient, determining the need to call the anesthesiologist. We carried out a sensitivity-specificity study of the trigger and multivariate analysis of the factors involved in developing potentially fatal disorders (PFD), reintervention, critical care admission, and stay.
Results The protocol was triggered in 75 patients (6.43%). The leading cause of alarm activation was the altered systolic blood pressure (32 [42.7%] patients), and eleven developed PFD. Twenty-eight patients were false-negatives. Sensitivity and specificity of MEWC protocol were 0.28 (0.15, 0.45) and 0.94 (0.93, 0.96), respectively. Multivariate analysis showed a relationship between alarm activation and PFD.
Conclusion Our MEWC protocol presented low sensitivity and high specificity, having a significant number of false-negative patients.
Introduction
Lowering morbidity and mortality in the obstetric patient continues to be a quality-of-care criterion for healthcare centers. The development over the few decades shows a remarkable improvement in healthcare indicators in this population group [1]. Although mortality rates are very low for developed countries, the impact is high, both for the social repercussion that it involves and the years of life lost.
One of the approaches to reduce maternal morbidity and mortality has been using tools that would enable the rapid identification of patients who would benefit most from an aggressive intervention or a higher level of care. In December 2007, a review on maternal mortality concluded that 40-50% of the UK’s mortality rates are preventable. However, the early warning signs were often not recognized [2]. Since then, most UK hospitals have adopted Maternity Early Obstetric Warning Criteria (MEWS) protocols, with these being an auditable quality criterion for the centres [3, 4].
Unfortunately, the implementation of these protocols is not universal, and its use is not common practice outside English-speaking countries. Furthermore, although the adoption of these protocols has shown an improvement in the quality of assistance to pregnant women and the detection of adverse effects, it has not shown any impact on reducing maternal mortality [5].
This study evaluates the usefulness and feasibility of implementing a modified obstetric early warning system in our center and studying its possible relationship with the detection and identification of adverse effects.
Methods
Study design and setting
This study analyses the implementation of a MEWC protocol in a tertiary-care hospital (Hospital Fundación Jiménez Díaz, Madrid, Spain). The hospital has all the medical and surgical specialties but does not have a specific building for gynecology and obstetrics. We designed a single-arm prospective cohort study developed during the first six months after implementing the protocol (first day of implementation: 15th March 2018). Our center’s ethics committee approved our research before the start of patient recruitment (Date of Approval: 12th March 2018, Code: FJD-MEOWS-17-01. S1 File, original in Spanish). This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies [6].
Sample size calculation
For calculating the sample size [7], we considered that our model should have a minimum sensitivity and specificity of 90%. We knew that the incidence of maternal morbidity in our environment was 5% [8]. Thus, the required sample size with a precision of 0.1, a type I error of 0.05, and a dropout rate of 10% were 692 patients. Since the number of births in the hospital was around one thousand between 1st September 2017 and 28th February 2018, we decided to include all patients who gave birth between 15th March and 15th September 2018, except those who refused to provide their data to the study after informed consent.
Procedures
We designed our center’s MEWC protocol based on the one described by Mhyre et al. [9]. We monitored systolic and diastolic blood pressure, heart rate, oxygen saturation, and diuresis at 10, 30, 60, 90, and 120 minutes. By consensus between anesthesia and obstetrics departments, we included uterine involution measurement (by manual exploration), diuresis, and any bleeding greater than 500 ml in the protocol. We defined bleeding or lack of uterine involution as obstetric causes. S2 File shows the data collection form.
Two months before the start of implementation, we provided training sessions on the usefulness of the MEWC protocols and the process of implementing them in our hospital to professionals from both services, including midwives. After implementing the protocol, nurses and midwives monitored patients for 2 hours after giving birth or undergoing a cesarean delivery by filling in the data form, either in the delivery room or in the PACU. They also collected data on previous parity, preeclampsia, and multiple births as possible risk factors. For each variable in the protocol, specific values obliged the midwives to call the obstetricians, who had to assess the patient within 15 minutes. If the obstetricians did not resolve the alarm, they would make a second call to the anesthesiologists. Independent of the form’s data collection, the anesthesiologists followed all patients who had given birth until discharge.
Outcomes
Following the recommendations made by the WHO [10], the outcome of our study was the rate of potential fatal disorder (PFD) during the stay, defined as one of the following criteria: access to a critical care unit (CCU), surgery within two hours of delivery, or length of stay of more than seven days depending on the activation of the alert [11]. As secondary outcomes, we measured the alert activation relation on each of the PFD criteria separately.
Statistical analysis
We used R v4.0.2 (The R Foundation for Statistical Computing, Vienna, Austria) and RStudio 1.2.5033 (RStudio PBC, Boston, USA) to perform statistical analysis. We analyzed outcomes depending on the alarm activation. We described discrete and continuous variables as number and percentage and median (interquartile range [IQR]), and their differences analyzed using the Pearson test or the Wilcoxon rank-sum tests. We calculated the sensitivity, specificity, positive and negative predictive value (PPV and NPV) of alert activation for PFD and each criterion separately, with a 95% confidence interval. We performed a multivariate logistic analysis to study the association of the outcomes with alarm activation, preeclampsia, parity, type of delivery, and multiple births. We presented the results in forest plots as an odds ratio with a 95% confidence interval. We used Cox regression for multivariate analysis of length of stay, showing the results in forest plot as a hazard ratio with a 95% confidence interval. To avoid errors by multiple comparisons, we calculated the respective q-value for each p-value to maintain a false discovery rate below 5% [12]. We considered comparisons in which the p-value and q-value were below .05 as being statistically significant. We provide the original study databases, the step-by-step statistical analysis, and the document in R-Markdown format in S3 File, S4 File, and S5 File.
Results
During the study period, there were 1169 deliveries at our hospital. Only three patients declined to participate in the study. Since there were no losses to follow-up, we included 1166 patients. Fig 1 shows the STROBE flow chart.
The patients had a median age of 34 (31-37) years, without previous deliveries, and had a rate of cesarean delivery and instrumental delivery of 23.2% and 15.1%, respectively. The protocol alarm was activated in 75 patients (6.43%). The cohort of subjects with an activated alarm had a higher rate of cesarean delivery (37.3% vs 22.3%, p = 0.005), preeclampsia (17.3% vs 4.8%, p ¡ 0.001), and multiple birth (6.7% vs 1.5%). Also, the PFD rate was higher (14.7% vs 2.6%, p ¡ 0.001), as was the CCU admission rate (12% vs 0.8%, p ¡ 0.001) and length of stay (median: 2.9 [2.3-3.6 vs 2.5 [2.1-3.1] days, p = 0.005). Table 1 shows the demographic data and the outcome results according to MEWC protocol activation.
The leading cause of alarm activation was altered systolic blood pressure (32 [42.7%] patients), followed by obstetric causes (24 [32%] patients). The median time of alarm activation, obstetric assessment, call to the anesthesiologist, and anesthetic assessment was 10, 11, 15, and 20 minutes from the start of postpartum monitoring (Table 2).
Our MEWC had a sensitivity of 0.28 (0.15, 0.45), a specificity of 0.94 (0.93, 0.96), a positive predictive value of 0.15 (0.08-0.25) and a negative predictive value of 0.97 (0.96, 0.98) for PFD detection (Table 3). Furthermore, the protocol showed a specificity of 0.94 and a negative predictive value of 0.99 for each of the PFD criteria separately (intervention, CCU admission, and length of stay longer than seven days).
With regard to multivariate regression analysis, we only found a relationship with PFD with 2 factors, preeclampsia [Odds Ratio = 7.81 (3.50, 16.7), p < 0.001] and MEWC alarm activation [Odds Ratio = 3.97 (1.63, 8.95), p = 0.001] (Fig 2). The only factor related to the intervention was the MEWC alarm activation (Odds Ratio = 4.73 [1.04, 15.7], p = 0.02, Fig 3). The main factor related to critical care admission was preeclampsia (Odds Ratio = 23.2 [7.75-75.3], p < 0.001), followed by MEWC alarm activation (Odds Ratio = 9.73 [2.98-31.5], p < 0.001) and cesarean delivery (Odds Ratio = 4.40 [1.22-18], p = 0.03) (Fig 4). We found no relationship between length of stay and MEWC alarm activation in the Cox regression (Fig 5).
No significant p-value was rejected after calculating its q-value within the multiple comparability study (S4 File).
Discussion
The first thing that draws attention to implementing the MEWC in our hospital is the data on sensitivity, specificity, positive, and negative predictive value of alarm activation. Our protocol’s sensitivity is very low (0.28), while specificity and negative predictive value are very high (0.94 and 0.99, respectively). For practical purposes, these results mean that we can be confident that patients without activation of the MEWC protocol alarm are doubtful to suffer a PFD. However, the cost was not to detect many patients who developed PFD (28 out of 39, 71.1% of total cases). Although we do not know the causes of such low sensitivity levels, we suspect that it is probably related to the patient’s monitoring period or the different parameters that form the protocol.
Our study monitored the patients for only 2 hours after delivery. However, other studies with better sensitivity results had a much more excellent follow-up. Singh et al. [13] published the first study to validate MEWC, carrying out a prospective observational study in 676 obstetric patients to detect morbidity between the 20th week of gestation and six months postpartum. 30% of the patients had an altered trigger. They defined a MEWC sensitivity of 0.98 and a specificity of 79%. The introduction of such an extended period of obstetric monitoring is probably impossible to implement in our center. Given our hospital’s structure, our patients go from the delivery room to conventional hospitalization rooms, where the monitoring is not comparable to that of a post-anesthesia recovery unit. Still, it makes us suspect that if we extend the monitoring period from admission to 24 hours after birth, we should reduce the number of false negatives.
On the other hand, the protocol modification by adding obstetric factors has also been able to modify the sensitivity and specificity of the protocol. Initially, we can think that the more complex a protocol is, the more difficult it is to generate false negatives, and the more secure its implementation is. However, this is not as true as it seems. Mhyre et al. [9] described that the simpler the warning criteria, the shorter the time for recognition, diagnosis, and treatment of women who are more at risk of developing obstetric complications. The more uncomplicated measures are more reliable, less vulnerable to human calculation errors, and have higher reproducibility. Mhyre et al. [9] insist on creating multidisciplinary teams that define their warning criteria for each center and review the evidence to date using these criteria9. In 2015 Edwards et al. [14] compared the predictive value of six different MEWS to identify severe sepsis in women with chorioamnionitis and informed of a sensitivity range of 40%-100% with a degree of 4%-97% specificity. The authors concluded that the MEWC tools with simpler designs tended to be more sensitive, while more complex ones tended to be more specific. By adding the obstetric items, we probably complicated the protocol and caused an increase in specificity and a decrease in sensitivity.
Likewise, a complex MEWC protocol that causes a high number of false positives is a severe limitation in its application since it can lead to professional fatigue due to the high number of false alarms15. Of course, no MEWC protocol is ideal, and we fully agree with Friedman et al. [15]. They argued that since each hospital has a unique environment, each hospital should develop its protocol and improve it, depending on its particular outcomes. We are currently in that phase, the improvement phase. Our study results encourage us to persevere in perfecting the protocol, either by increasing the time of monitoring the patient or by altering the variables to be monitored. However, further studies are necessary to validate and generalize MEWC principles.
Conclusion
Our MEWC protocol presented low sensitivity and high specificity with a high negative predictive value, having many false-negative patients. Our study showed an association between alarm activation and PFD, reintervention rate, and critical care admission rate, with preeclampsia being the most related factor in all of them.
Data Availability
All data has been self collected during fieldwork and is available for review if needed.
Supporting information
S1 File. Ethics committe approval document. Original in Spanish. Date of Approval: 12th March 2018, Code: FJD-MEOWS-17-01.
S2 File. Data collection form
S3 File. Original study databases.
S4 File. Statistical analysis. Html format.
S5 File. Statistical analysis raw R code. R-Markdown format.