Abstract
Objective To evaluate the implementation of an early warning system in obstetric patients (MEWC) during the first two hours after delivery in a tertiary-care hospital.
Method The MEWC system implementation was carried out from 15th March to 15th September 2018, over 1166 patients. The parameters collected were systolic and diastolic blood pressure, heart rate, oxygen saturation, diuresis, uterine involution, and bleeding. A trigger was defined as any abnormal parameter that prompted the need for bedside examination by an obstetrician and an anesthesiologist. We carried out a sensitivity-specificity study of the trigger and multivariate analysis of the factors involved in developing potentially fatal disorders (PFD), reintervention, critical care admission and stay.
Results The protocol was triggered in 75 patients (6.43%). The leading cause of alarm activation was altered systolic blood pressure (32 [42.7%] patients), followed by obstetric causes (24 [32%] patients). Of these patients, eleven developed PFD. Twenty-eight patients were false-negatives. Sensitivity and specificity of MEWC protocol were 0.28 (0.15, 0.45) and 0.94 (0.93, 0.96), respectively. Multivariate analysis showed a relationship between alarm activation and PFD.
Conclusions: Our MEWC protocol presented low sensitivity and high specificity, having a significant number of false-negative patients.
Introduction
Lowering morbidity and mortality in the obstetric patient continues to be a quality-of-care criterion for healthcare centres. The development over the few decades shows a remarkable improvement in healthcare indicators in this population group1. Although mortality rates are very low for developed countries, the impact is high, both for the social repercussion that it involves and the years of life lost.
One of the approaches to reduce maternal morbidity and mortality has been using tools that would enable the rapid identification of patients who would benefit most from an aggressive intervention or a higher level of care. In December 2007, a review on maternal mortality concluded that 40-50% of the UK’s mortality rates are preventable. However, the early warning signs were often not recognized.2 Since then, most UK hospitals have adopted Maternity Early Obstetric Warning Criteria (MEWS) protocols, with these being an auditable quality criterion for the centres3,4.
Unfortunately, the implementation of these protocols is not universal, and its use is not common practice outside English-speaking countries. Furthermore, although the adoption of these protocols has shown an improvement in the quality of assistance to pregnant women and the detection of adverse effects, it has not shown any impact on reducing maternal mortality5.
This study aims to evaluate the usefulness and feasibility of implementing a modified obstetric early warning system in our centre and studying its possible relationship with the detection and identification of adverse effects on the mother.
Methods
This study analyses the implementation of a MEWC protocol on a tertiary-care hospital (Hospital Fundación Jiménez Díaz, Madrid, Spain). We designed a single-arm prospective cohort study developed during the first six months after implementing the protocol between 15th March and 15th September 2018. Our centre’s ethics committee approved our research before the start of patient recruitment (Date of Approval: 12th March 2018, Code: FJD-MEOWS-17-01. Supplementary File 1, original in Spanish). This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies6.
After receiving informed consent, we included every patient undergoing eutocic or cesarean delivery consecutively, except those who refused the study’s inclusion. Given our research’s nature, we were unable to make a formal calculation of the required sample size, as the sample would depend directly on the number of births during the six-month study period. However, knowing from previous studies that the incidence of maternal morbidity in our environment was 5%7 and that in previous semesters, the number of births in the hospital was around one thousand. We estimated that our study would have a precision of ±1.4%.
Procedures
We designed our centre’s MEWC protocol based on the one described by Mhyre et al.8 that included the monitoring of systolic and diastolic blood pressure, heart and respiratory rate, oxygen saturation and diuresis at 10, 30, 60, 90 and 120 minutes. By consensus between the departments of anesthesia and obstetrics, we removed respiratory rate monitoring from the protocol. We included the measurement of uterine involution, diuresis, and any bleeding greater than 500 ml. The data collection form can be seen in Supplementary File 2.
Two months before the start of implementation, we provided training sessions on the usefulness of the MEWC protocols and the process of implementing them in our hospital to professionals from both services, including midwives. After implementing the protocol, nurses and midwives monitored patients for 2 hours after giving birth or undergoing a cesarean delivery by filling in the data form, either in the delivery room or in the PACU. They also collected data on age, previous parity, maternal diabetes, preeclampsia and multiple births as possible risk factors. For each variable in the protocol, specific values obliged the midwives to call the obstetricians if they were exceeded, who had to assess the patient within 15 minutes. If the trigger were not resolved, the obstetrician would make a second call to the anesthesiologists. Independently of the form’s data collection, the anesthesiologists followed all patients who had given birth up until discharge.
Outcomes
The primary outcome of our study was the rate of potential fatal disorder (PFD) during the stay, defined as one of the following criteria: access to a critical care unit (CCU), surgery within two hours of delivery, or length of stay of more than seven days depending on the activation of the alert9. As secondary outcomes, we measured the alert activation relation on each of the PFD criteria separately.
Statistical analysis
We used R v3.6.3 (The R Foundation for Statistical Computing, Vienna, Austria), and RStudio 1.2.5033 (RStudio PBC, Boston, USA) to perform statistical analysis. We analyzed outcomes depending on whether the alarm was activated or not. We described discrete and continuous variables as number and percentage and median (interquartile range [IQR]), and their differences analyzed using the Pearson test or the Wilcoxon rank-sum tests. We calculated the sensitivity, specificity, positive and negative predictive value (PPV and NPV) of alert activation for PFD and each criterion separately, with a 95% confidence interval. We performed a multivariate logistic analysis to study the association of PFD, access to CCU, and reintervention with alarm activation, clinical and demographic data, presenting the results in forest plots as odds ratio with 95% confidence interval. We used Cox regression for multivariate analysis of length of stay, showing the results in forest plot as a hazard ratio with 95% confidence interval. To avoid errors by multiple comparisons, we calculated the respective q-value for each p-value to maintain a false discovery rate below 5%10. We considered comparisons in which p-value and q-value were below .05 as being statistically significant.
Results
We identified 1169 eligible patients, of which three declined to participate in the study. Since there were no losses to follow-up, we included 1166 patients in the study. Figure 1 shows the STROBE flow chart.
The patients had a median age of 34 (31-37) years, were mainly primiparous (median of previous births: 0 (0-1)) and had a rate of cesarean delivery and instrumental delivery of 23.2% and 15.1%, respectively. The protocol alarm was activated in 75 patients (6.43%). The leading cause of alarm activation was altered systolic blood pressure (32 [42.7%] patients), followed by obstetric causes (24 [32%] patients). The median time of alarm activation, obstetric assessment, call to the anesthesiologist, and anesthetic assessment was 10, 11, 15, and 20 minutes from the start of postpartum monitoring (Table 1).
Table 2 shows the demographic data and the outcome results according to MEWC protocol activation. The cohort of subjects with an activated alarm had a higher rate of cesarean delivery (37.3% vs 22.3%, p = 0.005), preeclampsia (17.3% vs 4.8%, p < 0.001), and multiple birth (6.7% vs 1.5%). Also, the PFD rate was higher (14.7% vs 2.6%, p < 0.001), as was the CCU admission rate (12% vs 0.8%, p < 0.001) and length of stay (median: 2.9 [2.3-3.6 vs 2.5 [2.1-3.1] days).
Table 3 shows the sensitivity and specificity of the protocol. Our adapted MEWC had a sensitivity of 0.28 (0.15, 0.45), a specificity of 0.94 (0.93, 0.96), a positive predictive value of 0.15 (0.08-0.25) and a negative predictive value of 0.97 (0.96, 0.98) for PFD detection. Furthermore, the protocol showed a specificity of 0.94 and a negative predictive value of 0.99 for each of the PFD criteria separately (reintervention, CCU admission and length of stay longer than seven days).
With regard to multivariate regression analysis, we only found a relationship with PFD with 2 factors, preeclampsia [Odds Ratio = 8.25 (3.67, 17.9), p < 0.001] and MEWC alarm activation [Odds Ratio = 3.98 (1.63, 8.99), p = 0.001] (Figure 2). The only factor related to the reintervention was the MEWC alarm activation (Odds Ratio = 4.58 [1, 15.4], p = 0.02, Figure 3-A). The main factor related to critical care admission was preeclampsia (Odds Ratio = 27.4 [8.87-94], p < 0.001), followed by MEWC alarm activation (Odds Ratio = 11.0 [3.34-36.4], p < 0.001) and cesarean delivery (Odds Ratio = 6.86 [1.87-29.3], p <0.001) (Figure 3-B). We found no relationship between length of stay and MEWC alarm activation in the Cox regression (Figure 3-C).
No significant p-value was rejected after the calculation of q-value within the multiple comparability study.
Discussion
The first thing that draws attention to the results of implementing the MEWC in our hospital is the data on sensitivity, specificity, positive and negative predictive value of alarm activation. Our protocol’s sensitivity is very low (0.28), while specificity and negative predictive value are very high (0.94 and 0.99, respectively). For practical purposes, these results mean that we can be confident that patients without activation of the MEWC protocol alarm are highly unlikely to suffer a PFD. However, the cost was not to detect many patients who developed PFD (28 out of 39, 71.1% of total cases). Although we do not know the causes that have led to such low sensitivity levels, we suspect that it is probably related either to the patient’s monitoring period or to the different parameters that form the protocol.
Our study monitored the patients for only 2 hours after delivery. However, other studies with better sensitivity results had a much more excellent follow-up. Singh et al.11 published the first study to validate MEWC, carrying out a prospective observational study in 676 obstetric patients to detect morbidity between the 20th week of gestation and six months postpartum. 30% of the patients had an altered trigger. They defined a MEWC sensitivity of 0.98 and a specificity of 79%. The introduction of such an extended period of obstetric monitoring is probably impossible to implement in our centre. Still, it does make us suspect that if we extend the monitoring period from the time of admission to 24 hours after birth, the number of false negatives should be reduced.
On the other hand, the modification of the protocol’s composition by eliminating the respiratory rate and adding obstetric factors has also been able to modify the sensitivity and specificity of the protocol. Initially, we can think that the more complex a protocol is, the more difficult it is to generate false negatives and the more secure its implementation is. However, this is not as true as it seems. Mhyre et al.8 described that the simpler the warning criteria, the shorter the time for recognition, diagnosis and treatment of women who are more at risk of developing obstetric complications. The more uncomplicated measures are more reliable, less vulnerable to human calculation errors and have higher reproducibility. Mhyre et al. insist on creating multidisciplinary teams that define their warning criteria for each centre and review the evidence to date of the use of these criteria8. In 2015 Edwards et al. 12 compared the predictive value of six different MEWS to identify severe sepsis in women with chorioamnionitis and informed of a sensitivity range of 40%-100% with a degree of 4%-97% specificity. The authors concluded that the MEWC tools with simpler designs tended to be more sensitive while more complex ones tended to be more specific. Probably, by adding the obstetric items, we increased the complexity of the protocol and caused an increase in specificity.
Likewise, a complex MEWC protocol that causes a high number of false positives is a severe limitation in its application since it can lead to professional fatigue due to the high number of false alarms13. Of course, no MEWC protocol is ideal, and we fully agree with Friedman et al.13. They exposed that the data cannot be extrapolated to other healthcare centres, arguing that the results should be obtained for individual hospitals. Further studies are necessary to validate and generalize MEWC principles.
Lastly, although the results found have not been as satisfactory as we would have liked, there are positive readings about the implementation of our MEWC protocol. Multivariate analyses have shown us a direct relationship between the activation of the MEWC protocol alarm and the appearance of a PFD, a reintervention or a CCU admission. Probably, as an isolated event, the activation of the alarm has low sensitivity, but, together with the other correlated factors, it will allow us to create risk profiles. If we analyze our sample, which contained 65 patients with preeclampsia, 14 of them, i.e. 21%, suffered a PFD, and 35.9% of PFD was in paeclampsia patients. The odds ratio found in the relationship between preeclampsia and PFD [8.25 (3.67, 17.9)] indicates that when the MEWC alarm is activated in a preeclampsia patient, the risk of PFD will be very high. Therefore, this patient profile should be monitored very closely.
Conclusion
Our MEWC protocol presented low sensitivity and high specificity with a high negative predictive value, having many false-negative patients. Our study showed an association between alarm activation and PFD, reintervention rate and critical care admission rate, with preeclampsia being the most related factor in all of them.
Data Availability
All data has been self collected during fieldwork and is available for review if needed.
Footnotes
Conflict of interest: The authors have no conflict of interest concerning this manuscript.
Funding: The authors did not receive public or private funds for the realization of this manuscript.