ABSTRACT
The current epidemic of COVID-19 is unparalleled in recent history as are the social distancing interventions that have led to a significant halt on the economic and social life of so many countries. However, there is very little empirical evidence about which social distancing measures have the most impact. We report a quasi-experimental study of the impact of various interventions for control of the outbreak. Data on case numbers and deaths were taken from the daily published figures by the European Centre for Disease Control and dates of initiation of various control strategies from the Institute of Health Metrics and Evaluation website and published sources. Our primary analyses were modelled in R using Bayesian generalised additive mixed models (GAMM). We found that closure of education facilities, prohibiting mass gatherings and closure of some nonessential businesses were associated with reduced incidence whereas stay at home orders, closure of all non-businesses and requiring the wearing of facemasks or coverings in public was not associated with any independent additional impact. Our results could help inform strategies for coming out of lockdown.
Introduction
The current pandemic of COVID-19 is unprecedented in recent history. Not only is the impact of the epidemic being measured by the number of cases and deaths, but also by its impact on overloaded health services and deleterious impacts on quality of life and near-future economic prospects. The impact on wider society is seen through the almost total stasis of our social and cultural life. To a large extent the impact of social distancing has been shown in China, Italy and Spain that have already turned the tide on their country’s epidemics using often severe social distancing strategies. What these examples do not do is indicate the relative importance of the different non-pharmaceutical/ social distancing interventions. Given the potentially high economic and social costs arising from stringent control measures1–5, it is imperative to determine which social distancing measures are most effective at controlling the pandemic. Any relaxation of current control measures should be informed by such knowledge. At present much policy is being driven by the results of mathematical models6. However, there is already debate about the validity and limitations of the different models for policy making and modelling approaches that have been used7–10. What has been missing is empirical evidence of what aspects of currently applied non-pharmaceutical control measures have or have not been effective.
A quasi-experimental study design is an intervention study where the allocation to receive the intervention (or not) is not randomly made. At present, most European states have introduced a similar suite of interventions aimed at reducing contact between individuals to reduce transmission. However, the different types of intervention used and their timing vary from one country to another. No measure was imposed by all European countries and where measured were imposed, they were often imposed at different points in the development of the epidemics. This situation offers a unique opportunity to investigate the putative impacts of the various types of intervention, as each individual-country epidemic forms what is effectively a chronosequence of disease spread. The intervention strategies can then be compared as interrupted time series.
We report here preliminary analyses of trends in both reported cases and deaths across 30 European countries with rather different approaches to and timing of restrictions. We use a quasi-experimental approach to identify what affects such restrictions may have had on the control of the epidemic.
Methods
Data on new cases and deaths reported by all countries were obtained by the European Centre for Disease Control (https://www.ecdc.europa.eu/en/publications-data/download-todays-data-geographic-distribution-covid-19-cases-worldwide). Data up to 24th April are included. For the UK we used only pillar 1 case numbers. Pillar 1: swab testing in Public Health England laboratories and National Health Service hospitals for those with a clinical need, and health and care workers. Pillar 2 results (swab testing for health, social care and other essential workers and their households) as reported daily on https://www.gov.uk/guidance/coronavirus-covid-19-information-for-the-public#history were removed from the case numbers, as pillar 2 sampling was only introduced late in the course of the UK epidemic and has inflated total case numbers relative to earlier in the UK outbreak. We also adjusted by the number of tests reported per 1 million population taken as of 16th April from WorldoMeter (https://www.worldometers.info/coronavirus/). In order to compare time series for different countries with different dates of onset for their own epidemics we chose to define the onset as the first day after the latest time where there were two or more consecutive days with no cases reported.
The dates when (if at all) various of four social restrictions were imposed for 30 European countries are given by the Institute of Health Metrics and Evaluation Data (IHME) (https://covid19.healthdata.org/). The six categories are “Mass gathering restrictions”, “Initial business closure”, “Educational facilities closed”, “Non-essential services closed” and “Stay at home order” and “Travel severely limited”. However, no country was in this analysis listed as having severe travel restrictions and this will dropped from any further analysis. The IHME definitions of these are given on their website. Paraphrasing the definitions.
Mass gathering restrictions are mandatory restrictions on private of public gatherings of any number.
The first time that there was any mandatory closure of businesses, not necessarily all businesses. Usually such initial closures would primarily affect business such as bars and restaurants.
Where non-essential businesses are ordered to close this includes many more businesses than may be in the previous category and include general retail stores and services such as hairdressers.
Education facilities closure includes all levels (primary, secondary and higher) education that stop direct teacher to student teaching.
Stay at home orders affect all individuals unless travelling for essential services, allowing only close contact with people of the same household and perhaps some outdoors exercise.
In addition, we sought information on when countries started to advise or require their citizens to wear face masks or coverings. The dates when government recommendations or compulsory orders about face coverings (whichever date was earlier) started over the majority of the population were collected from credible sources, as listed in Supplemental file 1. However, there was substantial heterogeneity in how the wearing of face coverings in the community was encouraged or mandated and in what contexts, such as always outside the home or just in shops or on public transport. This heterogeneity combined with their relative recent introduction means that we do not yet endorse using the results about face covering use (in our main model) being used to inform public policy. We also adjusted by the number of tests reported per 1 million population taken on the 17th April from WorldoMeter (https://www.worldometers.info/coronavirus/).
We undertook two sets of analyses. The first is a multi-level mixed effects regression analysis in STATA v 16.1. We used a mixed effects negative binomial regression model with cases or deaths on a specific day as the outcome variable, country population as the exposure variable, country as a mixed affect, and days from start of the epidemic as a fixed effect. All main interventions were included as categorical fixed effects in the model as the number of weeks after the start of the intervention with day 1 being the day following the intervention implementation.
The second analysis was done in R using Bayesian generalised additive mixed models (GAMM) to adjust for spatial dependency in disease between nation states. The variance in the COVID-19 data was four orders of magnitude larger than the mean number of cases, and three orders of magnitude larger than the mean number of deaths. Consequently, models were fit using a negative binomial specification to account for potential over-dispersion in the data.
Let Yi,t be the number of COVID-19 cases or deaths for country i = 1, …, I at time t = 1, …, T The general algebraic definition of the models is given by:
where Yi,t is the number of COVID-19 cases or deaths for country i = 1, …, I at time t = 1, …, μi,t is the predicted number of COVID-19 cases or deaths for country i and time t, and ⍰ > 0 is the negative binomial dispersion parameter. A logarithmic link function of the expected number of cases or deaths is modelled as:
where α corresponds to the intercept; log(Pi,d[t]) denotes the logarithm of the population at risk for country i and day d[t] included as an offset to adjust case counts by population. Di,d[t] is a linear term for the number of days since the outbreak started, with coefficient δ. Ri,d[t] is a linear function of the number of COVID-19 tests carried out per country i at day d[t], with regression coefficient ⍰. X is a matrix of k intervention measures (e.g. school and business closures) with regression coefficients ϐ. Intervention measures comprise of an index of 1, …,N number of days following the intervention being implemented. We assumed that the imposition of each intervention led to a progressive change in effect. Intervention measures were included in the model as a random effect to account for potential non-linearities in the exposure-response relationship. Unknown confounding factors with spatial dependency that represent, for example, human mobility, were incorporated using spatially correlated (i.e. structured) random effects (ui) and independent, identical and normal distributed (i.e. unstructured) random effects (νi) for each country i. Spatial random effects were specified using a Besag-York-Mollie model to account for spatial dependencies and unstructured variation between countries11. Goodness of fit was evaluated using the Deviance Information Criterion (DIC). Models were fitted in R version 3.6.1 using the INLA package.
Results
Table 1 gives the estimated date of the start of the epidemic in each country and when each of the five intervention types were implemented, according to the IHME website. “Travel severely limited” was not introduced in any European country. “Mass gathering restrictions”, “initial business closure”, “educational facilities closed”, “non-essential services closed” and “stay at home order” were respectively implemented by 29, 28, 29, 23 and 19 countries. In three countries (Germany, Italy and Spain) the restrictions were not implemented uniformly through the country and so we took the median date. Italy was the first country to enter the epidemic on 22nd February and Lithuania the last on 14th march. Half of all countries had their epidemic start on or before 27th February.
Timing of estimated start of each country’s main epidemic and the introduction of social distancing measure across 20 European countries (all dates in 2020).
Hierarchical probabilistic models were used to estimate the effects of interventions on the number of COVID-19 cases and deaths across 30 European countries. The model metrics are presented in Table 2.
Model metrics
The exposure-response relationships estimated by the models are presented in Figures 1 (cases) and 2 (deaths). The X axis represents the days since the intervention started and the Y axis indicates the logarithm of the risk ratio. It can be observed that mass gathering restrictions have a negative effect on the number of cases with less cases occurring as the number of days since intervention started increases. A similar effect is observed for the initial closure of business and the closure of education facilities with less cases occurring as the number of days since the intervention increases. The closure of non-essential business does not appear to have a significant effect on the number of COVID-19 cases. This is evident as the estimated relationship and its 95% credible interval stay close to zero on the Y axis. Surprisingly, stay-home measures showed a positive association with cases. This means that as the number of lock-down days increased, so did the number of cases. The use of face coverings initially seems to have had a protective effect. However, after day 15 of the face covering advisories or requirements, the number of cases started to rise. Similar patterns were observed for the relationship between face coverings and deaths. Negative associations were estimated for mass gatherings, initial business closure and the closure of educational facilities; while a non-significant effect was estimated for non-essential business closure. There was a positive association with the usage of face coverings (masks), while the stay-home measures showed an inverted U quadratic effect with an initial rise of cases up to day 20 of the intervention followed by a decrease in cases. These results would suggest that the widespread use of face masks or coverings in the community do not provide any benefit. Indeed, there is even a suggestion that they may actually increase risk, but as stated previously, we feel that the data on face coverings are too preliminary to inform public policy. We have more confidence that results for stay at home orders suggest that such orders may not be required to ensure outbreak control.
Incidence Rate Ratios following implementation of country level non-pharmaceutical control measure and daily reported COVID 19 case numbers in 30 European countries.
Figures 3 and 4 shows the association between actual cases and deaths in each country, expressed as 7 day rolling means, and the numbers predicted by the models on cases and deaths. Although for many countries there is a reasonable correlation between the two this is not the case for all countries and particularly the smaller countries. This is most noticeable for Sweden which had lower numbers of cases and deaths than predicted, though this could be explained by partial implementation of controls and unmandated behavioural change in the population. This observation would suggest that, at least for some countries, our model does not capture all the temporally changing variables influencing the spread of the disease.
Incidence Rate Ratios following implementation of country level non-pharmaceutical control measure and daily reported deaths from COVID-19 in 30 European countries.
Comparison of predicted daily reports of case numbers of COVID-19 with seven day rolling average actual numbers across 30 European countries.
Comparison of predicted daily numbers of reports of deaths COVID-19 with seven day rolling average actual numbers across 30 European countries.
The maps of the posterior means of the country-specific risk ratios are shown in Figure 5. These maps can be interpreted as the residual risk ratio for each country after accounting for all other covariates in the model. Figure 5 also shows the country-specific posterior probability of exceeding one case or one death. The proportion of spatial variance explained by the models is 16% for the case-specific model, and 15% for the death-specific model.
Posterior mean of the country-specific risk ratio of COVID-19 A) cases and C) deaths; and posterior probability of exceeding one COVID-19 B) case or D) death.
For comparison, the analysis using a multilevel mixed effects model has results shown in Table 4. The conclusions of this analysis were the same as for the hierarchical probabilistic models described above. In addition, we looked at the impact of removing each intervention or all interventions on the model log likelihoods. The biggest impact came from removing educational closures from the model. The next biggest changes came from removal of stay home, and masks from the model, but neither of these interventions were associated with a decline in epidemic risk.
Results of multi-level mixed effects of each intervention on case numbers and deaths
Log likelihood of each model for full model compared with models excluding each of the interventions and all interventions
Discussion
We have undertaken a quasi-experimental study of the impact of various forms of social distancing interventions on the epidemics of COVID-19 infection in 30 different European countries. Our analyses confirm that the imposition of non-pharmaceutical control measures have been effective in controlling epidemics in each country. However, we were unable to demonstrate a strong impact from every intervention. Closure of educational facilities, banning mass gatherings and early closure of some but not necessarily all commercial businesses were all associated with reduction of the spread of infection. Widespread closure of all non-essential businesses and stay at home orders seem not to have had much if any value. The results on face coverings are too preliminary to be reliable but what results are available do not support their widespread use in the community.
Spatiotemporal hierarchical models, like the one presented here, have the advantage of being able to explicitly quantify the probability that an epidemic may or may not occur at a specific time or location. We have previously used similar models to account for spatiotemporal effects in health outcomes11. Public health officials may be more inclined to deploy interventions if the probability of an epidemic exceeds a certain value. Ideally, public health decision-makers should agree on the specific epidemic thresholds (i.e. the incidence above which the disease requires imposition of control measures) to make model predictions meaningful.
Whether or not school closures are likely to have been important in controlling the spread of epidemic disease is an issue of some debate in both the scientific and lay media. There has been uncertainty about how beneficial the closing of educational establishments can be on respiratory disease transmission12–14. The value of school closures is particularly uncertain for COVID-19 given the observation that children have only mild or no symptoms15. Decline in the infectiousness of the SARS-1 outbreak in Hong Kong in 2003 was also a time when many interventions were implemented, including school closures16, making it hard to disentangle contributions of each individual measure. In the current pandemic, Hong Kong managed to substantially reduce the transmissibility of COVID-19 fairly early in the outbreak by a limited number of interventions one of which was keeping schools closed17. However, there were also substantial behavioural change in the population at the time coincident with these interventions. Viner and colleagues state “Data from the SARS outbreak in mainland China, Hong Kong, and Singapore suggest that school closures did not contribute to the control of the epidemic”. However, this is not a valid argument against school closures as peak shedding of virus in SARS-CoV was around day 10 whereas peak shedding of SARS-CoV-2 is much earlier and possibly before symptoms develop18–20. In contrast to COVID-19, SARS was primarily infectious after onset of symptoms when most cases would have been hospitalised or at least quarantined. Throat swabs from children have shown similar viral load to those in adults21, yet a review of contact tracing studies failed to find incidents where transmission occurred from children to adults21. We cannot resolve the lack of consensus in these lines of evidence, about whether children can pass SARS-COV-2 to adults. What our study also does not do is identify which level of school closure has the most benefit whether it is primary, junior, senior school or even higher education. This will need to be the focus of further research. Note that the results presented here are based on total closure. It is possible that partial school closures such as three-day weekends could have worthwhile impacts on the spread of infection22.
The second greatest impact on the epidemiology of the European COVID-19 was from banning mass gatherings (which could be of any size), both public and private gatherings. A 2018 review of the impact of mass gatherings on outbreaks of infectious disease23 found that most evidence was linked to the Islamic Hajj pilgrimage, where most infections were respiratory, mainly rhinovirus, human coronaviruses and influenza A virus. The evidence for outbreaks arising from other mass gatherings such as music festivals or sporting events is less established, but not absent. Several outbreaks of respiratory infectious disease have been linked to open air festivals and other music festivals23, 24. For instance, during the 2009 influenza season pandemic influenza A(H1N1)pdm09 outbreaks were recorded at three of Europe’s six largest music festivals, while some 40% of pandemic flu cases that season in Serbia were linked with the Exit music festival.
The link with business closures is particularly interesting. The two variables relating to business closures are when the first closures occurred and when all non-essential closures were enforced. For many countries but not all, these two events occurred at the same time. So, there may be substantial collinearity which would underestimate the impact of one or both. Nevertheless, it is worthy of note that the strongest association was with the initial closures. Given that those initial closures were mostly directed at business where people congregate (i.e. the hospitality industry), this would suggest that these businesses are where the most impact may be had. Although outbreaks of food poisoning are frequently linked with restaurants, outbreaks of respiratory infections are much more rarely so. One exception was an outbreak of SARS at a restaurant where live palm civets were caged close to customer seating25.
Our findings on facemasks or coverings are perhaps counterintuitive especially given the strong debate on their use. In a recent systematic review we concluded that the evidence in favour of face mask use outside of hospital was weak.26 On the other hand a recent modelling study concluded that community facemask use could reduce the spread of COVID-19.27 Our results on face coverings should be considered to be preliminary because the use of coverings was recommended or required only relatively late in the epidemics in each European country. The results for face covering are too preliminary to inform policy but indicates that face covering as an intervention merits close monitoring.
Limitations
Although our study suggests that closures of educational interventions and banning mass gatherings are the most important measures, this is caveated with several observations. Collinearity makes it hard to separate out individual intervention effects. Moreover, many interventions were implemented in different ways and at different points in the local epidemic. For example, in accordance with the IHME assignment, we treated Sweden as a country without school closures because schools for persons under 16 stayed open, although upper secondary and tertiary education facilities were actually shut in Sweden from late March 202026. Similarly, the exact timing of restrictions being introduced varied over time in Italy, Spain and between individual federal states in Germany27. Which types of work places could stay open varied; while the acceptable reasons for being outdoors also varied between countries. Stay at home orders in some countries may be an advisory that is not enforced whilst in others it is enforced by police with penalties. In some countries, children may go outside and outdoor exercise is permitted whilst in others either may be banned. In some countries, severe travel restrictions may be a separate intervention whilst in others it may be a consequence of a stay at home order and not be identified separately. Face covering interventions varied hugely between countries: most made face covering voluntary and some only suggested it in specific settings. Because of this variety in how interventions are implemented and described, the results for the potential of stay at home advisories especially may be underestimated. All models are simplifications of the complex nature of reality; our modelling was unable capture many subtle variations in how control measures were implemented. We acknowledge that lack of direct observation of these variations may have biased our results.
Conclusion
Relaxing stay-at-home orders and allowing reopening of non-essential businesses appear to be the lowest risk measures to relax as part of plans to carefully lift COVID-19 lockdown measures. Before now, there was relatively little unclear empirical evidence on the relative value of different interventions. And yet, the reasons to implement minimal control measures are compelling, given the social and economic harm linked to tight control measures. Hence, whilst we need to be cautious about using such preliminary results, public health officials will have to use evidence as it emerges rather than expect to wait for a final full view to decide what might be (was) the best control strategy. Careful monitoring of how relaxation of each control measure affects transmissibility of COVID-19 is required and will help to minimise the inevitably imperfect results.
Data Availability
Data and model code can be shared upon request to authors.
Footnotes
FUNDING: Professor Hunter and Dr. Brainard were funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response at King’s College London in partnership with Public Health England (PHE), in collaboration with the University of East Anglia. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, UEA, the Department of Health or Public Health England. Dr. Colón-González is funded by the UK Space Agency Dengue Mosquito Simulation from Satellites (D-MOSS) project.
CONFLICTS OF INTEREST: The authors declare that we have no conflicts of interest.
Author contributions: PRH and JB conceived of the study. PRH and JB collected the data. PRH, FCG and SR undertook and refined analysis. PRH wrote the first draft which was revised by all authors.
REFERENCES
