A systematic review and meta-analysis of longitudinal cohort studies comparing mental health before versus during the COVID-19 pandemic

Introduction

As of February 2020, the COVID-19 pandemic has been responsible for more than 2.5 million deaths worldwide (1). Soon after the COVID-19 crisis was declared a pandemic by the World Health Organisation (WHO) on the 11^th March, concerns were raised over a potential parallel mental health crisis fuelled by the pandemic and the associated social restrictions imposed by governments to reduce virus transmission (2, 3). The mental health of ‘at risk’ groups during the pandemic, such as those with pre-existing mental health conditions, has also been highlighted as cause for concern (3, 4).

There are considerable methodological challenges to quantifying the impact that the pandemic has had on mental health. During the outbreak of the pandemic studies indicated that many participants perceived their mental health had worsened (5, 6), but retrospective reports of change in mental health are prone to substantial bias (7, 8). Other studies have found a greater incidence of mental health problems in cohorts recruited during the pandemic compared to different cohorts who completed measures prior to the pandemic (9, 10). However, differences in how samples were recruited (e.g. greater reliance of online and non-probability based samples during the pandemic) and differences in demographic profiles of pre vs. post-pandemic outbreak cohorts also make inferring change in mental health attributable to the pandemic difficult (11).

A number of longitudinal cohort studies have sampled the same participants before and during the pandemic to examine how mental health has changed. In a large nationally representative sample of UK adults, Daly et al. found that non-specific general mental health symptoms (GHQ-12) increased in April-June 2020 compared to a pre-pandemic baseline (12). Other longitudinal cohort studies have found little change in mental health (13) or mixed evidence (14). Longitudinal studies are uniquely placed to characterise the time course of the mental health burden associated with the COVID-19 pandemic. Although the effects of the pandemic on mental health could be long lasting (15), a recent multi-wave longitudinal study of US adults found that after an initial increase in distress during the early stages of the pandemic, distress reduced to pre-pandemic levels within a few months (16).

There has been no systematic review and meta-analysis of longitudinal cohort studies with pre-pandemic measurement of mental health (17-19). We systematically reviewed and meta-analysed studies that examined longitudinal changes in mental health among the same sample of participants before vs. during the pandemic, in order to quantify the size of the mental health burden associated with the pandemic and how this burden changed over time. Secondary aims were to examine whether changes have been symptom specific and whether changes differed based on population demographics.

Methods

Results

Article identification

After removal of duplicates, title and abstract screening of electronic database search results and identification of eligible articles through other sources, 153 articles were full-text screened. A total of 65 articles were eligible for inclusion. See Figure 1 for study selection flow chart.

• Download figure
• Open in new tab

Figure 1: Study search and inclusion flow chart

Not a research report indicates manuscript did not report on a study. Excluded as unable to include in meta-analysis (N=1) was due to authors not providing the required statistical information to compute effect size.

Risk of bias

Due to the small number of studies examining changes in % of sample meeting questionnaire cut-offs, risk of bias analyses were limited to SMC studies. See Table 2 for risk of bias summary information and results and Table S2 for study-level information. We found limited evidence that the risk of bias indicators predicted size of change in symptoms, except that change in mental health tended to be larger when delivery mode of questionnaire was consistent pre vs. post pandemic. In an exploratory analysis limited to a more homogenous collection of studies that were of lower risk of bias results were consistent with the main analyses. See Figure 2.

• Download figure
• Open in new tab

Figure 2: Forest plot of effect sizes from studies of depression, general mental health and anxiety symptoms with lower risk of bias.

Analyses of more homogenous collection of studies (depression, anxiety, general mental health measures only) that were of lower risk of bias (used representative sampling, did not report inconsistent mode of survey delivery, reported no conflicts of interest) of ∼27,736 participants, SMC = .100 [95% CI: .033 to.166], z = 2.95, p = .003, I2 =98.0%). General (general population sample), MH (sample with pre-existing mental health condition).

Change in numbers exceeding questionnaire cut-offs for mental health problems

Twenty-four comparisons across 12 studies (∼21,825 participants) were included. There was a significant effect (single level meta-analysis), with increased odds of exceeding a questionnaire cut-off for mental health problems from pre-post pandemic (Marginal Odds Ratio = 1.31 [95% CI: 1.10 to 1.55], z = 3.18, p = .001, I² = 93.2%), where an OR of 1.5 is tconsidered a small sized effect. See Figure 3. There was no evidence of publication bias or influential cases. See online supplementary materials for full results.

• Download figure
• Open in new tab

Figure 3: Forest plot of effect sizes for exceeding questionnaire cut offs.

General MH (measure of general mental health functioning)

General Discussion

Across sixty-five longitudinal cohort studies comparing mental health prior to and during the COVID-19 pandemic there was a statistically small overall increase in mental health symptoms. The increase in mental health symptoms was largest among studies that sampled participants in the early stages of the pandemic (March-April) and severity of mental health symptoms decreased significantly over the following months (May-July). This pattern of results may represent an acute and normal response to an unforeseen and distressing traumatic event (22), which was then followed by a period of psychological adaptation and resilience (16, 23). In line with this interpretation, a large study of US adults found that perceived risk and worries about financial instability, infection and death were highest during the early stages of the pandemic(24). However, as more information about the pandemic became available perceived risks decreased and this predicted recovery to pre-pandemic levels of distress (24). Similarly, in a large sample of UK adults recruited after the pandemic outbreak, both anxiety and depressive symptoms showed a trajectory of recovery from the beginning of April onwards (25).

Worsening of mental health symptoms was largest in studies examining depression and mood disorders symptoms, and a small increase in depression symptoms was still observed during May-June. Change in anxiety symptoms was smaller and was not significant in samples measured in May-June. Changes in non-specific general mental health functioning (including distress) and well-being were small and non-significant. The more pronounced change in depressive symptoms observed may reflect the effects of isolation and sadness caused by social restrictions and loss of life during the pandemic (26). Increases in mental health problems were observed across most population sub-groups sampled (e.g. general population, university students, existing physical health condition).

Contrary to concerns raised early in the pandemic (3, 4), changes in mental health were less pronounced among people with pre-existing mental health conditions and overall there was no statistically significant change in mental symptoms in this group. These findings may in part be explained by regression to the mean and naturally occurring recovery of more severe mental health symptoms over time (27), as well as stay at home restrictions that potentially provided a more structured routine and reduced exposure to external stressors (e.g. large social gatherings) among those with severe mental health conditions (22). A number of countries, such as China, introduced new mental health services in response to the pandemic (4) and this may have relieved symptoms among those with and without pre-existing mental health conditions. In contrast, mental health symptoms increased among those with pre-existing physical health conditions, which may reflect the elevated risk posed by COVID-19 to this group.

Increases in mental health symptoms were observed in both North American and European samples, though there were a limited number of studies from other continents. We examined whether a range of country-level factors explained heterogeneity in mental health change across samples, including number of country level COVID-19 cases and attributed deaths in the month that mental health was measured, levels of government financial support and social restrictions in place to reduce virus transmission. We found no evidence that country level factors explained heterogeneity. However, it is difficult to make firm conclusions because the majority of studies were conducted in the early phase of the pandemic when deaths from COVID-19 were high and restrictive measures had already introduced relatively high levels of restriction. A number of countries also implemented restrictions regionally.

A smaller sub-set of studies examined change in proportion of sampled population exceeding questionnaire cut-offs for clinically relevant mental health symptoms. In line with the main analyses, there was a statistically small increase in likelihood of meeting questionnaire cut-offs. A limitation of these studies is that although questionnaires used have been shown to be valid indicators of clinically relevant mental health disorder symptomology, it was common for response formats to be altered to ask participants to report on shorter time frames (e.g. the last week). Therefore, these studies provide an indication of acute symptom severity rather than clinical diagnostic value (e.g. anxiety disorder diagnosis typically requires symptoms for several months (28)).

In contrast to the fear that the COVID-19 pandemic would cause a parallel and longstanding mental health crisis (2, 3), the present findings suggest that overall there has been considerable resilience in mental health. Data on recorded suicides align with this, as there have been stable rates or decreases reported across a number of countries (29). However, there is a need for continued mental health provision and monitoring particularly during periods of increased COVID-19 infection and death. The increase in depression and mood disorder symptoms that did not return to pre-pandemic levels warrants attention, as even a small upward shift in depressive symptoms may have meaningful cumulative consequences on the population-level.

There are limitations to the studies included. The majority of studies sampled populations in developed countries during the early stages of the pandemic. Given that there have been second waves of the pandemic in many countries during early 2021, it will be important to continue to monitor mental health. Heterogeneity of models ended to be high and although some of this variability was explained by symptom type and population groups sampled, it will be important to identify further explanatory factors. For example, there are population sub-groups who may be at greater risk of declines in mental health that we were unable to examine and who are likely to be under-represented in studies of the general population. Due to stressful working conditions frontline healthcare workers may be at increased risk and there may be long-term mental health effects among those who become seriously ill from COVID-19, live in nursing homes or those who suffered financially as a result of the pandemic (30). Participants with pre-existing mental health conditions tended to be grouped in studies and therefore we were unable to examine changes in symptoms among specific patient groups (e.g. schizophrenia). We found little evidence that study outcomes were strongly related to individual risk of bias indicators and in an analysis limited to relatively high-quality studies results were similar to the main analyses. Level of attrition was high across studies and this is a limitation. Although there was no significant evidence of attrition being associated with change in symptoms in meta-regression or sub-group analyses and lower levels of attrition tended to be associated with a smaller as opposed to larger increase in symptoms.

Conclusions

Among longitudinal cohort studies examining mental health prior to and during the COVID-19 pandemic in 2020 there was a significant but statistically small increase in mental health symptoms. Increases were larger for depressive symptoms compared to anxiety and measures of overall mental health functioning. The overall increase in mental health symptoms was most pronounced during the first two months of the pandemic, before decreasing and being comparable to pre-pandemic levels by mid-2020. There was no evidence of a worsening of mental health symptoms among samples of participants with a pre-existing mental health condition.

Data sharing

Study data file and pre-registered analysis protocol are available online https://osf.io/rg24v/. Analysis code and files are available on request.

Contributors

The study was designed by all authors. AJ analysed the data. The first draft was written by ER. All authors critically revised the manuscript and agree to be accountable for all aspects of the work.

Role of the funding source

There was no funding source for this study.

Declaration of interests

All authors report no conflicts of interest. ER has previously received funding from the American Beverage Association and Unilever for projects unrelated to the present research.