ABSTRACT
Different modeling approaches can be used to calculate excess deaths for the COVID-19 pandemic period. We compared 6 calculations of excess deaths (4 previously published and two new ones that we performed with and without age-adjustment) for 2020-2021. With each approach, we calculated excess deaths metrics and the ratio R of excess deaths over recorded COVID-19 deaths. The main analysis focused on 33 high-income countries with weekly deaths in the Human Mortality Database (HMD at mortality.org) and reliable death registration. Secondary analyses compared calculations for other countries, whenever available. Across the 33 high-income countries, excess deaths were 2.0-2.8 million without age-adjustment, and 1.6-2.1 million with age-adjustment with large differences across countries. In our analyses after age-adjustment, 8 of 33 countries had no overall excess deaths; there was a death deficit in children; and 0.478 million (29.7%) of the excess deaths were in people <65 years old. In countries like France, Germany, Italy, and Spain excess death estimates differed 2 to 4-fold between highest and lowest figures. The R values’ range exceeded 0.3 in all 33 countries. In 16 of 33 countries, the range of R exceeded 1. In 25 of 33 countries some calculations suggest R>1 (excess deaths exceeding COVID-19 deaths) while others suggest R<1 (excess deaths smaller than COVID-19 deaths). Inferred data from 4 evaluations for 42 countries and from 3 evaluations for another 98 countries are very tenuous Estimates of excess deaths are analysis-dependent and age-adjustment is important to consider. Excess deaths may be lower than previously calculated.
SIGNIFICANCE STATEMENT Excess deaths are a key metric for assessing the impact of a pandemic. They reflect the composite impact of deaths from infection, from indirect pandemic effects, and from the measures taken. Different modeling approaches can be used to calculate excess deaths for the COVID-19 pandemic. Here, we compare four previous calculations of excess deaths and two new ones that we performed with and without adjusting for changing age structure in the estimation. Proper age-adjustment results in substantial reduction in estimates of excess deaths for 2020-2021. While results from different calculation methods are correlated, the absolute differences in estimated excess deaths are very high in most countries. Extrapolations to countries without reliable death registration is extremely tenuous.
INTRODUCTION
Many studies estimate excess deaths in specific locations, countries, regions, or worldwide during the COVID-19 pandemic.1–4 Excess deaths reflect a composite of deaths from SARS-CoV-2 infection plus indirect effects of the pandemic (e.g. health system strain) and measures taken.5, 6 It has been argued7, 8 that excess deaths are a more appropriate measure of impact than recorded COVID-19 deaths. Recorded COVID-19 deaths may be under- or over-counted in different time periods and locations5 and do not capture indirect effects of the pandemic and the measures taken. However, excess deaths calculations require modeling of the expected deaths that entails many assumptions and analytical choices. To obtain excess deaths estimates one needs to define a control (reference) pre-pandemic period, use some model for extrapolating expected deaths in the pandemic period and compare them against observed deaths. There are many different possibilities on how to select the pre-pandemic reference period and on how to model data and extrapolations.9, 10 A major analytical decision is whether to account for changes in the age structure of the population over time. With an aging population (particularly in high-income countries), mortality rates may increase over time, countering the anticipated decrease in mortality from better healthcare and overall human progress. These changes may be better addressed by considering age structure9, 11–13 rather than simply relying on regression trends of overall population data regardless of age.
Another pivotal dilemma in excess death calculations is what sources of data to use; and which countries are considered to have sufficiently reliable data. Death registration is sub-standard in most countries around the world: many deaths remain unrecorded.14, 15 Information on causes of death has flaws even in the most developed countries16, 17 while the pandemic generated new death coding challenges.18 Changes in deaths over time may be confounded by changes in death registration and recorded COVID-19 deaths depend on coding. Even population counts have uncertainty and this applies also to age-stratified estimates from different sources using different imputations to estimate age-stratified population for recent years. Even in high-income countries, most of them have not had a formal census performed for many years. Finally, several highly visible studies that attempted to calculate excess mortality world-wide,1–4 first estimated excess mortality in countries with trustworthy death data on all-cause mortality; then, they extrapolated across other countries worldwide, using the profile of various characteristics in these countries versus those with trustworthy data. Consequently, proper calculation of excess mortality in countries with most trustworthy mortality data has critical importance even for worldwide estimates.
Here, we compare the results of different evaluations that have attempted to calculate global excess mortality during the COVID-19 pandemic in 2020-2021.1–4 We focus primarily on high-income countries with the most reliable death registration systems and discuss the implications for extrapolations to a global level. We compared 4 widely publicized excess death calculations published in eLife, Economist, the Lancet, and by the World Health Organization (WHO)1–4, 19, 20 and included also our calculations. Our calculations explicitly explored what difference it would make to model deaths using separate death and population data for age strata. Besides raw excess mortality estimates, we focused on the ratio of excess mortality over recorded COVID-19 deaths. This ratio is critical in understanding whether excess mortality confers more information compared with just focusing on routinely recorded COVID-19 deaths. Ideally, one would like to see consistency in this ratio regardless of modeling and analytical choices, while large inconsistency would put the added value of excess mortality calculations in question.
RESULTS
Characteristics of the compared excess death calculations
Table 1 summarizes the main features of the compared excess death calculations. As shown, the evaluations differed substantially in defining the reference period, the choice of analytical model, exclusions, use of adjustments, eligibility criteria, data sources, extrapolations and imputations. The Economist methods used complex machine learning not described in very broad terms; the WHO and Lancet methods were convoluted and not all can be easily reproduced.
Comparison of excess deaths metrics in 33 countries with most-reliable data
33 countries were included in the main analysis, selected because they were high-income according to the World Bank, and they had detailed available weekly data on observed deaths in the three pre-pandemic years and also during 2020-2021 (Table 2). These countries had a total population of almost 1 billion and almost 20 million deaths in 2020-2021, of which 9.5% were recorded as COVID-19 deaths. The total of calculated excess deaths in these 33 countries ranged from 2.0 million (eLife) to 2.8 million (Lancet), with WHO (2.1 million) and Economist (2.2 million) being closer to eLife. Our own calculations without age adjustment gave a similar total (2.3 million), but with age-adjustment the excess deaths were only 1.6 million.
There were very large differences across countries and this was evident also when excess deaths were estimated as a proportion above the observed deaths, Ep (Supplementary Table 1), deaths per million population, Em (Supplementary Table 2) and ratio of excess deaths over recorded COVID-19 deaths, R (Table 3) metrics. Australia, New Zealand, and in some analyses also Iceland showed no excess deaths at all, even without age-adjustment. With age-adjustment, however, in our calculations several other countries such as Norway, Finland, Denmark, Sweden, and South Korea also showed no excess deaths. In the WHO age-adjusted calculations, no excess deaths were seen in Australia, New Zealand, Iceland, and Norway. Among countries with excess deaths in all analyses, the excess death estimate differed by 2-4 times between the highest (non-age-adjusted) and lowest (age-adjusted) estimates in countries like France, Germany, Italy, and Spain. Age-adjustment led to modest reductions of estimates for the USA.
The pairwise correlations between eLife, Economist, WHO and our two calculations (with and without age-adjustments) were consistently extremely high for R (all r>=0.977). The same picture was seen largely for these 5 calculations for Ep (all r>=0.930) and for Em (all r>=0.951). The Lancet calculations had modestly lower correlation with the other 4 evaluations (range r=0.808-0.841 for R, range r=0.835-0.919 for Ep, 0.917-0.958 for Em). The correlations for R between different evaluations, however, were modest/poor when Australia and New Zealand (that were outliers with very negative values of R) were excluded, (range -0.386 to 0.657). The two other metrics Ep (range, 0.794-0.989) and Em (range 0.904-0.996) show good correlation values even when these outliers are excluded.
Even when correlations were high, given the substantial differences in the absolute estimates of excess deaths with the different calculations in each country, the range of R values was always large (Table 3). The R values’ range always exceeded 0.3 in all 33 countries. In 16 of 33 countries, the range of R across different evaluations exceeded 1, i.e. it was as large as the number of recorded COVID-19 deaths itself. 6 countries had consistently R >1, i.e. more excess deaths than recorded COVID-19 deaths, with all empirical evaluations. Conversely, only 2 countries, Australia and New Zealand had consistently R<1. The large majority of countries (25 of 33) had some calculations suggesting R>1 (i.e. excess deaths greater than COVID-19 deaths) and others suggesting R<1 (excess deaths smaller than COVID-19 deaths).
Table 4 shows the break-down of excess deaths per age stratum in each country for our age-adjusted analyses. In total, 0.478 of the 1.609 million excess deaths were in people <65 years old (29.7%), but the percentage varied widely across countries. 30 of 33 countries had death deficit for children 0-14 years old (all, except for Iceland, Luxembourg, and Netherland that also had minimal excess deaths in children) and overall across all 33 countries there was a death deficit of 7,737 deaths for children 0-14 years old. 10 countries had death deficit even for people <65 years old. Conversely, for 4 countries, more than 25% of the excess deaths was in people <65 years old (Canada 45.5%, USA 41.6%, Chile 37.8%, UK 28.7%).
Other countries
eLife, Economist, Lancet, and WHO had estimates of excess deaths in 42 additional countries (Supplementary Table 3). These countries had a total population of almost 1.4 billion, the total reported COVID-19 deaths were almost 2.3 million and the excess death estimates were about double with all 4 evaluations (4.7 million per eLife, 4.8 million per Economist, 5.5 million per Lancet, 4.4 million per WHO) with higher correlations between eLife, Economist and WHO and more modest correlations with Lancet for all three excess death metrics. For several countries, differences across evaluations were very large. Two countries had death deficits by some evaluations, but not with others (Singapore range -1,770 to 1,776 and Japan range -19,469 to 111,000).
Another 98 countries (total population 5.3 billion) had excess death values obtained only per Economist, Lancet and WHO (Supplementary Table 4) for a total of 10.7, 9.8, and 7.9 million, respectively (6-9 times higher than recorded COVID-19 deaths). The correlation of the calculations was modest for all three metrics (r=0.499-0.570, r=0.552-0.662, r=0.530-0.801 for Ep, Em, and R, respectively). Very large differences across different calculations for the same country were very common. For 11 countries, there was an estimated death deficit based on some calculation but not with all 3. For China, the range was extreme (452,669 excess deaths per Economist; -52,064 deficit per WHO).
DISCUSSION
Across 33 high-income countries with total population of approximately 1 billion and highly thorough death registration, different empirical estimates of excess deaths in 2020-2021 ranged widely from 1.6 million, i.e. substantially fewer than the 1.9 million recorded COVID-19 deaths, to 2.8 million, almost a million more. Countries with highest estimates of excess deaths were more or less the same in all evaluations; and countries with the most favorable picture performed well across the different evaluations. However, large differences emerged in the magnitude of the estimates for each country. The largest divergence was produced by whether age adjustment was used in calculating excess deaths. Age-adjusted estimates were lower. They are more appropriate, since COVID-19 has impressive age-related risk gradient for death.20, 21 Modest changes in age structure, in particular with aging populations, may produce major differences in estimates. With age-adjustment, several high-income countries showed no or minimal excess deaths; and for many others, the estimates of excess deaths markedly decreased. However, even age-adjusted excess death estimates differed substantially depending on the choice of data source for age-stratified population in the recent years as well as modeling choices.
Our age-adjusted analyses also revealed large differences across countries in the proportion of excess deaths accounted by non-elderly age strata. A few countries had more than a quarter of the excess deaths in people <65 years old. There are several possible factors that may explain this pattern. Adverse risk profile of the non-elderly populations, including high prevalence of obesity; inequalities and disadvantaged populations without good health care; and increased fatalities due to opioid overdose and other non-COVID-9 causes of excess death may explain in part the USA pattern. Also for Canada, USA and UK, many deaths in elderly people may have occurred in long-term care residents with very limited life expectancy. Deaths from SARS-CoV-2 among patients with limited life expectancy result in no excess deaths if the time window assessed after their infection is shorter than their life expectancy.23
The ratio R of excess deaths over recorded COVID-19 deaths varied substantially across different evaluations. Only 5 countries in Eastern Europe and USA had consistently R >1, i.e. more excess deaths than recorded COVID-19 deaths. Conversely, only Australia and New Zealand had consistently R<1. R values varied widely for all countries. In most countries, the uncertainty in the range of excess death estimates exceeded the number of recorded COVID-19 deaths itself. This large variability questions to what extent excess deaths can give much better insights on the total pandemic toll than COVID-19 recorded deaths.7 Excess death calculations are dependent on how they are calculated. In some cases, like Eastern Europe, they can tell that COVID-19 deaths have been undercounted and/or the numbers of other deaths have escalated during the pandemic. However, they cannot differentiate the relative contribution of these two factors nor can they give a precise estimate of either or their combination. In a few countries with limited SARS-CoV-2 deaths they can be reassuring that indirect pandemic effects and measures did not escalate fatalities, at least during 2020-2021. However, for most countries, excess death calculations are so model-dependent that they should be seen with great caution.
Besides the high-income countries with meticulous weekly death registration data, excess death calculations for the rest of the world are very tenuous exercises. For 42 countries where eLife, Economist, Lancet and WHO generated estimates, on average excess deaths were ∼2-times the COVID-19 recorded deaths and among another 98 countries where Economist and Lancet provided estimates, overall R was 6-9. However, given the low reliability of the data, the immense uncertainty surrounding these estimates cannot be overstated. More importantly, these calculations offer no causal insights. Excess deaths may be due to the virus, the indirect pandemic effects, or/and disruptive measures taken, even more so in countries with very frail healthcare systems, widespread poverty, and/or even high rates of hunger. The 2020-2021 crisis may have indeed resulted in many deaths, but causes may be very complex.
For some of these additional countries, calculations were run based on available mortality data. Even then, in most cases death registration is unreliable and the impact of changes in death registration during the pandemic compound any calculation. Perhaps age-adjustment would also lead to different estimates in these countries, as for the 33 most data-reliable countries. Even for these 33 countries, population estimates (including age-stratified population counts) are projected with different methods and typically no formal population census has been performed for several years. This adds further uncertainty to excess death calculations that are highly susceptible to minor differences especially in the population of elderly strata. For most countries, excess death calculations are indirectly imputed from the countries with mortality data. The methods employed by the Economist are not described in sufficient detail to allow probing validity and reproducibility. Lancet and WHO calculations provide more elaboration, mostly proving their complexity. The uncertainty in excess death estimates apparently far exceeds the width of published confidence intervals.
Hence, extrapolations from the 33 main analysis countries to other countries need to be extremely cautious. Among the total excess deaths, deaths due specifically to viral infection in the other countries may be proportionally far less. Other countries have far smaller percentage of elderly people and very few nursing home residents. Conversely, they have far more frail health systems, societies, and economies. Therefore, probably indirect effects of the pandemic and measures were perhaps more important contributors to total excess deaths than SARS-CoV-2 infection.
Nepomuceno et al. have also assessed the impact of different analytical choices on excess death calculations for 2020.8 They find modest differences among different approaches in countries with reliable death registration. Age-stratification has been also to be important in previous assessments for specific countries.24, 25 Germany is a classic example. Our age-adjusted estimate is 55,000 excess deaths, while without age-adjustment we calculated 129,000 excess deaths and Lancet calculated 203,000 excess deaths – compared with 111,000 COVID-19 reported deaths. Baum23 calculated only 22,000 excess deaths after age adjustment, while Koenig24 without age adjustment found more excess deaths than recorded COVID-19 deaths. In Germany, the number of people aged >80 years increased from 4.6 million in 2016 to 5.8 million in 2020, so consideration of age is crucial.11 Further diversity stems from whether modeling anticipates increasing life expectancy (decreasing mortality rate) over time.11 However, this approach may calculate spuriously high excess deaths: calculations assume a desired mortality rate lower than even attained in reality in the past. Medical and overall progress cannot guarantee continuing to decrease overall mortality in high-income countries, especially in old, frail people. In fact, care of such people may have deteriorated over time (e.g. with privatization and deterioration of long-term care) and the pandemic brought this to light.26 Moreover, data from much longer-term periods of observation suggest multiple, overlapping, complex long-term trends in winter mortality.27 Finally, some other models may diverge in their calculations, if they exclude certain periods. E.g., the popular Euromomo model using a Serfling model in its core but excludes weeks of high influenza activity from the modeling: thus it generates high excess death estimates.28
Some other caveats should be discussed. First, finer adjustment for age (e.g. in more narrow age bins), and more comprehensive adjustment for other factors (e.g. gender, frailty, long-term care facility residence, comorbidities) may be able to offer even more accurate estimates of excess deaths. Second, even in countries with reliable data, some deaths are registered with delays, although with >4 months after the end of 2021 our main analyses for the 33 countries should not be affected much. Third, there can be debate on whether/how natural disasters and wars should be excluded. Fourth, long-term effects of both the pandemic and measures taken on healthcare, other aspects of health, education, society, and economy remain uncaptured in the 2020-2021 window. Comparisons of different approaches to excess death calculations should continue for longer periods of follow-up, also in the post-pandemic endemic phase. The boundaries between pandemic and endemic phase can be debated.29 Regardless, the relative performance of different countries and their excess death ranking may change substantially over time. Sixth, analyzing specific causes of death may be informative, but suffers from major misclassification even in high-income countries.
Acknowledging these caveats, our analyses map the magnitude and uncertainty of excess deaths during 2020-2021. In countries with reliable data, age-adjustment suggests that excess deaths is lower than what has been previously published in calculations without age-adjustment. Excess death calculations convey some broad picture, especially for countries that fared very well or very poorly. Large differences in the impact of a pandemic across countries has been seen also in previous pandemics30, 31 for reasons that often remain largely unexplained. In depth assessments with death certificate audits, medical record audits, and autopsies may yield more granular insights about deaths and their causes, but these approaches also have limitations and feasibility challenges. For most countries worldwide, the tremendous uncertainty in the sparse data and indirect inferences should be mostly a call for improving the completeness and accuracy of death registration and investment in more rigorous demography infrastructures in the future.
MATERIALS AND METHODS
Compared published excess mortality estimates
We considered four previously published pandemic excess mortality evaluations1–4, 18, 19 that have calculated both country-specific and global estimates and which use diverse methods to extrapolate from pre-pandemic reference periods to the pandemic period.
Karlinsky and Kobak published their evaluation in eLife in mid-2021;1 the Economist team released their estimates in late 2021;2 the COVID-19 Excess Mortality Collaborators published their estimates in the Lancet in early 2022 considering the two-year period 2020-2021;3 and WHO released in May 2022 its updated estimates covering the same two year period 2020-2021. We call these four evaluations for convenience eLife, Economist, Lancet, and WHO respectively. Both the eLife and Economist models allow updating of excess mortality estimates over time and we have used the Our World in Data resource18, 19 that includes such updates. We used the estimates of excess mortality for the 2020-2021 two-year period for all 4 evaluations to maximize comparability. The three evaluations used also different sources for capturing the recorded numbers of COVID-19 deaths, which resulted in mostly minor discrepancies. Again, to maximize comparability, we used the same set of recorded numbers of COVID-19 for comparingagainst the excess mortality estimates of each evaluation: this set is identical to the set used by the Lancet evaluation, with the exception of Spain and UK where the Lancet numbers of recorded deaths were too high by 10% and 16%, respectively (perhaps due to clerical error) and where we used the Johns Hopkins data instead.32
The reported cumulative counts of COVID-19 deaths are taken from the Johns Hopkins Repository as reported by Our World in Data. The same values are reported by two of the four methods we use: eLife and Economist. Lancet reported values in https://ghdx.healthdata.org/sites/default/files/record-attached-files/IHME_EM_COVID_19_2020_2021_DATA_Y2022M03D10.CSV are sometimes quite different. Specifically for Spain, United Kingdom, Kazakhstan, Mexico, Russia, Georgia & Tajikistan where discrepancies are over 10% and some times as large as 115% (Russia is 651,000 in Lancet and 302,671 in OWID).
For each of the 4 evaluations, we extracted information on the following methodological features: reference period selected; modeling of reference period (static average, linear, spline, Poisson seasonality, other); exclusion of heat waves, wars, natural disasters, other; unit of modeling data (week, month, quarter, other); pandemic time period covered in the original publication/release; source of data for all-cause mortality; source of data for COVID-19 deaths used in original analysis; age and/or gender adjustment in calculations of excess deaths (if yes, how); any other adjustment in the calculations (if yes, specify); eligibility criteria and number of countries modeled directly; eligibility criteria and number of countries inferred from the directly modeled countries; and how these were inferred. Details on the data sources and modeling methods for these 4 evaluations can be found in references 1-4, 18 and 19.
Evaluation considering age strata in the calculations
We performed also our own calculation of excess deaths focused on considering the impact of age-adjustment on the calculations. We considered three pre-pandemic years (2017 to 2019) as the reference period. We used the following age strata: 0-14, 15-64, 65-75, 75-85, and >85 years old. For each age stratum, we obtained the average mortality, the number of deaths per million for the population of the specific age stratum. Then we extrapolated to the two pandemic years, again correcting for the population size in the specific age stratum. Finally, expected deaths were summed across the population strata. An illustrative worked example of the age-adjusted calculations is shown in Figure 1 using the data for Germany. This analytical approach corrects for changes in total population of a country over time, as well as changes in the proportion of elderly people. If the total population and/or the proportion of older people is increasing in recent years over time, the expected deaths by the age-adjusted scheme will be more compared to analyses that do not consider population changes and age-stratification. The inverse will happen, if the total population and/or the proportion of older people is decreasing in recent years over time. We used the Human Mortality Database (https://www.mortality.org),33 specifically the Short Term Mortality Fluctuations file (https://www.mortality.org/Public/STMF/Outputs/stmf.csv) that includes weekly all-cause deaths and mortality values allowing the population size to be calculated as their ratio for each week.
Populations and total amounts of death on a single year are used to obtain the mortality values for the reference years (2017-2019) for each of the age strata. The average of this value is taken as the reference mortality for that strata. Expected deaths for a non COVID scenario for 2020 and 2021 are obtained from the population and reference mortality data for each strata. Excess deaths are calculated as the difference between the actual deaths and the expected deaths. Expected and excess deaths for the non age adjusted case are also reported for comparison. The table reports the real values for Germany as an example. This table is provided as Excel in the Supplement so that method can be easily used on other data.
Some clarifications are required for these analyses. First, on the calculation of the Weekly Population from the weekly Death Count and Weekly Mortality given in the MDH file stmf.csv. As Population = Death/Mortality, when Death = 0, Mortality = 0, so that Population is not defined. In these cases, we get the Population value from adjacent weeks with non-zero Death in the same year. Another problem is that some years, like 2015 & 2020 have an extra week 53, a leap week. This is because 52 weeks is 364 days whereas an average year is 365.25 days (a leap year every fourth year adds an extra day). Since mortality data are reported weekly, these years would get an apparent extra week of deaths. To correct this issue, we consider a standard year to be 365.25/7 = 52.1786 weeks and distribute weekly counts uniformly into years. Thus 2017 gets all its 52 week plus 0.1786 of week 1 of 2018. Year 2018 loses 0.1786 of week 1 and needs to get an additional 2*0.1786 of week 1 of 2019, and so on. In this way all the five years 2017 to 2021 are adjusted to have 52.1786 weeks.
We averaged the mortality values for each age-stratum over the reference years. We did not exclude any deaths or periods due to heat waves or other natural or man-made events, since it is subjective to arbitrate which ones should be excluded; moreover, we wanted to compare the COVID-19 pandemic with three recent years that had not raised any concerns about undue levels of deaths. We also repeated the excess death calculations using the same exact process but without considering age-strata.
We used population data, including also age-strata, from mortality.org.34, 35 for consistency across all evaluations in calculating excess deaths per million even though different evaluations had used different sources originally, e.g. WHO had used population data from the World Population Prospects 2019 with projections.36
Countries considered in main comparison
We focused our primary comparison on countries that have excellent death registration, are high income, and include data with weekly deaths in the Human Mortality database. These countries have the most reliable evidence to allow proper excess death calculations and changes in deaths over time cannot be due to changes in death registration (e.g. either improvements in death registration during the pandemic as countries put more resources on capturing information or worsening of death registration during the pandemic due to the pandemonium and acute death peaks). We defined high-income countries by World Bank criteria.37 The Short Term Mortality Fluctuations file in the Human Mortality database includes detailed weekly deaths for the period 2017-2019 and also for the pandemic years 2020-2021 on 35 countries, all of which except Bulgaria and Russia are high-income, thus 33 countries were included in the main analyses. Of note, Canada and Australia did not have data for the entire 2 years of 2020-2021 and we performed calculations that cover the available time periods.
Excess death metrics of interest
We calculated excess deaths for the full two years 2020-2021 for each country, E. We expressed them also as percentage above the expected deaths, Ep and as excess deaths per million Em. E.g. if 60,000 deaths happened in 2020-2021 in one country of 4,000,000 people and 50,000 were expected, the percentage EP is (60,000-50,000)/50,000=12% and Em is (60,000-50,000)/4,000,000=2,500 per million. As the main metric of interest, we used the ratio of estimated excess mortality E divided by the number of officially recorded COVID-19 deaths during 2020-2021, hence called the excess ratio R. E.g. R=1.20 means that the estimated excess mortality is 20% higher than the officially recorded COVID-19 deaths and R=0.90 means that the estimated excess mortality is 10% lower than the officially recorded COVID-19 deaths.
Analyses
We calculated the total number of excess deaths across the eligible countries with each of the different calculation methods, and as compared with the total recorded COVID-19 deaths. We also calculated the Pearson correlation coefficients of excess death metrics across the eligible countries based on the different calculation methods (a full list of correlation coefficient estimates appears in Supplementary Table 5).
We noted in how many of the 33 eligible countries the different calculations of excess deaths had consistently R above 1 or R below 1 (i.e. they all agreed that excess deaths were more than the recorded COVID-19 deaths or they all agreed that excess deaths were less than the recorded COVID-19 deaths); and in how many countries had R values that were all within a range of 0.3 between the highest and lowest estimate (a higher range means that divergence in the excess death estimates exceeds 30% of the recorded COVID-19 deaths); and within a range of 1 (a higher range means that the divergence in the excess death estimates exceeds the number of recorded COVID-19 deaths itself).
For the remaining, non-eligible countries, we evaluated excess death metrics and their comparison across different evaluations as exploratory analyses.
All analyses were done independently by two analysts (ML and FZ) and then compared notes with arbitration (including discussion with the third author, JPAI) for any disagreements until both analysts obtained the same results.
Data availability
All data are in the manuscript, tables, and supplementary tables and in the publicly available databases listed in Supplementary Links to Data
Data Availability
All data are in the manuscript and in publicly available datasets
Supplementary Links to Data
HMD
1-May-2022
https://www.mortality.org/Public/STMF/Outputs/stmf.csv
Use this for 33 set Populations, Total deaths & Excess deaths in age-bands
OWID for COVID-19 reported cases and excess death by eLife method. 22-Apr-2022
https://covid.ourworldindata.org/data/owid-covid-data.csv
Use this for eLife & Reported COVID-19 Cumulative count of Death to date of Completion
WPR for estimate Deaths 14-Apr-2022
https://worldpopulationreview.com/country-rankings/death-rate-by-country Use rateGHDE?
Expected death is 2 years (rateGHDE X pop2021) csvData.csv
https://worldpopulationreview.com/countries Use pop2021 in column [D]
csvData.csv
World Bank 14-Apr-2022
https://data.worldbank.org/indicator/SP.DYN.CDRT.IN?end=2017&start=2017&view=bar
API_SP.DYN.CDRT.IN_DS2_en_csv_v2_3932073.csv
Lancet
5-Apr-2022
Use for Lancet Reported deaths and Excess Death for 2020 &; 2021
Economist 3-Apr-2022
https://ourworldindata.org/excess-mortality-covid#estimated-excess-mortality-from-the-economist
excess-deaths-cumulative-economist-single-entity.csv
Footnotes
Author Contributions: M.L. and J.P.A.I. had the original idea. M.L. analyzed the data with contributions from F.Z. and J.P.A.I. J.P.A.I. and M.L. wrote the paper, and all three authors interpreted the data, edited the paper, and approved the final version.
Competing Interest Statement: No conflicts of interest. Classification: Biological Sciences: Medical Sciences
Funding: NIH R35 GM122543
Data statement: All data are in the manuscript and in publicly available datasets