A Net Benefit Approach for the Optimal Allocation of a COVID-19 Vaccine

Background

Coronavirus disease 2019 (COVID-19) is a severe, novel virus that has spread globally. On 30 January 2020, the World Health Organization (WHO) declared a Public Health Emergency of International Concern and declared a Pandemic on 11 March 2020 [1]. As of 27 November 2020, there have been over 60 million confirmed cases and over 1.4 million deaths worldwide. In Alberta, Canada, the first presumptive COVID-19 case was identified on 5 March 2020; as of 27 November, there have been over 51 thousand confirmed cases and 510 deaths in the province [2].

Key public health interventions for COVID-19 mitigation include hand hygiene, mask wearing, physical distancing in various locations, travel restrictions, testing, contact tracing, isolation of cases, and quarantine of close contacts. Public health policies and individual decisions combine to produce changes in human behaviours, including reductions in contact. The implementation of a combination of public health interventions is required to reduce viral spread and avoid overwhelming acute care systems. Decision makers are challenged with choosing the optimal combination of interventions to mitigate public health risks of COVID-19 while meeting other population needs including access to the health system for diagnosis and treatment of other conditions, economic activity, and social well-being. Once available, additional effective interventions such as vaccination and antiviral treatment will further mitigate the impact of the COVID-19 pandemic.

Over 200 COVID-19 vaccination products are currently under development, several of which have reached the phase III trial stage [3]. These vaccines may reduce disease burden by preventing infection in exposed individuals, reducing severity of infection in infected individuals, and/or by preventing secondary infections. Internationally, decision makers will need to prioritise access to limited vaccine stockpiles [4-6]. Many countries have produced decision frameworks [7], and researchers have developed commentaries on ethical, economic, and epidemiologic justifications for different allocation objectives and strategies [8-14].

Mathematical models with health economic outputs can assist in public health decision-making by assessing outcomes associated with various public health interventions. These models can simulate a range of vaccine allocation scenarios, and demonstrate outcome dynamics with changing vaccine characteristics, allocation, public policy, and population behaviours. Hundreds of models have been developed internationally to support decision making and public health messaging. A handful of studies evaluating vaccine allocation strategies have been published to preprint servers [13, 15-17]. Only one of the studies evaluates scenarios with concurrent public health mitigation strategies [13]. None of the studies consider interventions targeting individuals by both age and risk group, with well-defined vaccine allocation policies or health economic outputs, limiting their value to decision makers.

Aim and Objective

The aim of this study is to identify optimal vaccine allocation strategies defined by age and risk target groups, coverage, effectiveness, and cost of vaccine, within a dynamic context where other public health responses and population behaviour change. In this study we demonstrate the use an epidemiological model of COVID-19 that has been enhanced to produce expected cost and Quality Adjusted Life Year (QALY) outcomes. Model outputs include total cases, hospitalisations, deaths, and net monetary benefit (NMB) [18]. We use the model to simulate hypothetical scenarios where vaccine is allocated beginning on October 15, 2020 with vaccine characteristic scenarios ranging from moderately optimistic to ‘worst-case scenario’, and NMB is used as the objective for optimisation. This range of scenarios is designed to highlight key considerations and demonstrate outcome trade-offs.

Model Structure

CRAM is stratified into 16 age groups (five-year ranges, from 0 to 75 years). Each of these 16 groups is divided further into two risk groups (high-risk and not high-risk), by risk of poor outcome due to COVID-19. An additional group is used to represent the Long-Term Care (LTC) population, who are all defined as high risk individuals aged 75 years and older. This results in 34 age-risk groups. CRAM is based on the structure of an existing model developed for the province of Alberta [15]. CRAM has five overarching compartments: Susceptible (which includes Susceptible, No Distancing, Vaccine, and Vaccine Failure), Exposed, Infected (composed of Pre-Symptomatic, LTC, Community, Hospital, and Pre-Isolation), Isolation, and Recovered (comprised of Immunised, Recovered, and Dead) (see Figure 1). The model structure was developed to produce outcomes of interest to decision-makers and to leverage the highest quality data available in real-time during the COVID-19 pandemic.

• Download figure
• Open in new tab

Figure 1.

CRAM Structure Note: Figure intended for colour printing.

A fraction of the population is initially allocated to the Exposed compartment, with the remaining population allocated to the Susceptible compartment. Susceptible individuals are vaccinated based on program start date, vaccine prioritisation policy, number of doses remaining, and the daily maximum number of vaccines possible. Vaccine prioritisation model scenarios provide vaccine access to different age and risk groups on different dates. Once vaccinated, after the period required to develop antibodies, individuals move either to the Immunised compartment, or to the Vaccine Failure compartment (and remain susceptible) as a function of the vaccine effectiveness. If the number of individuals who are eligible for vaccine on a given day exceeds the maximum vaccine capacity, the available doses are prorated across eligible age and risk groups by group size.

Susceptible individuals are exposed to the virus as a function of the force of infection which is determined by the transmissibility of the virus, seasonality, number of persons infected, and rate of contact between individuals across age groups. Susceptible individuals exposed to the virus move to the Exposed compartment and following the latent period they move from the Exposed to Infected.

Depending on public health guidance, some individuals may move from the Susceptible compartment to the Susceptible, No Distancing compartment, which can be used to separate segments of the population following different distancing measures. It can be used, for example, to simulate in-person schooling. Children who are not high risk move to the Susceptible, No Distancing compartment with higher rates of contact due to in-person education.

Upon arrival in the Infected compartment, individuals are Pre-Symptomatic, defined as cases in the community that are shedding virus but not yet experiencing or displaying symptoms. Once the pre-symptomatic period has elapsed, individuals are allocated between the Community and Hospital compartments, dependent on their age and risk group. If a case isolation policy is in place, a subset of the Community compartment will move to the Pre-Isolation and subsequently the Isolation compartments, where they remain for the duration of the remaining infectious period. All cases contribute to the force of infection prior to moving to the Isolation compartment, but individuals in the Isolation compartment do not contribute to the force of infection. In CRAM the Isolation compartment represents the isolation of cases but not the isolation of close contacts.

Entry into LTC and Hospital varies by age group, with high risk individuals over the age of 75 moving from the pre-symptomatic to the LTC compartment. Individuals move from the LTC compartment at the end of the infectious period to either the Recovered or Dead compartments in proportions dependent upon their risk group. Entry into the Hospital compartment is a function of age and risk, fit by an exponential regression on admissions by age, and adjusted for risk by an odds ratio. The Hospital compartment includes both general inpatient cases, and those admitted to intensive care units (ICUs). At the end either the infectious period or the hospital length of stay (LOS), individuals move from the Community, LTC or Hospital compartment to either the Recovered, or Dead compartment, at a proportion dependent upon their risk group.

We used a synthetic contact matrix disaggregated into four locations: home, workplace, school, and other, where other comprises all remaining possible locations [19]. Disaggregating the total contact matrix by location allows for adjustments in contact rates at locations to align with policy interventions. For example, we can apply work-based interventions to specific age groups by adjusting the work contact matrix at various points in time. The total contact matrix is generated by summing the contacts for each age group across each of the matrices. Contact reductions for seniors aged 75 years and older and for high-risk individuals are applied directly to the total contact matrix. Complete details of model equations and assumptions are available in Appendix 1.

Epidemiologic Inputs

CRAM was parameterised wherever possible with data from Alberta, including estimates for the total population by age group and risk of severe outcome, hospitalisation rate by age group, and LOS for hospitalised cases. Odds ratios for the risk of hospitalisation were estimated by risk group using logistic regression. Active cases in the model include both diagnosed and undiagnosed cases, fitted using seroprevalence data from 20 May 2020 provided by Alberta Health. Details on all model inputs are provided in Table 1.

Data from the Google Community Mobility report [20] were used as key inputs for contact reductions in workplaces and other locations. A rapid expert elicitation exercise was conducted to obtain estimates for unknown model parameters. The initial rapid elicitation activity surveyed experts in Alberta on reductions in contacts for seniors aged 75 years and older, high-risk individuals, and reduction in contacts in home locations. A second rapid elicitation exercise determined likely return-to-school rates and in-school contact reductions by age group. We adapted elicitation methods and expert training materials from the SHELF (Sheffield ELicitation Framework) tool [21], the results of which are included in Appendix 2.

Economic Evaluation

The study is conducted from a health system payer perspective. Outcomes are tabulated from February 5 ^th, 2020 to January 1^st, 2021, spanning 332 days. Health system costs include hospitalisation, incremental LTC costs for infection prevention measures, and the cost of purchasing and administering a vaccine. Health utility losses include disutility from infection, hospitalisations and LTC infections (during stay and following discharge), and deaths. We assessed economic outcomes using the NMB framework [18]. We apply the health opportunity costs estimated for Canada [25], using the value of $30,000 per QALY that is recommended for all Canadian provinces. All costs are converted to 2020 Canadian dollars, and discounted at an annual rate of 1.5%, according to CADTH guidelines [26].

As no health utility estimates are currently available for COVID-19 disease, we identified utility impacts of similar conditions. We estimated disutility for community COVID-19 infections from an EQ-5D survey of individuals with lab confirmed influenza B infections, stratified by age group [27]. Because hospitalised COVID-19 cases are often diagnosed with pneumonia, we used disutility estimates for viral pneumonia inpatients [28] to estimate in-facility treatment losses, as well as utility losses for the year following facility discharge. We estimated utility losses from premature death using the life table approach developed by Briggs et al. [29], adjusting for baseline utility, increased relative mortality rates, and discounted. All disutility values are converted to QALYs. Costs related to community infections are not included. We estimated incremental direct costs associated with COVID-19 hospitalisation and ICU admission treatment based on published Alberta per-diem costs, as well as the incremental cost of infection prevention and control for LTC cases. Cost and disutility estimates are presented as incremental values in Table 2. These incremental costs and disutility values are applied to the relevant compartments, and the resulting NBM is estimated for each scenario.

Fitting

We fit five parameters: virus transmissibility, the probability of hospitalisation, infections in LTC, and the probability of death from either facility or community infections. Using a deterministic version of the model defined with mean values of each input parameter we fit observed surveillance data for daily hospital census counts, as well as cumulative daily death and LTC counts from 6 April to 13 August. Given the interdependence in the fitting parameters, we used simultaneous fitting methods by optimizing model fit over a Latin hypercube generated from plausible values for each parameter. Optimal parameter values are identified when the sum of the normalised difference in model and observed outcomes are minimised.

Outputs

The CRAM sampled from the probability distributions of model inputs, as either 10% of results from the model fitting, or as estimated from the parameter source (Tables 1 & 2), to account for outcome uncertainty over 200 iterations, and then generated model outputs from the sampled values. Outputs include cumulative estimates for the mean number of active cases, total infections, and hospitalisations, with credibility intervals to present results from all simulations.

The economic outputs include expected NMB, a summary statistic calculated as (benefit * cost-effectiveness threshold)-cost, so that comparisons without the use of ratios can be made [18]. For each of the vaccine scenarios, incremental net benefit (iNB) is estimated relative to the baseline with no vaccine.

Scenarios

In the baseline scenario (‘No Vaccine Baseline’), no vaccine is available, and all contact matrixes and time variant inputs are held constant at the last observed value from model fitting. Specifically, we maintained contact reductions at the values determined through the expert elicitation for ‘home’ locations, as well as for seniors and high-risk individuals, and a fall return to school. Moreover, we kept the most recent observation of reduction in time spent in workplaces and ‘other’ locations from the Google Community Mobility reports consistent over the model run. In all scenarios, the vaccine is ‘sterilizing’, meaning that the vaccine protects individuals from infection.

Vaccine scenarios represent different vaccine allocation strategies, as well as various vaccine characteristics (i.e., effectiveness, coverage), as described in Table 3. Vaccine scenarios are characterised by the maximum number of doses that can be delivered per day, the start date of the vaccine program, the definition of the priority group, the interval (if any) between access for the priority group and the general population, the effectiveness of the vaccine, and vaccine coverage as a proportion of the total population.

If the number of individuals eligible for immunisation exceeded the maximum number of doses that could be delivered on a given day, the maximum number of doses were pro-rated across each eligible age and risk group daily. This means that a group representing 5% of the eligible population would receive 5% of the total number of doses available on a given day. Each of the scenarios begins on October 15, 2020, before the anticipated distribution of a vaccine in Canada.

To examine the impact of variations in vaccine effectiveness, we included a scenario where effectiveness varies by age, so that the vaccine is 60% effective over the age of 60, and 80% effective otherwise. In all other scenarios, effectiveness is constant across age groups. The scenario with 80% effectiveness approximates preliminary results from three phase three trials which have reported efficacy ranging from approximately 70 to 90% [33].

In each of the scenarios, we assumed that the vaccine cost per dose inclusive of delivery is $40, and there was a 30-day interval between access for different priority groups, starting on October 15, 2020. Extrapolating from the maximum estimate of vaccines per day in Alberta from [34], we assumed that up to 80,000 individuals can be immunised daily. Each of the comparison scenarios contains the ‘Vaccine Baseline’ scenario with age and risk-based prioritisation, 80% effectiveness, and 40% coverage, as a common reference point across scenarios. Scenarios that prioritise based on age give first access to those aged 55 and over, while programs that prioritise high-risk individuals prioritise all high-risk individuals irrespective of age.

We also defined a worst-case scenario with low vaccine effectiveness (40%) and coverage (20%). Under this worst-case scenario we modelled additional public health measures, including a 50% reduction in contacts for seniors, and complete school closures (each policy implemented on November 1, 2020). These simulations allow model outcomes to compare vaccine policies in combination with other mitigation strategies.

Coverage, Effectiveness, and Prioritisation Scenario Results

Worst-Case Scenarios: Concurrent Public Health Responses

Comparisons to Other Studies

While we are not aware of any other studies examining vaccine allocation within a health economic perspective, a handful of preprint studies have used analytic methods to test outcomes under different vaccine allocation scenarios [13, 15-17]. The results from the CRAM align with the other studies: the age-prioritised scenario minimises hospitalisations, while the unrestricted scenario minimises total infections (Table 4). Like [17], who project that coverage between 40-59% of the population will be required for suppression in Canada, the CRAM demonstrates that infections are significantly reduced with coverage of 70%, when age and risk prioritisation is implemented.

Several elements of CRAM compare favourably to the other studies. With the exception of Chen et al. [13], other studies do not consider concurrent or future public health policies/ and or behavioural factors. This appears to be a large oversight given that any allocation decision must not only consider the epidemiology at the time of allocation, but also which other interventions might be used to amplify the impact of targeted vaccine strategies.

CRAM allows allocation to both age and risk groups, which are not evaluated together in many other studies [13, 15-17]. It is likely that governments would wish to consider allocation to high risk individuals rather than broad age groups, and high-risk individuals are likely to take greater infection prevention measures, so these dynamics are important to incorporate. Furthermore, most studies are calibrated to published R0 figures. Because the CRAM is fit to surveillance data, model predictions for a broader range of outcomes (infections, hospitalisations, deaths) are provided with greater accuracy. Finally, model inputs to the CRAM were informed wherever possible with jurisdictional data, and expert opinion was used rather than using assumed values, as in the studies discussed.

There are also elements where other approaches are strong relative to CRAM. Two studies [12, 15] consider vaccines which are both ‘leaky’ (reducing the probability a susceptible individual is infected) and ‘all-or-none’ (as in the CRAM). Meehan et al. [17] evaluate vaccines which either prevent infection, or prevent disease. Bubar et al. [15] compare scenarios where serological testing is included in vaccine scenarios, or not. While inclusion and evaluation of these elements is beyond the scope of this paper, it is reasonable to assume that the relative dynamics demonstrated by other authors (for example relative outcomes under leaky or all-or-none vaccine assumptions) will hold for the results of the CRAM.

Strengths

Our approach has several strengths relative to other modelling studies. The direct interpretation of the NMB from a health system payer perspective provides important insights. Generally, all measures that can reduce the spread of COVID-19 have a large social value, due to the very high economic losses it causes. [35] A general expectation is that economic and social outcomes will be relatively worse under more restrictive public health responses (such as business and school closures), while direct health outcomes will improve. The inclusion of broader health outcomes is undetermined because outcomes like excess mortality and morbidity, due to health system avoidance or deferral of services will have dynamic impacts on NMB under various public health responses. Interventions such as vaccine are expected to increase NMB from a societal perspective, limited only by the cost of vaccine procurement and delivery. By estimating the INB from a payer perspective for each strategy, the minimum NMB of economic and social impacts to indicate a different policy direction are made explicit.

The structure of CRAM was developed to align with available Alberta surveillance data, namely the number of hospitalisations and cumulative infections on key dates indicated by seroprevalence studies. While case classification in each healthcare setting provides unique challenges and remains subject to ascertainment and reporting biases, these measurements can be estimated with the greatest amount of certainty, and therefore both the fitting and compartmental structure of CRAM are developed around these data, mitigating these issues to the greatest extent possible. Laboratory-confirmed cases and asymptomatic infections are not included within the model structure due to data quality. Laboratory testing protocols have changed rapidly since March 2020 [36], and therefore incorporating the number of laboratory-confirmed cases into the model structure would have been inappropriate.

An important feature of CRAM structure is that it allows users to evaluate a broad and flexible range of concurrent interventions. Our results demonstrate that optimal vaccine allocation changes depending on the public health measures (like school closures) or behavioural changes (contact reductions for seniors) as well as vaccine program features. By creating a structure that captures relative changes in behaviours and policies by age and risk group and by location over time, our model is designed to capture dynamic population changes to the greatest degree possible.

Limitations

A key challenge for all COVID-19 modelling and simulation studies has been data quality. The challenge is two-fold: model inputs and parameters (such as the duration of infection and transmissibility by age group) are uncertain and estimated from small observational studies. In addition, jurisdictional observational data are subject to bias. The high degree of uncertainty in many of the model elements necessitates constant review and reappraisal of the model structure and inputs. Constant evolution of the evidence also highlights the importance of the stochastic approach used by this study. By sampling from the distributions of several parameters, the stochastic results quantify the resulting uncertainty in model outputs.

Seasonal dynamics related to COVID-19 remain controversial. Several studies have found empirical and modelling evidence to support changes in virus transmission consistent with seasonal behaviour, and these seasonal changes in transmission can be driven by both behavioural and environmental factors. A review on the stability of coronaviruses including COVID-19 under various environmental and climatic conditions [37] found early evidence of virus sensitivity to temperature, light, and humidity in laboratory studies [38-42]. Many studies analysing the natural history of the virus imply the presence of seasonal dynamics [43-50], while others found a limited effect [51, 52]. Natural history studies are challenged with data quality issues, confounding, and a very short period of observation [53]. There is overall agreement that the transmission of COVID-19 is likely to demonstrate some seasonal behaviour, but that dynamics are complex due to the high susceptibility of the global population and that effective pandemic control cannot rely on seasonal dynamics. As demonstrated by the findings of a rapid literature review [54], the evidence around relative differences in susceptibility and transmission by age is highly uncertain, in particular for children. None of the studies identified through the rapid review included parameters suitable for inputting into CRAM. A modelling study estimating relative susceptibility by age group was instead used as a source for these inputs [23]. Like any of the inputs, if evidence were to emerge that the dynamics of susceptibility by age are different from those used to fit our model, there would be implications for the validity of these results.

Due to the widespread impacts of COVID-19 on population health, economy, and social outcomes, a societal perspective is likely to be the most appropriate perspective for economic evaluations of COVID-19 response policies. The estimates from a health system payer perspective generated in this study do not capture the full health, social, and economic impacts of the COVID-19 pandemic and response. Accurate estimates of the additional societal impacts are unlikely to become available in the near future. In this context, our analysis provides an important baseline estimate of the cost-effectiveness of vaccines which future evaluation can build upon. As additional research estimates the broader economic impact of different public health policies (for example, business or school closures) these may be layered on to estimates from the CRAM, the flexible structure of which allows the estimation NMB for a broad range of strategies.

Community infection comprises multiple disease states, which are not separated due to data quality. There is no reliable estimate of the number of infected individuals in the community for Alberta, as current testing protocols and changes to testing eligibility undermine the estimation of the number of infected individuals over time. Similarly, evidence on the proportion of infected cases which are asymptomatic is rapidly evolving [55]. Due to the absence of these inputs, community cases are not differentiated by the presence of symptoms. Related to this is the exclusion of costs related to community infections, which could be included in future work. We did not consider non-sterilizing vaccines, which would require use of an asymptomatic compartment. Finally, CRAM structure does not allow for the isolation of close contacts, limiting the range of isolation and quarantine scenarios that can be evaluated.

Policy Implications

The results of this paper have three key policy implications: (i) that optimal vaccine allocation will depend on the public health policies, and human behaviours in place at the time of allocation; (ii) the outcomes of vaccine allocation policies can be greatly supported with interventions targeting contact reduction in critical sub-populations; and (iii) the identification of the optimal strategy depends on which outcomes are prioritised.

In allocating vaccine access, decision makers must not only consider the interventions and behaviours in place at the time of allocation, but also those which could be in place in the future. Optimal allocation will depend on public health policy and human behaviour, and therefore decision makers must understand that allocation decisions are made within a dynamic environment. The value of vaccination may be improved by adjustments to complementary public health policies.

Decision makers should define the range of public health policies that can be implemented alongside vaccine allocation within an ethical framework. For example, in our model NMB will be maximised if seniors continue to reduce their contacts through the fall, and vaccines were allocated without prioritizing older and high-risk population groups. This would mean that the people with the greatest risk of disease and poor outcomes, would also bear the highest burden of additional social isolation. Because of this, the suite of public health policy options should be determined prior to dynamic policy optimisation, to ensure that the optimal strategy is achieved within a framework reflective of social values and equity [8].

Decision makers must evaluate a broad range of outcome measures to prioritise and provide robust justification for their choice. This paper uses NMB because it captures the relative disutility for epidemiologic outcomes (total cases, hospitalisations, and deaths), as well as health system resource implications due to illness. In this way, multiple factors such as age and outcome severity are built into the measure. NMB will also allow for simple and direct inclusion of societal outcomes, if and when they become available.

Future Research

Ongoing research and observational data should continue to be incorporated into the model, through re-fitting data, and adjusting model structure where evidence indicates important new relationships and dynamics. The results of this paper indicate that outcomes are highly responsive to behavioural factors such as changes in contact rates. This indicates that while the results of this paper provide important insights into the general allocation problem, model reanalysis with the most up to date available information is critical to optimal strategy identification.

While the results presented in this study pertain to a model calibrated to Alberta, Canada, the key findings are relevant to many other jurisdictions. The approach demonstrates how the net benefit approach, implemented through mathematical modelling, can support a principled and defensible vaccine allocation within a dynamic policy environment.