Simulation-based validation of a method to detect changes in SARS-CoV-2 reinfection risk

Background Given the high global seroprevalence of SARS-CoV-2, understanding the risk of reinfection becomes increasingly important. Models developed to track trends in reinfection risk should be robust against possible biases arising from imperfect data observation processes.

Objectives We performed simulation-based validation of an existing catalytic model designed to detect changes in the risk of reinfection by SARS-CoV-2.

Methods The catalytic model assumes the risk of reinfection is proportional to observed infections. Validation involved using simulated primary infections, consistent with the number of observed infections in South Africa. We then simulated reinfection datasets that incorporated different processes that may bias inference, including imperfect observation and mortality, to assess the performance of the catalytic model. A Bayesian approach was used to fit the model to simulated data, assuming a negative binomial distribution around the expected number of reinfections, and model projections were compared to the simulated data generated using different magnitudes of change in reinfection risk. We assessed the approach’s ability to accurately detect changes in reinfection risk when included in the simulations, as well as the occurrence of false positives when reinfection risk remained constant.

Key Findings The model parameters converged in most scenarios leading to model outputs aligning with anticipated outcomes. The model successfully detected changes in the risk of reinfection when such a change was introduced to the data. Low observation probabilities (10%) of both primary- and re-infections resulted in low numbers of observed cases from the simulated data and poor convergence.

Limitations The model’s performance was assessed on simulated data representative of the South African SARS-CoV-2 epidemic, reflecting its timing of waves and outbreak magnitude. Model performance under similar scenarios may be different in settings with smaller epidemics (and therefore smaller numbers of reinfections).

Conclusions Ensuring model parameter convergence is essential to avoid false-positive detection of shifts in reinfection risk. While the model is robust in most scenarios of imperfect observation and mortality, further simulation-based validation for regions experiencing smaller outbreaks is recommended. Caution must be exercised in directly extrapolating results across different epidemiological contexts without additional validation efforts.