Understanding the build-up of immunity with successive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants and the epidemiological conditions that favor rapidly expanding epidemics will help facilitate future pandemic control. We analyzed high-resolution infection and serology data from two longitudinal household cohorts in South Africa to reveal high cumulative infection rates and durable cross-protective immunity conferred by prior infection in the pre-Omicron era. Building on the history of past exposures to different SARS-CoV-2 variants and vaccination in the cohort most representative of South Africa's high urbanization rate, we used mathematical models to explore the fitness advantage of the Omicron variant and its epidemic trajectory. Modeling suggests that the Omicron wave likely infected a large fraction (44 to 81%) of the population, leaving a complex landscape of population immunity primed and boosted with antigenically distinct variants. We project that future SARS-CoV-2 resurgences are likely under a range of scenarios of viral characteristics, population contacts, and residual cross-protection.