Radiosensitivity Index (RSI) is not fit to be used for dose-adjustments: a pan-cancer analysis

Radiotherapy has been striving to find markers of radiotherapy sensitivity for decades. In recent years the community has spent significant resources on exploring the wide range of omics data-sets to find that elusive perfect biomarker. One such candidate termed the Radiosensitivity Index, RSI for short, has been heavily publicized as a marker suitable for making dose-adjustments in the clinical setting. However, none of the analyses conducted, thus far, has assessed whether RSI explains enough of the outcome variance to elucidate a dose-response empirically. Here we re-analyze a pan-cancer data-set and find that RSI is no better than random chance at explaining outcome variance, overall survival times. For completeness, we then assessed whether RSI captured a sufficient amount of outcome variance to elucidate a dose-response, it did not. These results suggest that like the initial in-vitro analysis 12 years previously RSI is not a marker of radiotherapy sensitivity and is thus not fit to be used in any dose-adjustment algorithms.


Introduction
The radiotherapy community has for decades been striving for an easy-to-use assay to measure the radiosensitivity of an individual patient's tumour. Initially these approaches centered around developing short-term cultures (1) and lately the community has moved towards using omics data (2). The most prominent candidate biomarker over recent years, by virtue of the publicity given to it by its authors and not by the quality of evidence, is a marker termed "Radiosensitivity Index", "RSI" for short (3).
In a previous publication we highlighted that in the original study where "RSI" was developed its performance in the referenced test-set was worse than random chance (4). That is, within an in-vitro assay "RSI" was unable to explain any of the variance in response seen across a panel of cell-lines within its own developmental test-set. Note, this issue has been raised before (5) but ignored by the developers of "RSI".
Within this article we shall re-evaluate the latest data-set, released by Scott et al. (6), and assess if "RSI" shows any promise compared to its original evaluation 12 years ago. The goal of the analysis herein is to assess if "RSI" explains enough of the variance in a response variable (e.g., overall survival [OS]) such that it elucidates a dose-response. What do we mean by this statement?
Let's return to the preclinical setting to expand on this point. Let's assume we have 100 cell-lines, either patient-derived or from a cell-bank, and that each of these cell-lines produces a different doseresponse within say an in-vitro radiosensitivity assay, see Figure 1. Note, we used a linear-quadratic model and simply allowed alpha to vary and kept beta fixed for simplicity. If we had a perfect biomarker that fully explained the variance of the dose-response curves, once we account for that biomarker, all the dose-response curves would collapse onto one single dose-response curve. The biomarker would provide a link between the variance in response and dose that makes the doseresponse look clearer. Thus, we could now use such a biomarker to dose-adjust with high degree of confidence of the resultant effect. If the biomarker was that good, we would only need a small variation in dose with a sufficient sample size to see part of the curve clearly. However, if the biomarker explains little of the variance, i.e., does no better than random chance, then a doseresponse will never be seen regardless of the variation in dose. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 23, 2021. ; https://doi.org/10.1101/2021.08.13.21262017 doi: medRxiv preprint Figure 1: Plots showing how a biomarker that can perfectly predict each cell-lines alpha (concordance = 1) can collapse all the dose-response profiles to a single one (panel A) and a biomarker which is no better than random chance (concordance ~0.5) cannot (panel B).
In reality the variance in a clinical end-point such as OS or any other time-to-event end-point will never be fully explained by any single biomarker. The question is whether enough of the variance can be explained such that a dose-response can be seen in the data. If we believe that we will never see perfect correlations (such as top-panel, Figure 1), how much of a correlation do we need to see given the dose-ranges used to see a dose-response? What other factors, which explain more of the variance, would we need to adjust for before we see a dose-response? It may be that a dose-response may never be seen. Note in pharmacology careful consideration has to be given for numerous sources of heterogeneity to have a chance of seeing a dose/exposure-response in efficacy and even then there is no guarantee (7,8).
Now with this concept in mind let's re-analyze the data from the latest publication on "RSI" (6). The response variable of interest here will be OS. Local progression will not be considered as it's an interval censored event and it's likely the time intervals between assessments for patients was highly inconsistent and not recorded to a high standard given that the data is not from clinical trials.

Methods
"RSI" values, OS times (calculated from treatment initiation until death) and total radiation dose administered for patients who were treated with radiotherapy were taken from Scott et al. (6) (We refer the reader to that paper for further details on each cohort.) The triple negative breast cohort from Scott et al. (6) was not used as it only contained 9 events and such a low number of events can lead to very large uncertainty in estimates of the survival probability without any predictors, see Figure  2. Any estimates from such a study are likely to be highly uncertain. Data on the melanoma study, reported in the original analysis (6), was not in the data-set provided at https://github.com/gsedor/GARD_Meta-Analysis at the time of writing and so that cohort was not considered either. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 23, 2021. ; https://doi.org/10.1101/2021.08.13.21262017 doi: medRxiv preprint For each tumour type, we first fitted a Cox proportional hazards (PH) model with "RSI" as the sole predictor and reported the concordance index (c-index) with standard errors. Note a c-index value of 0.5 suggests that the predictor is no better than random chance whereas a value of 1 implies perfect concordance, recall Figure 1. Next, we considered a Cox PH model with both "RSI" and (total radiotherapy) dose as predictors. In this second model our interest was in whether dose becomes a predictor of OS after adjusting for "RSI". If "RSI" explains enough of the variance in OS, then we expect to see good precision on the hazard ratio for dose. In this second analysis we shall report hazard ratios with 95 percent confidence intervals and p-values from the likelihood ratio-test assessing the fit between a model with "RSI" alone and a model with "RSI" and dose.
For both sets of analyses, we considered each individual tumour site and a pooled analysis using tumour site as a stratification factor, i.e., the baseline hazard is different for each tumour site. That analysis assumes though there is no interaction between tumour site and "RSI" and this may not be true. Therefore, we also assessed the no-interaction assumption by comparing a model with/without an interaction between tumor site and "RSI", see Chapter 5 in Kleinbaum and Klein (9). Note that the no interaction assumption was not assessed in the original article by Scott et al. for any of their pooled analyses (6)) All analyses were performed in R v4.1.0. The code that generates the results and data can be found here: https://github.com/mcbi9hm2/ClinicalRSICritique1. Table 1 shows that "RSI" as a univariable predictor shows very poor discrimination, with low c-index values across all cohorts and in the pooled analysis. The interaction between tumor site and "RSI" was not needed, p=0.455. "RSI" does not appear to be better than random chance in any cohort and so explains none of the survival variance, i.e., we are in the bottom panel of Figure 1. Upon inclusion of dose, we see that RSI has not explained sufficient variance in OS to elucidate a dose-response. This second result is not surprising given RSI failed to explain any of the survival variance in a univariable analysis. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

Results
The copyright holder for this this version posted August 23, 2021. ;  For completeness, the distribution of total radiotherapy doses is shown in Figure 3. Had "RSI" explained any of the outcome variance then a discussion around whether there was sufficient variance in dose and what causes the dose variance across these cohorts to elucidate a dose-response could have been had. However, this is not an issue if "RSI" explains none of the outcome variance.

Discussion
Generating evidence that a biomarker captures enough of the response variance (e.g., overall survival times) such that it can be used to make dose-adjustments is a hard task. In this short report we have first highlighted what it means to fully capture the dose-response with a biomarker and subsequently re-assessed a proposed biomarker for use within a dose-adjustment algorithm in radiotherapy.
We have previously shown that the biomarker termed "Radiosensitivity Index" ("RSI") was no better than random chance in explaining the variance within an in-vitro assay from the original publication . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 23, 2021. ; (4). Here we find "RSI" is no better than random chance at explaining overall survival variance across numerous cancer types i.e. the results mimic the preclinical findings. Furthermore, since it does not explain any of the survival variance it also fails to elucidate a dose-response in any tumour type. These results together with the re-analysis of the preclinical data highlight that "RSI" is not a marker of radiosensitivity, should not be referred to as such and should not be used within dose-adjustment algorithms based on the current evidence.
It could be argued that one will never be able to personalize the radiotherapy dose for the following reasons. The variance in survival times is likely made up of numerous factors, ranging from type of combination therapy given with RT (which has its own predictors of response e.g., genomic factors, levels of drug in plasma etc.) but also standard clinical variables around patient fitness, follow-up care etc. and more. A breakdown how each variable contributes to survival times would be needed, together with how much can't be explained, and then an assessment of how much of the variance due to radiotherapy can be explained by a candidate biomarker is needed. It may be more pertinent for the community to consider each disease in its own right, assess all available factors and decide whether they are currently explaining enough of the outcome variance to then pursue personalization of dose. It may be a personalization of both radiotherapy and systemic therapy doses.

Summary
This article highlights that "RSI" is no better than random chance at explaining the survival variance across numerous cancers. That is, "RSI" is not a marker of radiosensitivity either preclinically or clinically and should not be referred to as such. "RSI" is thus not fit to be used within dose-adjustment algorithms as is being proposed (6). Based on current evidence further research into "RSI" and doseadjustment algorithms using "RSI" is not warranted and the community should move on.