Sample size estimates for biomarker-based outcome measures in clinical trials in autosomal dominant Alzheimer’s disease

Participants

All data came from participants enrolled in DIAN-OBS - a worldwide, multi-modal study of ADAD mutation carriers and non-carrier family members[13], who serve as a valuable environmentally similar control group, enabling characterisation of the divergence of disease-related changes from normal aging. The DIAN-OBS study was reviewed and approved by the appropriate Institutioanl Review Boards and research ethics committee for each participating site. Informed consent was obstained from all participants.

DIAN-OBS was designed to parallel clinical trials: integrating rigorously collected, longitudinal data across multiple centres and including a wide array of imaging, fluid biomarker, and clinical measures. Indeed, the clinical trial DIAN-TU-001 (ClinicalTrials.gov Identifier: NCT04623242, NCT01760005) included many individuals from DIAN-OBS, allowing for the potential of a run-in phase as part of their design[12,14]. Detailed information concerning the DIAN-OBS study protocol, including MRI and PET image acquisition, has been reported previously[15].

Participants in DIAN-OBS are from families known to carry a pathological mutation in PSEN1, PSEN2, or APP genes. Data were taken from the 14^th semi-annual data freeze (2020), which included cognitive, biomarker and imaging data from 534 participants, 372 of whom have longitudinal data. Since age at symptomatic onset is relatively consistent within ADAD families [7] and most mutation carriers in DIAN-OBS are presymptomatic at enrolment, estimated years to expected symptom onset (EYO) can be calculated for individual participants[7] by subtracting the age their affected parent first developed symptoms from the participant’s age at their visit. In this way, EYO can be used to determine eligibility for trials amongst presymptomatic mutation carriers. While EYO is not clinically relevant for the non-carriers, estimating it helps ensure that the non-carrier group is demographically similar to the carrier group and helps to account for any age-related changes.

From this data freeze, 244 of the 372 participants with longitudinal data had a visit that would satisfy the basic eligibility criteria of DIAN-TU-001 and at least one subsequent follow-up. Eligibility criteria were: (1) an EYO from −15 to +10 years (i.e., between 15 years before and up to 10 years after predicted onset) and (2) a global Clinical Dementia Rating® (CDR) scale[16] between 0 and 1 inclusive. We refer to the participants who meet these criteria as “trial eligible”. In addition, we also performed analyses on a subsample of these trial-eligible participants with a global CDR score of 0 at baseline (i.e. cognitively normal). We term this group the “presymptomatic-only trial eligible” participants.

Image Processing

MRI images were processed by the central imaging core at Washington University using FreeSurfer 5.3 [17] as well as an in-house whole brain parcellation technique based on Geodesic Information Flow[18]. For bilateral structures, left and right volumetric measurements were summed. Total intracranial volume (TIV) was also extracted from the T1 image using Statistical Parametric Mapping 12 (SPM12) and served as a proxy for head size[19]. Direct measures of whole brain and ventricular atrophy were calculated using the boundary shift integral (BSI)[20–22]. Follow-up data acquired on different MRI scanners (58 scans from 40 participants) from their first visit were excluded as were data (40 scans from 26 participants) with significant motion, geometric distortion between timepoints, and non-AD pathology (infarcts, traumatic injury). ¹⁸F-flouroxyglucose (FDG) and ¹¹C-Pittsburgh compound B (PIB) PET images, measuring glucose metabolism and amyloid accumulation respectively, were processed using the PET Unified Pipeline (PUP) pipeline[23] that provides regional Standard Uptake Value Ratio (SUVR) measures for all FreeSurfer cortical regions of interest (ROIs), where the whole cerebellum served as the reference region. As there has been evidence in some cases of amyloid deposition in the cerebellum in ADAD[24,25], we examined SUVRs with the brainstem as a reference, but there was minimal change in the results. SUVR values were obtained with partial volume correction (PVC) using the geometric transfer matrix approach[26,27], as well as without PVC. The PVC PIB SUVR values were used for sample size analysis as they consistently produced greater differences in mean levels between carriers and non-carriers than the non-PVC PIB values. In contrast, the non-PVC FDG SUVR produced greater differences in mean levels between carriers and non-carriers than the PVC, so these values were included in the sample size analysis instead for the FDG outcome measure. As with the MRI biomarkers, scans acquired on different PET scanners (52 PIB scans from 35 participants and 48 FDG scans from 34 participants) were excluded from analysis.

Cerebrospinal fluid analysis

Collection of CSF was performed according to a protocol consistent with Alzheimer’s Disease Neuroimaging Initiative (ADNI) and analysis was performed by the central biomarker core at Washington University[28]. For this study, we included the Aβ 1-40, Aβ 1-42, p-tau181 and total tau measures from two immunoassays: the Luminex bead-based multi-plexed xMAP technology (INNO-BIA AlzBio3, Innogenetics) and the Lumipulse automated immunoassay system (LUMIPULSE G1200; Fujirebio, Malvern, PA, USA). For the CSF XMAP, both the cross-sectional and longitudinal processing pipelines were considered.

Choosing a target therapeutic effect

An important decision for sample size estimation is how to define a target treatment effect. The specified treatment effect should be large enough to represent a clinically meaningful benefit, but not so large that it would be implausible to achieve and result in the trial being underpowered.

Treatment effects are often expressed relative to what a “completely successful” treatment would achieve. In defining such a treatment effect, we believe that there needs to be a distinction between measures of primary pathology (PIB PET and CSF) and outcomes measuring downstream processes reflecting neurodegeneration (MRI, FDG PET, cognitive). For biomarkers of downstream processes, a treatment would be judged completely successful if it were to reduce the average rate of change in the biomarker to that observed in normal ageing. This is because current treatments are not (yet) expected either to reverse the course of neurodegeneration (e.g. to restore lost neurones) or to be able also to halt losses associated with normal ageing processes in individuals with no biomarker evidence of AD pathology. However, for biomarkers of primary pathology, there is growing evidence that slowing the rate of pathological accumulation in amyloid and tau will not be sufficient to modify the disease in manner that would be clinically meaningful to patients. Rather, large reductions in the absolute levels of primary pathology by the end of the study would be needed in order to provide a tangible, clinical benefit to patients[29]. In fact, results from recent trials of anti-amyloid therapies have shown it is possible to achieve very substantial reductions in PET and CSF amyloid outcomes[1,3,12,30], and in some cases also to show slowing of cognitive decline[1,2]. Therefore, for PIB PET and CSF outcome measures an effect on amyloid burden would be considered completely (100%) successful if average absolute levels were reduced to normal by the end of the study.

Other treatment effects can be defined relative to a completely successful one. For example, a 50% effective treatment acting on a marker of neurodegeneration would halve the average excess rate of change (over and above that seen in normal ageing) whereas a 50% effective treatment acting on a measure of amyloid burden would halve the average excess level. Deciding on a clinically relevant target treatment effect is not straightforward. For markers of neurodegeneration we chose a 50% reduction in the rate of change in carriers, relative to the rate of change in non-carriers (Figure 1, left panel). A reduction of this magnitude is similar to one thought to be clinically meaningful based on decline in the Preclinical Alzheimer’s Cognitive Composite (PACC) in Aβ+ cognitively normal patients compared to those that were Aβ-[9]. For measures of amyloid burden, we chose a reduction of 25% in excess level (level over and above that in non-carriers; Figure 1, right panel) by the end of the trial, as this would likely be the minimum level that could provide clinical benefit[31] for presymptomatic individuals. While phase III clinical trials of anti-amyloid DMTs have have shown far greater reductions in amyloid PET, our proposed level of 25% reduction has been observed in amyloid PET for some ADAD studies[32] and may be more in line with what is observed in CSF and plasma biomarkers. Further details on the definition of the target therapeutic effects are given in the Statistical Appendix.

• Download figure
• Open in new tab

Figure 1

Visual examples of the two types of treatment effects considered in this sample size analysis. (Left) The treatment effect is based on a reduction of the slope, with a 100% therapeutic effect being defined as one that reduces the slope in the treatment arm to the slope observed in the non-carriers. The outcome variable may undergo a transformation before analysis so that trajectories are more linear in nature. (Right) The treatment effect is based on reducing the outcome measure by the end of the study, with a 100% therapeutic effect defined as one that would reduce the absolute level of the outcome measure to the expected outcome in non-carriers. While the analysis may be conducted on the log transformed scale, the treatment effect is defined on the scale of the original outcome variable in order to make the definition of the treatment effect more interpretable.

Study Design and Statistical Methods

The methodology for assessing the trial designs is described in full in the Statistical Appendix. Briefly, a two-stage approach was used[33,34]. In stage 1, linear mixed models (LMM) were fitted to the observational repeated measures data from carriers and non-carriers in DIAN-OBS to obtain estimates of parameters that allow us to define plausible target therapeutic effects and to quantify components of variability. In stage 2, estimates from the LMM are used to compute sample size requirements for four-year trials with single measures of candidate outcome measures at baseline and follow-up (or a single direct measure of four-year change).

We selected candidate outcome measures from the ADAD literature; ideal outcome measures are sensitive biomarkers that reflect the key disease processes[17,28,35–37]. We selected PIB SUVR from six ROIs (precuneus, posterior cingulate, inferior parietal, interior temporal, middle temporal, and a mean cortical SUVR of the precuneus, prefrontal cortex, gyrus rectus, and lateral temporal regions) and six CSF measures (Ab1–40, Ab1–42, total tau, p-tau181, the Ab1–42 to Ab1–40 ratio, and the p-tau181 to Ab1-42 ratio) as measures of AD-related pathology. CSF total tau is included in this group although it is regarded variably as a marker of AD pathology and neurodegeneration. For the MRI measures of brain atrophy, volumes from five ROIs (whole brain, lateral ventricle, hippocampus, precuneus, and posterior cingulate), and two direct measures of change (brain and lateral ventricle BSI) were used. Measures of hypometabolism were assessed using six ROIs from FDG PET (precuneus, posterior cingulate, inferior parietal, hippocampus, banks of superior temporal sulcus (banks STS), and a mean cortical SUVR). Finally, cognitive decline was measured from the mini mental state exam (MMSE), the CDR sum-of-box scores (CDR-SB), and a cognitive composite including four scales: MMSE, the Logical Memory delayed recall score from the Weschler Memory Scale-Revised, animal naming, and the digit symbol score substitution from the Wechsler Adult Intelligence scale-Revised. To generate this composite, each scale was separately z-scored, and the mean of the four z-scores was taken. This composite is similar in nature to the composites used in other DIAN-OBS[38,39] studies as well as the composite used in DIAN-TU[40]. MRI, PET, and CSF markers were log-transformed to provide outcome measures on a scale representing annual percentage change from baseline. Cognitive scores were left untransformed, as this is common practice in phase III trials, and it allows a more intuitive interpretation of the resulting treatment effects.

We considered putative placebo-controlled two-arm parallel trials (1:1 randomisation) with a duration of four years. While a trial duration of four years tends to be longer than most phase 3 trials in sporadic AD, a longer duration is likely needed for studies involving presymptomatic participants to allow changes in the outcome that can be detected. This duration also matches the “common close” design of DIAN-TU-001, where all participants were followed up for at least four years. For all outcomes other than “direct” measures of change we assumed that the outcome measure would be obtained at baseline (pre-randomization) and then again at the end of the follow-up period (four years). Based on this design, we assumed that an analysis of covariance (ANCOVA) would be used for the statistical analysis, with MRI, PET and CSF markers log-transformed as in the analysis of DIAN-OBS. For volumetric MRI measures we assumed that TIV would be included as an additional covariate in the ANCOVA model, as TIV serves as a proxy for head size. For “direct” measures of change between the two timepoints, such as those obtained from the BSI, we assumed that between group comparisons would be carried out using a t-test. In addition to considering trial designs where all of the trial eligible participants from DIAN-OBS would be eligible, we also considered a presymptomatic trial design, enrolling the subset of participants with a global CDR score of 0 (i.e. cognitively normal participants).

To define target theraupeutic effects for these trial designs, estimates of the mean rates of change over time in these candidate outcome measures for both carriers and non-carriers are needed, as well as estimates of relevant variances and covariances in carriers. To obtain estimates of these key parameters, we fit LMMs (see Statistical Appendix for details) to the longitudinal data from the full sample and presymptomatic subset of DIAN-OBS. Data from DIAN-OBS participants were included if there were outcome measures available at both a “baseline” visit that satisfied the specified eligibility criteria and at least one subsequent follow-up visit. For volumetric MRI measures, TIV and its interaction with time were included in the LMM. We excluded outcome measures from sample size calculation when the LMM did not converge, or when they were not considered to be suitable candidates for a future trial (see Statistical Appendix for more details).

We assumed 40% dropout rate at 4 years in our putative trials. This is slightly conservative compared to recent clinical trials: 27% over four years for DIAN-TU-001 trials of gantenerumab and solanezumab[12], 23% for the TRAILBLAZER-ALZ-2 randomized controlled trial (RCT) of donanemab[2], 17% of 18 months for the Clarity AD RCT of lecanemab[1], and 29% for 4.5 years in the A4 study of solanezumab[41]. For all outcome measures, we obtained sample size estimates that would be required to detect a clinically meaningful benefit with 80% statistical power using a two-sided significance level of 5%. Uncertainty in the resulting sample size estimates was quantified using 95% bias-corrected and accelerated (BCa) confidence intervals obtained through bootstrapping. A modified version of the Stata package slopepower[42] was used to implement the sample size calculations (modifications including allowing adjustment for TIV and analysis of direct measures of change).

Sample sizes to detect a slowing in the rate of neurodegeneration

For four-year trials that included all trial eligible participants (global CDR=0-1), many neurodegeneration biomarkers provided similar sample sizes to detect a 50% reduction of slope (relative to non-carriers). Nine outcome measures (Cognitive: composite; MRI: ventricles, ventricular BSI, whole brain, precuneus and hippocampus; FDG PET: banks STS, inferior parietal, and precuneus) had sample size estimates less than 150 participants per arm. However, caution is warranted when making sample size recommendations for two reasons. First, the upper 95% confidence limits for these estimates (see Uncertainty in sample size estimates; below) are as high as 430 individuals/arm. Second, when a best-performing biomarker is selected from many, the performance of that biomarker is likely to be worse in a new setting due to effects analogous to the well-known phenomenon of regression to the mean.

Ventricular enlargement had some of the lowest sample size requirements across both trial scenarios. Ventricle enlargement can be measured with high precision due to its high-contrast boundaries. The measure is sensitive, but not specific, to pathological atrophy. However, evidence from clinical trials, particularly from anti-amyloid DMTs, indicate that ventricular enlargement may worsen in treatment compared to placebo[43], making its usefulness as an outcome uncertain for this class of DMT.

In trials recruiting individuals with global CDR=0 only, sample sizes were higher than trials including all eligible participants (global CDR=0-1): approximately 3.1-4.3 times higher for cognitive measures and 2.4-4.7 times for MRI. For trials involving CDR=0 participants only, no FDG measures had sufficiently different slopes between non-carriers and carriers to include them as a potential outcome measure (see statistical appendix). Previous results from DIAN-OBS show evidence of increased atrophy and hypometabolism during the presymptomatic stage of the disease. These changes were observed relatively close to EYO (within five years)[17,37,44], though some ROIs (precuneus, posterior cingulate, banks STS) do show changes as early as 12 years before EYO.

Sample sizes to detect a reduction of the outcome at the end of the study

From amyloid PET, we obtained sample size estimates of ∼40 participants per arm to detect a 25% reduction in the overall level of amyloid burden with 5% statistical significance and 80% power. However, 95% CIs extended up to around 140 participants per arm. When restricted to CDR=0 participants, the sample sizes for PIB PET did not increase as much as other outcomes. Sample sizes were comparable when using CSF measures of Aβ42. In both carriers and non-carriers, we found CSF p-tau181 declined over time when using the older XMAP assay, despite substantially increased values in carriers at baseline. This longitudinal decline has been previously observed in symptomatic DIAN-OBS participants[39]. Rates of change in the CSF Lumipulse assay for p-tau181 showed increased rates of change in carriers.

Recent trials have demonstrated that DMTs show large reductions in amyloid burden as measured by PET in individuals with mild AD. In Clarity AD, lecanemab showed evidence of amyloid removal (59 centiloids (77%) decrease from baseline) [1], while TRAILBLAZER-ALZ 2 reported a reduction of 88 centiloids (87%) over 18 months in participants treated with donanemab[2]. These reductions tend to be larger than our proposed target effect. However, in the DIAN-TU-001 clinical trial of ADAD, amyloid burden was reduced by 24% over four years in patients treated with gantenerumab compared to the shared placebo arm[12]. Recent results from the open-label extension suggest that asymptomatic participants treated with gantenerumab for the longest duration may experience delays in symptom onset[45]. We chose the target therapeutic effect of 25% to represent a minimum requirement that would have a resonable chance of producing a meaningful clinical benefit in trial scenarios at early stages of the disease process. This level of treatment effect may be more plausible for CSF and plasma markers of amyloid and tau, which have shown smaller treatment effects. Larger target treatment effects might require fewer participants than we report, but this is not guaranteed. Departures from normality might render the use of ANCOVA inappropriate and hence the basis of our sample size calculations suspect. Variability in the numbers of dropouts (which is reasonable to ignore when sample sizes per arm are large, but not when small) would need to be taken account of in the methodological approach in order to give realistic required sample sizes. For these reasons, we advise not to overrely on predictions using our methodology when these are much below 100 (50 per arm). If larger effects are anticipated, it may be more advisable to carry out trials with shorter durations than four years. These choices will depend on the ability to recruit and retain participants in this rare form of AD, and how effectiveness may vary at shorter durations due to titration regimes that aim to avoid ARIA.

Tau-specific PET tracers are increasingly being included in trials to determine effects on tau burden. Previous longitudinal tau PET studies in ADAD suggest that changes occur very close to expected symptom onset[46,47]. Hence they may be better suited for anti-tau therapies in carriers close to onset. Recent advances in plasma biomarkers could also serve as potential outcomes. Donanemab reduced plasma concentrations of p-tau217 by about 25% in TRAILBLAZER-ALZ 2[2].

Uncertainty in sample size estimates

While some point estimates of sample size appear promising, these estimates come with varying degrees of uncertainty, which must be considered when choosing outcome measures for upcoming clinical trials. One way that we can measure the level of uncertainty is to take the ratio between the upper limit on the confidence interval, which could represent a “worst case” scenario for the number of participants needed to sufficiently statistically power a trial, with the lower limit, which could represent a “best case” scenario. For the sample sizes based on all trial eligible subjects (CDR=0-1), the markers producing the largest level of uncertainty (as measured by this ratio) were FDG PET markers (ratios between 4.8 to 12), CSF Aβ 1-42 (ratio=4.8 for Lumipulse, 5.9 for XMAP), whole brain volume (ratio=5.7), and posterior cingulate volume (ratio=5.2). On the other end, BSI-based measures of whole brain volume (ratio=2.2) and lateral ventricles (ratio=1.9), cognitive composite(ratio=2.8), and PIB measures (ratio range=2.6-3.2) provided the lowest uncertainty. These measures had the greatest precision in measuring rate of change over time, likely making them more viable for shorter duration trials. For most outcomes, the level of uncertainty increased when trials were restricted to participants with global CDR=0. The exception were PIB PET markers, where the mean estimates and uncertainty ranges tended to be similar (CI ratio: 3.3-4.1). This is likely due to amyloid accumulation being one of the early observed changes in both sporadic AD and ADAD, with accumulation over time being similar for CDR=0 and CDR=0-1 participants (nearly parallel slopes in Figure 2 and Supplemetal Figure 1).

For many outcomes, particularly CDR-SB, CSF protein levels, FDG-PET and some regional PIB SUVR, there are a high number (> 1%) of bootstrap samples where the LMM failed to fit the data. As the number of bootstrap failures increase, more caution should be given to interpreting the level of uncertainty in the sample size estimates, as it is likely that these missing bootstraps more often represent samples with very high sample size estimates.

Some of the high levels of uncertainty could be attributed to heterogeneity between individuals with ADAD. While ADAD is considered a “pure” form of AD in terms of fewer co-morbidities, heterogeneous patterns of pathology have been observed between mutations in the PSEN1 and APP genes[48], as well as within PSEN1 mutations[49], which more frequently have atypical phenotypes[6]. Clinical stage is another source of heterogeneity; symptomatic participants have higher rates of atrophy, cognitive decline and hypometabolism compared to presymptomatic carriers, even at mildly symptomatic stages[5,17,35–37]. If there are clear dependencies of endpoints on variables such as CDR®, then one efficient approach would be to stratify at randomisation according to these variables and account for this stratified design in the statistical analysis[34].

DIAN is a closely monitored cohort of motivated individuals, many of whom begin in DIAN-OBS and then enroll in DIAN-TU. As a result, run-in studies, which have been shown to provide an increase in power[14,33], could be considered. Future work will explore sample size estimates for additional trial designs, such as run-in, common close, and adaptive trials.