Mental health in the UK Biobank: a roadmap to self-report measures and neuroimaging correlates

Dataset

Imaging data from 32,420 UKB participants were available at the time the study was performed. From this we selected multiple independent test cohorts (Fig. 1; Table 1). Subjects with a mean head motion greater than 0.2mm were removed resulting in the exclusion of 5,265 subjects. Subjects with any missing online questionnaire or scan 1 assessment center mental health data were also removed, resulting in the exclusion of additional 10,848 subjects (largely because the online questionnaire was only performed in a subset of UKB participants). From the remaining 16,307 subjects, we selected individuals who had undergone brain scans at two timepoints. These subjects make up the test-retest sample.

• Download figure
• Open in new tab

Figure 1:

UK Biobank subject inclusion chart.

Late onset depression (first episode at age 60+) is associated with different brain correlates (e.g., white matter hyperintensities) and different risk factors (e.g., vascular risk) compared with recurrent early onset depression (age of first episode before 60) [Salo et al., 2019]. Therefore, we assessed subjects for probable late-onset depression based on self-reported age at the first episode of depression (Data-field: 20433). Subjects who reported their first episode at 60 or older (N=418) were excluded.

The majority of individuals within the UKB cohort are expected to have no mental health conditions because it is a population sample. To ensure sufficient power to identify neural correlates of mental health, we wanted to reduce the expected over-representation of healthy individuals and ensure that our samples richly capture mental health variability. This was achieved by including equal numbers of participants with and without a history of mental health. From the UKB showcase we used: Seen doctor (GP) for nerves, anxiety, tension or depression (Data-field: 2090) to ensure our samples included an equal number of subjects who experienced mental health issues on at least one occasion, and those who have not. For each subject who had seen a GP for nerves, anxiety, tension or depression (N=4,531) we paired a matched subject from those who had never seen a GP for nerves, anxiety, tension or depression (i.e., subject pairs were identically matched for sex and age, and minimal difference in head motion). Subsequently, approximately two-thirds of the ‘never seen GP’ subjects together with their matched ‘seen GP’ subjects was randomly assigned to the exploratory sample, and the remaining subjects were assigned to the confirmatory sample. During subject assignment to groups, we preserved the matched characteristics within each resulting sample (Fig. 1). No subjects overlapped between the exploratory and confirmatory samples.

Mental health measures

The set of self-report questions related to mental health included in the UKB were informed by standardized measures, but did not simply cover a list of previously validated scales. Table 2 summarizes the 5 different UKB mental health measures, which will be used for neuroimaging and questionnaire comparison analyses and Fig. 2 provides an overview of the acquisition timing of these mental health measures relative to the scan days. The questions included in the online questionnaire enable calculation of the Generalized Anxiety Disorder (GAD-7) and Patient Health Questionnaire (PHQ-9) scores [Davis et al., 2020]. Using the Assessment center information, the Eysenck Neuroticism (N-12) score was calculated. Smith and colleagues used questions from the Assessment information to develop a categorical (case-control) measure of depression [Smith et al., 2013]. For the purposes of our study we adopted similar definitions to obtain a categorical assignment of Probable Depression Status, but we did not differentiate between single and recurrent episodes of depression. Depression status was set to 1 if subjects responded yes to variable IDs 4598 or 4631 (ever depressed | ever unenthusiastic/disinterested), and reported a duration of at least 1 week to variable IDs 4609 or 5375 (depression | unenthusiasm/disinterest), and had seen either a GP or psychiatrist for nerves, anxiety, tension, depression (i.e., responded yes to variable IDs 2090 or 2100).

• Download figure
• Open in new tab

Figure 2:

Schematic overview of the acquisition timing of UK Biobank mental health measures in relation to imaging acquisition. Mental health measures in light green were obtained on the day of scanning, whereas mental health measures in light blue were obtained at an independent time point that varied from 1,185 days before to 964 days after scan 1 across participants. The range of possible scores for each mental health measure is included. All five measures were included in neuroimaging and questionnaire comparison analyses in this paper.

For our study, we proposed a new summary measure of state depression using UKB questions included in the Assessment center information: Recent Depressive Symptoms (RDS-4), which is a continuous measure of depression symptomatology obtained on the day of scanning. The four self-report questions used for the RDS-4 assess depressed mood, disinterest, restlessness, and tiredness. Each question asks about recent experiences of symptoms (past 2 weeks). The response options for the four questions are: not at all (1), several days (2), more than half the days (3), and nearly every day (4). The summed score across these four variables therefore has a range of 4-16. Moreover, the RDS-4 questions correspond with several DSM-V diagnostic criteria for major depressive disorder and cover depression domains that are also considered in other measures such as the Hamilton and Montgomery–Åsberg scales.

There are a number of important differences between the RDS-4 and the other mental health measures. Compared to PHQ-9, the RDS-4 was obtained on the day of the imaging scan, whereas the PHQ-9 was undertaken at a time point that was independent from the scan date. Compared to probable depression status, the RDS-4 provides a continuous measure of recent symptom severity, whereas probable depression status is a categorical (case-control) measure of lifetime occurrence of depression. Compared to N-12, RDS-4 is a measure of recent (‘state’) depressive symptoms, whereas N-12 is a more general measure of personality (‘trait’). Compared to GAD-7, the RDS-4 focuses on depression and the GAD-7 focuses on anxiety.

Imaging acquisition

UKB structural modalities include: T1-weighted (T1), T2-weighted (T2), susceptibility-weighted MRI (swMRI); diffusion MRI (dMRI); and functional modalities: task-based fMRI (tfMRI) and resting-state fMRI (rsfMRI). MRI data were obtained using a Siemens Magnetom Skyra 3T scanner. For T1 structural scans, 3D MPRAGE acquisition was used to acquire 1mm isotropic resolution. For T2 scans fluid-attenuated inversion recovery (FLAIR) contrast was used with the 3D SPACE optimized readout providing a strong contrast for white matter hyperintensities. For swMRI, a 3D gradient echo acquisition was used (resolution: 0.8×0.8×3mm), obtaining two echo times (TE=9.4 and TE=20 ms). Diffusion data was acquired with b-values of 1000 and 2000 s/mm2, at 2mm spatial resolution, with a factor 3 multiband acceleration and 50 distinct diffusion-encoding directions. Both tfMRI and rs-fMRI used identical acquisition parameters (spatial resolution= 2.4mm, TR= 0.735s, factor = 8 multiband accelerator). Task fMRI used the Hariri faces/shapes “emotion” task as employed in the HCP [Barch et al., 2013; Hariri et al., 2002], with a shorter total length and reduced repeats of the total stimulus block. For further information on UKB imaging, please refer to [Miller et al., 2016].

Imaging derived phenotypes

In addition to raw and processed imaging data, Image Derived Phenotypes (IDPs) are available for download. IDPs are derived from calculations that combine many images and/or voxels to produce a scalar quantity from the processed imaging data [Miller et al., 2016]. Examples of IDPs include regional volumes from structural MRI and ‘edges’ from resting state functional MRI (i.e., connectivity between a pair of networks).

The IDPs included in this paper are summarized in Table 3, and further information can be found in [Miller et al., 2016] as well as the UKB showcase brain imaging documentation resource (https://biobank.ndph.ox.ac.uk/showcase/showcase/docs/brain_mri.pdf). Briefly, resting state IDPs were obtained using Independent Components Analysis performed at two different dimensionalities (25 and 100), which resulted in 21 and 55 signal networks, respectively. Subject-specific BOLD time series for each network were calculated using dual regression [Nickerson et al., 2017], and the amplitude for each network (temporal standard deviation) and functional connectivity between pairs of networks (full or partial correlation coefficients) were calculated. Resting state IDPs from both ICA dimensionalities were included as they may offer complimentary information at different levels of functional organization. From T1-weighted images, gray matter volumes were obtained with FSL FIRST and FAST, and cortical area and thickness were calculated with Freesurfer. Total volume of white matter hyperintensities was estimated based on T1-weighted and T2-flair images using FSL’s BIANCA algorithm [Griffanti et al., 2016]. From the diffusion data, weighted mean fractional anisotropy (FA) and mean diffusivity (MD) were obtained using FSL’s DTIFIT tool. Task fMRI IDPs reflect summary measures of activation (the median and 90th percentile for both the percent signal change and the z-statistic) in regions selected from the group-level activation map. Susceptibility weighted IDPs were generated from the signal decay times predicted from the magnitude images at the two TEs such that the IDPs equate to the median signal decay times.

Confound variables

All analyses were corrected for the ‘simple’ set of confounds described in [Alfaro-Almagro et al., 2021], namely: scanning site, age, age squared, sex, age * sex, head size, head motion in resting fMRI and in task fMRI scans, date, and date squared. This confound set was previously shown to explain 4.4% of variance in UKB imaging variables on average, and captured the most important sources of confound variation [Alfaro-Almagro et al., 2021].

Correlations amongst mental health variables in the UKB

To characterize the degree of overlapping information between mental health measures, Spearman rank correlations were computed between all measures of mental health using data from the exploratory sample (N=6,636).

Data used to compute RDS-4 and N-12 were collected at the scan date (assessment center information), whereas GAD-7 and PHQ-9 were computed from data obtained from the online questionnaire. The absolute number of days elapsed between the two data collections ranged from 0 to 1,185 days. To investigate the effects of measurement latency on mental health measure correlation, Spearman rank correlations between the RDS-4 and PHQ-9 (both measures of depression) were computed as a function of elapsed time between measurement (see Supplementary Materials section S1 and Fig. S1).

To test whether self-report measures differed significantly based on probable depression status, a two-sample Kolmogorov-Smirnov test was performed to ascertain whether subjects with a positive depression status had different distributions of depression scores than subjects with no depression status.

Mapping between mental health variables in the UKB

To gain insights into how the different measures of mental health included in the UKB relate to each other, we used equipercentile linking in the exploratory sample. Here, the stepwise percentiles for each measure were calculated, and for each score in one measure the equivalent percentile rank in a different measure was mapped [Kolen and Brennan, 2014]. We further calculated the Cronbach alpha for the newly proposed RDS-4 score to measure internal consistency in the exploratory sample.

Mechanical Turk study to validate RDS-4

To further validate the proposed RDS-4 score, we performed an independent study using the Amazon Mechanical Turk platform via CloudResearch.com [Litman et al., 2017]. Participants were paid a nominal compensation for questionnaire completion. 134 participants aged 60+ completed the study. This study was reviewed by the Washington University in St Louis IRB board and approved as exempt (IRB #201909165) because participants were fully anonymous (the option of ‘anonymized worker IDs in CloudResearch was adopted) and no participant key was available to any member of the research team.

Participants completed the same set of mental health questionnaires at two timepoints 7 days apart using the Qualtrics software (Qualtrics, Provo, UT). The following questionnaires were presented in randomized order: RDS-4, PHQ-9, CES-D (Center for Epidemiological Studies - Depression; [Radloff, 1977]) and MASQ-30 (short-form Mood and Anxiety Symptoms Questionnaire; [Wardenaar et al., 2010; Watson and Clark, 1991]). The latter two measures were included because they are commonly used measures of depression that can be considered ‘gold standard’ for self-report. Although these measures are not available in the UKB, our goal was to validate the RDS-4 against these standardized measures.

We undertook multiple steps to avoid low quality responses, which can be a concern in Mechanical Turk questionnaire research. Firstly, we adopted premium options in CloudResearch, such as only including ‘CloudResearch approved participants’ who undergo more extensive vetting. Secondly, we included two questions to assess the attention levels of the participants while performing the study (“If you are still paying attention, please select ‘yes’” & “Please answer this question with the ‘Most or all of the time’ option”). Participants who failed to answer these questions appropriately were excluded. Thirdly, we imposed a minimum duration for questionnaire completion at 172.5 seconds (which equals 2.5 seconds per question). Participants who completed the questionnaire in less than 172.5 seconds were excluded.

Spearman rank correlation was used to compare scores between the RDS-4, PHQ-9, CES-D, and MASQ-30 using data from time point 1. Intraclass correlation coefficient (ICC A,1; also known as criterion-referenced reliability [Koo and Li, 2016; McGraw and Wong, 1996]) was used to calculate the test-retest reliability between time point 1 and time point 2 separately for each measure.

Exploratory brain-mental health analysis

We used Canonical Correlation Analysis (CCA) as a data-driven approach to identify joint multivariate relationships between mental health measures and brain imaging variables [Hotelling, 1936]. Following nuisance regression to remove variance explained by nuisance regressors, dimensionality reduction was performed separately for resting state, structural, and task IDPs (Table 3) using Principal Component Analysis (PCA). The substantial differences in IDP numbers between resting state IDPs (3,466), structural IDPs (346), and task fMRI IDPs (16) was the reason for performing the dimensionality reduction separately to ensure that all classes of IDPs were represented in the input components. The top components explaining at least 50% of variance were retained for each of resting state, structural, and task IDPs. This threshold was chosen as a good trade-off between retaining a substantial amount of IDP variance for the CCA while limiting the number of input variables to the CCA to ensure a sufficient subject-to-variable ratio required for stable CCA results [Helmer et al., 2021]. The structural and task IDP matrix included a small number of missing values, which were excluded for the nuisance regression and then imputed using nearest neighbor imputation (Matlab’s knnimpute.m). The combined set of IDP eigenvectors were entered into the CCA against 5 mental health input variables corresponding to summary scores from GAD-7, N-12, PHQ-9, RDS-4 and probable depression status (residuals after regressing out confound variables). CCA was performed on N=6,636 subjects in the exploratory sample. Permutation testing with 2000 permutations was used to obtain p-values for the resulting canonical correlations. Here, the subject order of IDP component inputs and mental health inputs were independently shuffled to break subject correspondence. This is especially important for CCA because the canonical correlation is explicitly maximized and therefore it is important to compare the canonical correlation to the empirical null distribution obtained with permutation testing (which does not center around zero but shows relatively high null correlations) [Smith et al., 2015].

To calculate the univariate contributions (or ‘loadings’) from individual IDPs to the CCA result, we correlated subject scores against original IDPs. For this purpose, the ‘U’ and ‘V’ canonical subject scores from the strongest CCA result were averaged within each subject to obtain a CCA summary subject score (UV). Here, U = XA and V = YB, where X are the IDP principal component inputs and Y are the mental health inputs. A and B are the canonical coefficients for IDP eigenvectors and mental health variables respectively, which are optimized such that the correlation between U and V is maximized. We could calculate IDP contributions by correlating U with the IDPs, but the resulting correlations would potentially be inflated because U is optimized for X. Therefore, using the averaged UV subject score for correlations with the IDPs provides a more realistic and unbiased measure of individual IDP correlations [Bijsterbosch et al., 2018]. Bonferroni correction for multiple comparisons was performed for these post-hoc correlations that were used to estimate univariate contributions from each original IDP (i.e., p-value below 0.05/(3,466+346+16)=1.3*10⁻⁵, where 3,466 is the number of resting state IDPs, 346 is the number of structural IDPs, and 16 is the number of task IDPs). IDPs that survived correction were selected for subsequent tests of effect size in the confirmatory sample. These IDPs are referred to as ‘selected brain variables’ in subsequent confirmatory analyses.

The multivariate CCA results were also replicated in the independent confirmatory sample by projecting the resting state, structural, and task IDPs onto the same PCA subspace (i.e., not repeating the PCA, but using the weights from the exploratory sample), and multiplying brain eigenvectors as well as mental health scores by their respective canonical coefficients (i.e., A & B as estimated from the exploratory sample). The CCA replication was tested based on the correlation between the resulting U and V (i.e., the canonical correlation). We also performed the same post-hoc univariate correlations between averaged UV and individual IDPs as described above to assess the replicability of IDP contributions to the CCA.

Confirmatory analysis of effect size

The independent confirmatory sample (N=2,426) was used to test univariate effect sizes of selected brain variables from CCA analysis (i.e., significant IDPs after Bonferroni correction). Specifically, we performed a Cohen’s d test based on probable depression status, and calculated the Pearson’s r from the correlations between the selected brain variables and each of the four mental health variables (i.e., RDS-4, PHQ-9, N-12 and GAD-7), respectively. These analyses were repeated for each imaging modality including surface area, gray matter volume, cortical thickness, white matter hyperintensity, fractional anisotropy, median T2*, task activity, resting-state network amplitude and edge connectivity at both dimensionalities (i.e., 25 and 100). We de-confounded both the brain variables and the mental health variables before running the aforementioned analyses. The only exception from de-confounding is the binary grouping based on probable depression status, as deconfounding would result in subject-specific values that are non-categorical, which is unsuitable for the Cohen’s d test.

Test-retest reliability of imaging measures

To assess the stability of IDPs across time, we performed test-retest reliability analyses using data from N=624 subjects that were scanned twice at separate time-points, with an inter-scan interval of approximately 2 years (Table 1). Data were de-confounded for this sample using the same approach employed for the exploratory CCA analysis. After data were de-confounded, intra-class correlations were computed between the IDPs collected at each scan time-point using the ICC(A,1) formulation to quantify the agreement between measurements collected at each timepoint [McGraw and Wong, 1996]. Test-retest reliability measures were grouped according to IDP measurement modality (e.g., cortical area, cortical volume, etc.) to allow for assessment of the ICC distributions for different modalities.

We also assessed the effect of inter-scan interval length on the test-retest correlation strengths by computing ICCs for each IDP after including regressing out the inter-scan interval (in days) from each IDP, thus removing any additional variance attributable to inter-subject differences in inter-scan interval lengths. Finally, we assessed whether ICCs were affected by mental health changes as indicated by the difference in the RDS-4 scores between time-points. Of the N=624 subjects included in the test-retest analyses, n=336 exhibited no change in RDS-4 scores between time-points, while n=288 exhibited changes in RDS-4 scores between time-points (i.e., at least 1 point difference in the RDS-4 scores). For these analyses, we separately computed ICCs for each mental health sub-group and then plotted the ICC distributions for each modality between the sub-groups. We also computed ICCs after regressing out mental health change values from each IDP.

Correlations amongst mental health variables in the UKB

Mental health measures showed moderate correlations with one another, indicating redundancy between these metrics (Fig. 3A). RDS-4 and N-12, which are both measured from questions administered on the scan date, had a Spearman rank correlation coefficient (SRCC) of ρ=0.57±0.01 (p ≈ 10⁻¹⁹⁹); PHQ-9 and GAD-7, which were both taken from the online questionnaire, have SRCC ρ=0.69±0.01 (p ≈ 10⁻²⁶⁷). Correlations between PHQ-9 and GAD-7 scores were significantly higher than between any other pairs of scores (p< 10⁻⁹).

• Download figure
• Open in new tab

Figure 3.

A) Spearman rank correlation coefficients between each pair of mental health measures. Variables measured on the same date are labeled the same color (green = assessment center day-of-scan information; blue = online questionnaire). B) Distributions of scores for subjects with probable depression status (pink) and without probable depression status (cyan). Subjects with probably depression status scores significantly higher on all mental health measures (KS-statistic χ > 0.19, p < 10⁻⁴⁸).

Both the RDS-4 and N-12 measures were collected at each scan time, which allows for an assessment of the within-measure two-year correlation of these measures on a sample of N=555 subjects from the test-retest sample (69 subjects were removed from the full N=624 test-retest sample due to missing mental health assessment center information on scan 2). Within this subgroup, the subjects’ RDS-4 measures showed a 2-year Spearman rank correlation coefficient of ρ = 0.57 between initial and follow-up scans, and N-12 showed a 2-year correlation of ρ = 0.85. It should be noted that this reflects correlation between scan timepoints between 761 and 980 days apart. Therefore, a given metric’s 2-year correlation (i.e., self-correlation over a long time period) effectively establishes an approximate upper bound on any correlation value between it and other metrics collected over the same time frame. Because anxiety and depression are not fixed states and scores may meaningfully differ between the two timepoints available in the UKB, we also performed a separate Mechanical Turk study to test the short-term (7-day) test-retest reliability of RDS-4 (see ‘Mechanical Turk study to validate RDS-4’ section).

We performed a two-sided, two-sample Kolmogorov-Smirnov test on RDS-4, PHQ-9, N-12, and GAD-7 scores over subjects with and without probable depression status. Subjects with probable depression scored significantly higher than subjects with no probable depression status on all measures (KS-statistic χ > 0.19, p < 10⁻⁴⁸; Fig. 2B).

Mapping between mental health variables in the UKB

Given that this is a largely healthy sample, as expected, the distributions for PHQ-9, RDS-4, and GAD-7 all reveal a large number of participants with scores on the lower end of the mental health measure, with a sharp decline seen in the number of participants scoring on the upper end of the mental health measures (Fig. 4A-D). Notably, the distribution of N-12 is relatively less skewed than PHQ-9, RDS-4, and GAD-7.

• Download figure
• Open in new tab

Figure 4.

A-D are the distributions of scores for participant responses to each questionnaire. E-G depict the equipercentile linkages of the scores for each questionnaire, mapping the equivalence of a score from one questionnaire to the score of the other questionnaire.

Equipercentile linkage was used to map between different measures of mental health. The results show a stable and approximately linear mapping between RDS-4 and PHQ-9 (Fig. 4E). Additionally, our results show stable mapping between RDS-4 and N-12 (Fig. 4F), and between N-12 and GAD-7 (Fig. 4G). These results are in line with the literature showing that the personality trait of neuroticism is closely associated with mental health [Lahey, 2009].

We calculated Cronbach’s internal consistency alpha for RDS-4, which measures the internal consistency. The Cronbach alpha for RDS-4 was 0.78, which indicates a moderate to strong internal reliability. This was similar to N-12 (Cronbach alpha = 0.83).

Mechanical Turk study to validate RDS-4

Out of 134 subjects who completed our separate validation study, 3 subjects were removed because they failed the attention questions and a further 44 subjects were removed because they completed the surveys too fast, resulting in N=87 subjects (53 female and 34 male; mean age 66.0 ± 4.8). The results showed that RDS-4 was highly correlated with other depression scales and achieved test-retest reliability comparable to other depression scales (Table 4).

Exploratory brain-mental health analysis

Prior to performing the CCA, the data reduction of resting state IDPs resulted in 100 components which explained 50.1% of variance. The data reduction of the structural IDPs resulted in 24 components which explained 50.6% of variance. The data reduction of the task IDPs resulted in 2 components which explained 51.2% of variance. Therefore, the total number of brain variables input into the CCA was 126 and this was tested against the 5 mental health variables. The CCA resulted in two significant canonical covariates (R1_UV = 0.207, p = 0.0005 & R2_UV = 0.174, p = 0.015). The first multivariate canonical correlation partly replicated in the independent confirmatory sample (R1_{UV(confirmatory)} = 0.125, p = 3.7*10⁻⁹, where the p-value was Bonferroni corrected for the maximum of 5 canonical correlations). Although the second canonical correlation also reached significance in the confirmatory sample (R2_{UV(confirmatory)} = 0.06, p_Bonferroni = 0.02) we did not perform post-hoc analysis for this finding due to the low canonical correlation in the replication sample. There are a number of factors that may have contributed to the replicability of the first canonical correlation. Firstly, the CCA was relatively well-powered with 50.7 subjects per input variable leading to relatively stable estimates [Helmer et al., 2021]. Secondly, the exploratory and confirmatory samples were well matched in terms of sample characteristics. Thirdly, data reduction of IDPs prior to CCA likely reduces measurement noise. Post-hoc correlations between the averaged UV subject scores and the mental health variables and IDPs also replicated well (Fig. 5 and S2).

• Download figure
• Open in new tab

Figure 5.

Canonical correlation results. A: post-hoc correlations for non-resting (structural & task) IDPs, showing only significant IDPs after Bonferroni correction. A similar figure for the resting state IDPs is included in the supplementary material (Fig. S2). B (insert): post-hoc CCA relations for mental health show that the first canonical covariate is broadly linked to affect-based mental health.

In terms of post-hoc correlations with IDPs, 770 resting state IDPs and 86 structural IDPs, and 1 task IDP were significantly correlated with the canonical covariate (UV) after Bonferroni correction for multiple comparisons. The post-hoc CCA results confirm many regions previously highlighted in the literature such as prefrontal and orbitofrontal cortices.

IDPs that contributed significantly to the CCA were also tested for univariate direct correlations with individual mental health variables in the independent confirmatory sample (see next section for the results). For these follow-up univariate tests, we furthermore supplemented the target IDPs with a literature-curated list (supplementary table 1) that partly overlaps with the data-driven IDP identification.

Confirmatory analysis of effect size

Our findings showed that univariate effect sizes of the relationship between IDPs and mental health determined in our robust population sample were very low. Overall, effect sizes of the differences in the brain variables (i.e., IDPs), indicated by Cohens’ d, based on probable depression status, was larger than the Pearson’s r values from correlations between IDPs and continuous mental health measures (Fig. 6). On average, resting-state node amplitude and edge connectivity derived from partial correlation matrices appeared to have the higher effect sizes in most mental health measures, and task activity and fractional anisotropy ranked high in some mental health measures. At the level of individual IDPs, edges derived from both partial and full correlation matrices emerged as the best “predictors’’ in explaining data variance in all mental health variables except for PHQ-9 where amplitude of a few resting-state nodes ranked at top (Figs. S3-S7). These findings together suggest an overall higher effect size of resting-state in contrast to non-resting state measures on the investigated mental health variables.

• Download figure
• Open in new tab

Figure 6.

Effect sizes are shown for the grouped brain variables of structural (Area, Volume, Cortical Thickness, Fractional Anisotropy and T2*) and functional (Task Activity, Amplitude, Full Network connectivity matrix and Partial Network Connectivity Matrix) modalities. Blue boxes indicate the middle 50% of the data (i.e., the range between the first and third quartile) and small black squares and blue lines inside each box represent the mean and median values, respectively. Outliers for each grouped brain IDP are shown as blue circles, which are above the 1.5 times of inter-quartile range (IQR), indicated by the whiskers extending from the boxes. For detailed assessments of effect sizes in specific IDPs see supplementary figures S3-S7.

Test-retest reliability of imaging measures

We next assessed the stability of IDPs over time in 624 subjects who had data from two separate scan sessions conducted approximately 2-2.5 years apart. Fig. 7A shows the distribution of inter-scan intervals for all 624 subjects. To assess test-retest reliability, ICCs were computed between the scan 1 measurements for each IDP and the corresponding scan 2 measurements for the same IDP. Then, the ICCs were assigned to categories based on the measurement modality of the corresponding IDPs: brain surface area (62 measures), brain volume (154 measures), cortical thickness (CT - 62 measures), fractional anisotropy (FA - 27 measures), mean diffusivity (MD - 27 measures), T2* value (T2 - 14 measures), task activation (TA - 16 measures), resting-state time-series amplitudes (AMP - 76 measures), full correlation-based resting-state networks (FNT - 1695 measures), and partial correlation-based resting-state networks (PNT - 1695 measures).

• Download figure
• Open in new tab

Figure 7.

Test-retest analyses. A. The histogram shows the inter-scan interval distribution for the 624 subjects included in these analyses. The x-axis shows days between scans, and the y-axis shows the number of subjects. B. The boxplots show the ICCs obtained using brain IDPs after standard confound regression (blue) vs. ICCs obtained using brain IDPs after standard confound regression plus regressing out effects of inter-scan interval length (orange). IDP measurement modality categories are organized along the x-axis, and the y-axis shows ICC values. See also Supplemental Figure S8.

Fig. 7B depicts the distributions of ICCs for each IDP measurement modality obtained using the confound-regressed data from both scan time-points, along with those obtained after additionally regressing out the effects of inter-scan interval length (i.e. days between scans). Notably, ICC distributions were highly similar for both analyses. In general, IDPs corresponding to measures of brain structure had higher ICCs than IDPs corresponding to measures of brain function. The highest ICCs were observed for IDPs corresponding to brain volume/brain area measures and the lowest ICCs were observed for IDPs corresponding to task measures. This pattern of results is not particularly surprising since macro-scale structural properties like regional volume are expected to be relatively stable over time, especially when considering relative between-subject correlations. Macro-scale functional properties like task activation magnitudes or network connectivity patterns exhibit higher variability over time due to influences of factors such as the level of task engagement (during task), cognitive state (during rest), and physiological state (e.g. hungry vs. sated, sleepy vs. alert), and therefore are expected to have somewhat reduced test-retest stability.

Analyses performed for sub-groups of patients that did (n=288) vs. did not (n=336) exhibit changes in mental health between time-points as determined by the difference between RDS-4 measures obtained at each time point yielded highly similar results, as did those obtained after regressing out the change in RDS-4 score (See Supplementary Material section 5 and Fig. S8). Overall, these results suggest that the test-retest reliability of the IDPs is largely independent of mental health change as indicated by the RDS-4.