Loneliness, Social Isolation, and Effects on Cognitive Decline in Patients with Dementia: A Retrospective Cohort Study Using Natural Language Processing

Study design

The study followed a retrospective cohort design defined through the extraction of information from EHRs collected by Oxford Health NHS Foundation Trust in the United Kingdom. The data is accessible through the UK-CRIS system, maintained by Akrivia Health (https://akriviahealth.com/). The system allows access to structured information, e.g. demographic information and diagnosis code, as well as unstructured textual information, such as clinical records. These documents collect free-text information on the history of mental disorders under treatment, relevant cognitive assessments, and any other clinically relevant discussion between services and support that went on throughout the treatment.

Cohort information

Our cohort included data from all patients with a diagnosis of Alzheimer’s or other forms of dementia (ICD codes: F00 – F00.9, F01, F02, F03, G30, but excluding F06.7 as MCI is rarely followed up in Memory Clinic). The full cohort included 34,469 patients who collectively contributed 6,388,715 medical documents from 06/03/2008 to 25/06/2022. To use the information that resides in rich clinical texts we developed Natural Language Processing (NLP) models that extracted information on cognitive health assessments in dementia and reports of loneliness and social isolation made by patients, carers, and clinical staff.

Cognitive Outcomes

The main analysis used the Montreal Cognitive Assessment (MoCA)²⁷ measure, while in supporting materials we report analysis using Mini-Mental State Examination (MMSE)²⁸ scores. Both measures are widely used tools for assessing cognitive function, particularly in patients with dementia. MoCA detects mild cognitive impairments and early-stage dementia through its heavier emphasis on frontal and parietal function²⁹, while MMSE, even though not as sensitive as MoCA, captures moderate to severe cognitive impairment. Both measures are frequently used in clinical practice and research³⁰. MoCA scores below the cut-off point of 26 points are taken as suggestive of MCI, below 17 as suggestive of moderate impairment, and under 10 as suggestive of severe cognitive impairment. The minimum clinically important difference the smallest change in the outcome that patients would find significant, is reported to be between 0.01 and 2 points depending on the severity of the disease³¹.

Procedure

We used structured and unstructured data from EHRs in this study. The unstructured data covered all textual records for the defined patient cohort, while structured data included information about the sex, ethnicity, and date of birth of patients, their marital and accommodation status, and ICD-10 codes for dementia and depression diagnosis (F32.0 to F34.1). To extract information about the cognition of patients, we used the previously published NLP model³² (for previous work on methodological considerations and description of the mental health EHR data see references^33–37).

NLP model for reports of SI and Loneliness

A novel NLP model was developed for the SI and loneliness reports. The model was implemented in Python and processed textual records for reports of SI and loneliness in two stages: pattern matching and classification stage. In the pattern matching stage, we used a statistical model for word processing from the Spacy library to identify words that describe SI and loneliness. This allowed us to find all documents including expressions such as “loneliness”, “social isolation”, “living alone”, etc. In the next, classification, stage, we used sentence transformer models from Huggingface’s Spacy-Setfit library to process and classify sentences with SI and loneliness mentions. Sentence transformers³⁸ are types of neural network models that produce numerical representations of sentence and paragraph-level linguistic content. This vector space encodes semantic relationships, allowing us to identify and categorise semantically similar sentences. We trained sentence transformers to classify sentences with reports of SI and loneliness into four different categories: a) SI, b) loneliness, c) non-informative isolation, and d) non-informative sentences.

Reports that mention lack of social contact, living alone, and being away from family, or that mention barriers in receiving support from family, were used as an indication of social isolation³. Loneliness was operationalized as consisting of reports on emotional aspects of feeling lonely and suffering due to the lack of social connections³⁹. The non-informative isolation category included reports of temporary and physical isolation (e.g. “isolated fall” or “isolating in the tv-room”), while the non-informative sentences category covered all incorrectly included sentences from the pattern matching stage forwarded to the sentence classification stage (see Table 1 for examples of sentences and NLP categories).

The full model was trained on a randomly selected subset of 11,000 medical documents from the corpus. The terms for the pattern matching stage were derived from a combination of the UCLA Loneliness Scale⁴⁰ and linguistic phrases observed in the training set, specifically those referring to reports of SI and loneliness.

Sentences identified by the pattern matching in the training data were then used to train the classifiers.

NLP accuracy

To evaluate the model’s performance in identifying reports of SI and loneliness, we conducted a manual annotation process using an unseen sample of 5,000 documents. The annotation was performed independently by the first and last authors. In instances of disagreement, the annotations were discussed collaboratively, and a consensus was reached to ensure consistent labelling.

The annotated data served as the ground truth for evaluating the model’s classification performance. Standard classifier performance metrics were employed, including Sensitivity, Specificity, and Balanced or F1 Accuracy. Across four different categories, the NLP model achieved average F1 accuracy of .74, reaching .83 accuracy for SI sentences (Sensitivity: .73 and Specificity: .93) and .91 accuracy for sentences reporting loneliness (Sensitivity: .88 and Specificity: .95). Full Python and R code, detailed measure of model performance, and Sensitivity analysis are reported in Supplemental materials (https://osf.io/2d5jg/).

Once trained, the NLP models were deployed on the data from the full cohort, effectively processing over 6 million medical records. To focus on the symptomatic interpretation of SI and loneliness, we only considered reports that were made 5 years before or after the initial dementia diagnosis. Our data indicates that most of these reports occur several months before and at the time of diagnosis, and that there was a substantial increase in both types of reports during the first year of the COVID-19 pandemic.

Participants

The final data used in this study combined patients with measures of cognitive performance, as measured by MoCA scores, clinical diagnosis information, and loneliness or SI reports, consisting of 4817 patients with 9298 observations. The patient flow chart in Figure 1 outlines the procedure used to derive the sample. Our procedure identified 382 patients (851 observations) with loneliness reports and 523 patients (1185 observations) with SI reports in comparison to the 3912 patients without such mentions, which were defined as a control group in our study (see Table 2 for a split between the groups on main variables). As the retrospective cohort data were collated from EHRs, diversity, equality and inclusion could not be directly addressed during data collation.

Statistical analysis

The data were analysed using a combination of generalised additive mixed-effect modelling (GAMMs)⁴¹ and linear mixed-effect modelling (LME)⁴². GAMM is a non-parametric data-driven method that estimates a non-linear relationship between predictors of interest and outcome variables. Using this model, we estimated and visualised average changes in cognitive function throughout the disease and tested how trajectories differed between patients who reported loneliness and SI in comparison to the control group of patients without such reports in their EHRs. To investigate the parametric effects of individual predictors on the slope of cognitive decline, we used LME.

The primary outcome in the analysis was cognitive function as measured by MoCA and MMSE tests, while we tested the effect of SI and loneliness, and controlled for the effects of Age, Type of Dementia, Sex, Marital Status, Accommodation Status, and whether patients had a diagnosis of Depression in their medical history. In case of missing information for marital and accommodation status, these patients were included in the analysis as Missing categories, while models without these predictors or cases with missing data are reported in the Sensitivity analysis. The random by-patient intercept effects were adjusted in all models, which allowed intercepts to vary for each patient³². The adjustment of the random-effect structure allowed us to model repeated measures of cognitive scores and guards against overfitting of the model.

Specifically, the random structure in GAMM and LME models enables us to weigh intra-individual changes over time and inter-individual differences, allowing us to estimate cognitive trajectories over time. Patients with multiple observations provide more information about the rate and direction of change, while those with fewer observations provide more information about the variability at the group level. In addition to modelling how SI and loneliness are associated with the cognitive ability of patients throughout their disease, we also explored the rate of cognitive change after patients’ first reports of loneliness and SI.

The sensitivity analysis included looking at reports of SI and loneliness in shorter time windows around the diagnosis date, different methods for dealing with missing observations in predictor variables, and MMSE score analysis. We also report analysis focusing on patients with both sets of reports, SI and loneliness, in their EHRs. In comparison to the other three groups, these patients are seen four times more frequently by the health and social services but have fewer measures of cognition, potentially indicating a more complex disease phenotype.

Ethical approvals

We state that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study was approved by the local UK-CRIS oversight committees and the University of Sheffield Ethics application review board (ID: 045869). Individual patient consent was not required for the use of anonymised data. The R and Python code used to analyse the data and develop NLP models is reported in supporting materials.

The effect of loneliness and SI on cognitive trajectories

The results show a significant difference in average MoCA scores between patients with loneliness and SI reports in their EHRs and those without such reports. In the case of patients with loneliness reports, MoCA scores (see Figure 2A) were lower throughout the disease, as illustrated by Figure 2C. When estimated using LME, we see that the scores of patients who reported being lonely are lower by 0.83 MoCA points at the time of dementia diagnosis (see Table 3).

Patients with SI reports have comparable MoCA scores to control patients before being diagnosed. Approximately six months before diagnosis, the slope of MoCA changes for SI patients becomes more severe and they start declining faster than control patients (See Figure 2B and 2D). This pattern is illustrated in the parametric model where the reference group of patients with no loneliness or SI reports show an average decline of –0.38 MoCA points per year (p < 0.001), whereas the significant interaction between SI reports and slope of diagnosis duration shows SI patients decline faster by a further –0.21 MoCA points per year relative to controls (p = 0.029). The interaction between loneliness reports and diagnosis duration was not significant, suggesting rates of decline were similar to that of patients with no SI or loneliness reports (see Table 3).

Cognitive change after first report of SI or loneliness

The change in MoCA scores after the first SI and loneliness report behaves differently between the two groups of patients. In the case of patients reporting loneliness, the cognitive function continues to decline at the same rate as before the report of loneliness (see Figure 3A). The cognitive function of patients experiencing SI improves on average after the first mention of SI (See Figure 3B). Looking at differences in MoCA scores before and after the reports, results show that MoCA scores are on average higher after the report in the SI group (Before = 17.32 versus After = 18.08, t = 2.11, df = 1048.6, p = 0.034), but not in the loneliness group (Before = 17.36 versus After = 17.08, t = –0.67, df = 770.62, p = 0.49). However, when we identify patients with at least one MoCA measure before and after the first report and calculate repeated measure t-tests, we see that both groups of patients decline in their MoCA scores (Loneliness reports: Before = 17.58 versus After = 16.62, t = 2.31, df = 85, p = 0.023 and SI reports: Before = 18.03 versus After = 16.87, df = 115, t = 3.35, p = 0.001).

Mechanisms of SI and loneliness

The conceptualisation of the two concepts, SI and loneliness, likely underlies the estimated differences between the three groups of patients in our study¹. SI represents more of a physical barrier to receiving social support and/or maintaining social connections, while loneliness reflects emotional aspects of this feeling. We found that socially isolated patients have stronger rates of cognitive deterioration several months before a dementia diagnosis while being comparable with controls in the preceding period. In the context of a life-changing and stigmatising diagnosis such as dementia, the lack of social contact prevents patients from receiving needed support, and their cognition declines at higher rates⁴³. When reported for the first time, we show that average values of MoCA scores increase for this subgroup of patients. This improvement may reflect positive action taken by healthcare and/or social services in response to the report. In contrast, patients who experience loneliness have worse cognition throughout the disease course. Such results indicate that loneliness might be an intermediary for depressive symptomatology or is caused by common origins^44–46. This interpretation is also supported by the demographic differences split by group, where we see that patients with reports of SI and loneliness also have higher rates of ICD-10 codes for depression. These patients also do not observe any improvement in their cognition after the first report of loneliness, which might be expected given that improvement in feeling of loneliness requires a change in the availability of social networks, psychological well-being, life satisfaction, activities, and other psychiatric symptoms⁴⁷.

Strengths and limitations

Our study illustrates the potential of medical records from mental health institutions to provide evidence-based results on the effect of symptoms in dementia diseases^9,48. Using large data and advanced statistical modelling, our study shows that SI can be seen as a disease progression factor. Not only is this a novel finding in the domain, contrasting with some previous studies which showed limited effects or their complete absence^20,22,23 but we show differing effects of SI and loneliness on the cognitive trajectories of patients with dementia.

There are several limitations when using large observational datasets⁴⁹. We cannot allocate patients’ membership in the group and can only control for a limited number of factors that could have moderating effects on SI and loneliness, such as depression. Lack of control over the allocation of patients and barriers when accessing healthcare may have led to inadequacies regarding diversity, equality and inclusion, which may reduce the wider generalisability of findings. Similarly, patients with SI/loneliness reports recorded considerably before diagnosis may be a sign of additional health care needs and could additionally limit the generalisability of findings. The correlational nature of the data limits the causal interpretation of our findings, and even though we see improvement in cognition after the first report of SI, we cannot ascertain what change to patients’ social circumstances followed.

There are multiple sources of support that patients can receive after SI is identified, such as closer family connections, social services provision of care or change to their living conditions (e.g. admittance to residential care). Our interpretation of SI relies on a subjective perception from the patient, carer or clinician, rather than an objective measure e.g. of frequency of social contact. Automated and trained NLP model architectures are probabilistic, and even though they achieve high levels of accuracy, they introduce an additional layer of noise to the later data analysis⁵⁰.

Clinical implications

While acknowledging limitations, we show that SI and loneliness could be seen as a disease progression factor for dementia patients, given their effect on cognitive trajectories, even before a formal diagnosis. This means patients experiencing SI or loneliness might benefit from closer monitoring of their cognitive health. While cognitive decline is expected in dementia, the rate of decline can have significant impacts on both patient care and quality of life. By recognising SI and loneliness as potential factors influencing the speed of decline, these findings could have direct implications for clinical practice, such as identifying potential avenues for intervention, informing treatment strategies, and strengthening rationale for social prescribing. We hope that these effects could steer the debate concerning modifiable symptoms, as part of a holistic assessment, that could be used to support the care of patients and outline a research approach that could provide us with more evidence-based studies of modifiable disease progression factors.