How reliable is the classification of cognitive impairment across different criteria in early and late stages of multiple sclerosis?

Highlights

• Twenty distinct approaches are in use to classify cognitive impairment.

• The choice of classification criteria is biased by disease duration.

• There is no consensus on how composite scores can be used for classification.

• High classification agreement can be achieved between a variety of strategies.

• The key factor that needs to be homogenized is classification stringency.

Abstract

Background

Prevalence rates of cognitive impairment (CI) in multiple sclerosis (MS) vary between 40% and 80%. Differences in classification criteria for CI may explain this variance.

Objective

This study reviewed and compared classification criteria for CI in patients with early and late MS.

Methods

The paper consists of two parts: a systematic review of published classification criteria and the presentation of new data. Criteria were reviewed in respect to percentage of abnormal parameters and cut-offs concerning standard deviations. Thereafter, criteria were applied to cognitive data of 25 patients with early MS (duration ≤ 2 y), 52 matched patients with late MS (≥ 12 y), and 75 matched controls. The test battery assessed alertness, divided attention, mental flexibility, verbal and visual learning, memory, and visuospatial abilities.

Results

Seventy classification criteria were revealed and grouped into 20 distinct approaches that can be subdivided into three basic classification strategies. Most commonly, CI was defined as performing 1.5 SD or 2 SD below the normative mean in 18–30% of test parameters (n = 42). Other criteria utilized cognitive domains (n = 6), composite indices (n = 8), or combinations of cut-offs and strategies. The stringency of the criteria was correlated with the prevalence rate of CI (r = − .43) and disease duration (r = .48). In the new data, a substantial effect of classification criteria was found with a prevalence rate ranging from 0 to 68% in early and 4 to 81% in late MS. Increased rates of CI in patients vs. controls were found following 18 out of 20 criteria in the sample of late MS. In early MS, an increased rate of CI was only found following a liberal 1.5 SD cut-off criterion. Inter-rater reliability between all criteria was moderate. However, between criteria of comparable stringency the inter-rater reliability was found to be strong.

Conclusion

Classification based on different published criteria is not fully comparable and criteria need to be better homogenized.

Introduction

Cognitive impairment (CI), also referred to as cognitive dysfunction, cognitive deficit, and cognitive disorder, is a common symptom of multiple sclerosis (MS) [1]. It has relevance for many aspects of quality of life, such as maintaining employment, daily-living activities, social life, ability to drive, and benefits from in-patient rehabilitation [2]. Complex attention, information processing speed, verbal and visual–spatial memory, and executive functions are frequently impaired in MS, whereas intelligence, language, semantic memory and attention span are widely preserved.

The first signs of CI can already be found at the time of MS onset [3], [4], [5], [6], [7]. In later disease stages, CI seems to progress in terms of prevalence and severity [8]. The reported prevalence of CI in MS ranges highly, between 40% and 80%. What is the reason for such a large variance?

Two factors that potentially influence prevalence rates are sample composition and neuropsychological assessment [9]. Advanced age, progressive disease course, later disease stage, higher physical disability, fatigue, and depression are sample characteristics that tend to be associated with CI [2]. Psychometrically, the classification of CI is based on the comparison of an individual test result to the mean of a normative sample. Commonly, performance below one, one-and-a-half, or two standard deviations (SD) of the normative mean is used as cut-off for impairment classification. By definition, lower cut-offs (e.g., 1 SD) result in higher prevalence rates of CI than conservative cut-offs (e.g., 2 SD).

In neuropsychological practice however, it is common to use a battery of tests instead of a single test. This also affects the prevalence rates of CI, because the likelihood of irrelevant impaired results rises with an increasing number of obtained scores [10]. Thus, the outcome of CI is highly influenced by the utilized cut-off and the number of tests [11].

In MS, research on cognitive functions focused on finding sensitive and brief test batteries for CI in MS [12], but hardly addressed the issue of classification criteria [13]. Many studies used standard test batteries like the brief repeatable battery of neuropsychological tests (BRB-N) and defined CI as performance below the fifth percentile on about 20% of test parameters or tests [9]. However, more liberal [3], [14], [15] or conservative criteria [16], as well as non-standardized test batteries have also been used. Furthermore, some authors defined CI by employing composite indices or domain specific criteria [17], [18], [19].

To our knowledge, there is no study that compared these different classification strategies. Thus, it remains unclear when to utilize which strategy and if different strategies lead to comparable outcomes. For the first time, a review of common classification strategies for CI in MS is provided and their outcomes are compared. Moreover, we evaluate their diagnostic validity in the cognitive assessment of groups in early and late disease stages as compared to matched controls. The major goal is to promote a more homogenized use of classification strategies that allows better comparability of future studies on cognitive impairment in MS.