Cognitive screening instruments for dementia: comparing metrics of test limitation

Abstract

Cognitive screening instruments (CSIs) for dementia and mild cognitive impairment are usually characterised in terms of measures of discrimination such as sensitivity, specificity, and likelihood ratios. However, CSIs also have limitations. Several metrics exist which may be used to denote test limitations but they are seldom examined. Data from several pragmatic test accuracy studies of CSIs were interrogated to calculate various measures of limitation, namely: misclassification rate; net harm to net benefit ratio; and the likelihood to be diagnosed or misdiagnosed. Intra- and inter-test performance for measures of discrimination and limitation were compared. The study found that some tests with very high sensitivity but low specificity for dementia fared poorly on measures of limitation, with high misclassification rates, low net harm to net benefit ratios, and low likelihoods to be diagnosed or misdiagnosed; some had likelihoods favouring misdiagnosis over diagnosis. Tests with a better balance of sensitivity and specificity fared better on measures of limitation. When choosing which CSIs to administer, measures of test limitation should be considered as well as measures of test discrimination. Although high test sensitivity may be desirable to avoid false negatives, false positives also have a cost. Identification of tests having high misclassification rate, low net harm to net benefit ratio, and low likelihood to be diagnosed or misdiagnosed, may have implications for their use in clinical practice.