ABSTRACT
Importance Isolated rapid-eye-movement sleep behavior disorder (iRBD) is in most cases a prodrome of neurodegenerative synucleinopathies, affecting 1-2 % of middle-aged and older adults, however accurate ambulatory diagnostic methods are lacking. Questionnaires lack specificity in non-clinic populations. Wrist actigraphy can detect characteristic features in individuals with RBD, however high frequency actigraphy has rarely been used.
Objective To develop a machine learning classifier using high frequency (1-second resolution) actigraphy and a short patient survey for detecting iRBD with high accuracy and precision.
Design Analysis of ≥7 nights home actigraphy data and 9-item questionnaire (RBD Innsbruck inventory and 3 prodromal synucleinopathy symptoms) in a dataset including patients with iRBD, sleep clinic patients with other sleep disorders, and community controls.
Setting Single-center study at the Stanford Sleep Center.
Participants Of 108 participants, 83 completed the study, including 41 definitive iRBD cases, and 42 age-and sex-matched controls, including 21 sleep clinic patients with RBD mimics (other parasomnias, sleep apnea, periodic limb movements) or other sleep disorders, and 21 community controls.
Exposure No intervention
Main Outcomes Sensitivity, specificity, accuracy, and precision of classifiers based on actigraphy, on questionnaire data, and both.
Results The actigraphy classifier achieved 95.1% (95 % CI: 88.1 - 98.7) sensitivity and 92.9% (95 % CI: 84.9 - 97.3) precision using all nights recorded, and 88.5 % sensitivity and 90.1 % precision over one night, with performance plateauing after 7 to 10 nights. The questionnaire classifier achieved 91.6 % accuracy and 92.5 % precision, exceeding performance of questionnaire alone. Concordant predictions between actigraphy and questionnaire reached specificity and precision of 100 % (95 % CI: 95.7 - 100.0) with 87.8 % sensitivity (95 % CI: 79.0 - 94.1).
Conclusions and Relevance Actigraphy detected iRBD with high accuracy in a mixed clinical and community cohort. Combined with a short clinical questionnaire, performance reached 100% precision and 87.8% sensitivity. This fully remote procedure can be used to diagnose iRBD in specialty outpatient settings. This cost-effective method has potential for large scale screening of iRBD in the general population.
Question Can ambulatory actigraphy and a short prodromal synucleinopathy questionnaire increase the accuracy of RBD screening?
Findings In a dataset of 41 iRBD cases and 42 clinic and community controls, the combination of actigraphy and a 9-item questionnaire reached specificity of 100 % (95% CI: 95.7 - 100.0) and sensitivity of 87.8 % (95% CI: 79.0 - 94.1).
Meaning Combination of ambulatory actigraphy and questionnaire can be used in both general and specialty outpatient settings for detecting individuals at high risk of having iRBD. This cost-effective, multimodal strategy has potential for large scale screening in the general population.
1 Introduction
Rapid-eye-movement (REM) sleep behavior disorder (RBD) is a motor disinhibition during REM sleep that causes frequent twitches, jerks, and unpredictable, often violent episodes of dream enactment1. In most cases, RBD is associated with an underlying alpha-synucleinopathy, more commonly Parkinson’s disease or Dementia with Lewy bodies2. In this setting, RBD and other prodromes, such as autonomic impairment (constipation and orthostasis) and hyposmia, can precede overt neurodegenerative disease by a decade or more3,4. Early diagnosis of “isolated RBD” (iRBD, formerly called idiopathic RBD)5,6 reduces risk of injuries through implementation of safety measures7 and provides lead time for neuroprotective interventions.
In the general population, the prevalence of iRBD is 1-2 % of middle-aged and older adults. In neurology and sleep clinics, the prevalence may be even higher, however diagnosing iRBD clinically is often challenging. Video-polysomnography (vPSG) provides a definitive diagnosis, but is a complex, time- and resource-intensive procedure, mostly available in academic sleep centers8. Further, an overnight stay in the sleep lab is not acceptable to all patients, and a single night stay may not be sufficient to capture dream-enactment behaviors. RBD questionnaires are a convenient initial step to diagnose iRBD but focus on the behavioral symptoms, which lack specificity since common conditions, such as obstructive sleep apnea, periodic limb movements, and non-REM parasomnias can present similarly. Although significantly higher than in non-clinical settings9–12, the positive predictive value of RBD questionnaires in clinic populations does not exceed 80%, which is arguably insufficient for screening a condition with such serious implications.
Actigraphy can be used to assess nighttime activity. Actigraphs contain accelerometers measuring movement, which have been used for over three decades as a noninvasive method for evaluating sleep and circadian disturbances13,14. However, studies using nocturnal actigraphy show inconsistent performance for distinguishing RBD from other conditions15–17. Limitations may be related to a reductionist approach comparing a single feature of movements (total activity count per night, activity frequency, etc.) while neglecting other characteristics of RBD, including its’ time distribution. One study described patterns of high activity coinciding with REM sleep and a feature called the “short burst inactivity” index, which was found to be elevated in iRBD17. A later study18 found that this feature alone could reach > 80 % accuracy by analyzing data from high-frequency actigraphy in 1- second windows. Conceivably, actigraphy could become a convenient ambulatory procedure in clinic populations with a high pretest probability.
In this study, we tested a new diagnostic procedure of iRBD combining the well-established Innsbruck RBD Inventory, a brief 3-item questionnaire screening for common synucleinopathy prodromes (olfactory loss, constipation, and orthostatic symptoms), and ambulatory high frequency actigraphy assisted by machine learning.
2 Methods
2.1 Participants
Participants were enrolled between April and December 2021. IRBD cases and clinical controls (CL) were recruited in the Stanford Sleep Center, and community CL were recruited via online advertisement. IRBD cases required a definitive diagnosis according to the International Classification of Sleep Disorders, 3rd edition (ICSD-3) criteria19, excluding participants with narcolepsy, overt synucleinopathy, or dementia. For the clinical CL set, goal was to include at least 25 % subjects with NREM parasomnias, 25% with untreated moderate or severe OSA, and 25% with restless legs syndrome (RLS) or PLM >15 per hour of sleep. Clinic CL required unequivocal absence of REM sleep without atonia (RSWA) within 3 years of study participation. Community CL were enrolled consecutively without predetermined ratio of sleep pathology, however, to minimize risk of enrolling participants with unknown RBD, they could not have a history of dream enactment. We estimated a sample size of 40 in each group based on a prior study18. This study was approved by the Institutional Review Board of Stanford and all participants provided written informed consent.
2.2 Procedure
Participants received a study packet containing an AX-6 (Axivity, Ltd) device recording at 25Hz and dynamic range ±8 gravity (g), sleep logs and questionnaires. They were asked to wear the actigraph on their dominant wrist for at least 14 nights, complete sleep diary, parasomnia log reporting any abnormal behaviors during sleep, the Innsbruck RBD inventory (RBD-I) and summary question20, and a novel 3-item questionnaire (3Q) on other prodromal synucleinopathy symptoms: hyposmia, constipation and orthostasis (see eMethods). Participants could continue their usual treatments and mailed back their study packet upon completion of the procedure.
2.3 Actigraphy
Actigraphy data was included in the analysis only if the device recorded at least 7 nights and all questionnaires and forms were fully completed. Data was edited to keep sleep periods based on sleep diaries.
2.3.1 Activity Count
The activity count was calculated in 1-second, 15-second, and 30-second bins according to the procedure shown in eMethods.
2.3.2 Actigraph Features
A set of features were derived to best capture activity that distinguish iRBD cases from CL. These include:
Features derived from the sleep diary: Total sleep time (TST); wake after sleep onset (WASO); sleep efficiency (SE), TST / (TST + WASO); sleep onset; and sleep offset.
Features from prior studies17,18, with varying parameters enabling a more data-driven search for optimal accuracy: Activity index (AI), % x-second epochs with any activity for x ∈{10, 30, 60}; short immobile bouts (SIB), (# zero-activity periods larger than x seconds and less than 60 seconds) / TST for x ∈{0, 1, 5}; and mean motor activity score (MMAS), average activity count.
Novel features that capture new RBD activity patterns, based on visual interpretation of the data from the first 15 consecutive cases and 5 clinic CL: Twitch activity (TA), (# activity counts with larger than 0 and smaller than x and with surrounding zero-activity) / TST for x ∈{0.5, 1, 1.5}; long immobile bouts (LIB), (# zero-activity periods longer than x seconds) / TST for x ∈{60, 120, 300}; Hjorth parameters of activity counts [Hjorth activity (HPA), Hjorth mobility (HPM), and Hjorth Complexity (HPC)]21.
2.3.3 Feature windows
Loss of skeletal muscle atonia and propensity for sudden motor activity during REM sleep would be expressed as periods of higher activity counts on actigraphy. However, frequent movements can also occur in NREM sleep due to PLM and OSA, and during brief periods of wake as sleep drive gradually reduces. Actigraphy cannot distinguish between sleep stages. To minimize the risk of analyzing NREM and wake periods, and maximize likelihood of including one to two REM cycles, we applied an empiric time window labelled “possible REM sleep” (pREM) starting 70 minutes, and ending 200 minutes after reported sleep onset. We also labeled periods as “wake” based on sleep diaries (see Figure 1). We computed features based on data within these windows by: 1) excluding or keeping wake windows; 2) excluding, keeping, or only keeping pREM. In total, this corresponds to 6 feature variations.
The data displayed are from one control and one iRBD case. The density of twitch activity is visualized by applying a moving average filter of 61 seconds to events of isolated activity counts less than 0.5 in amplitude. pREM: probable REM sleep window (70-200 minutes after estimated sleep onset)
2.4 Classification Models
We developed a framework for classification of iRBD vs. non-iRBD based on a single night of actigraphy which outputs an iRBD prediction score (“per night” prediction). Subsequently, all output iRBD scores for a same subject can be averaged, resulting in a single prediction (“per subject” prediction). Finally, actigraphy prediction could be compared to questionnaire-based prediction.
We compared the traditional single-feature approach against a large set of features in a machine learning model (multi-feature model). Specifically, we fitted a threshold for each feature and an ensemble of decision trees using all derived features (see eMethods). For comparison with prior methods, the single feature selected would be the SIB17,18, fitted to our data based on the lowest p-value in group comparisons using Mann-Whitney tests.
Similarly, this approach was applied to questionnaire data, which contained nine items: the Innsbruck summary question, five RBD-I items, and three 3Q items for hyposmia, constipation, and orthostasis. Machine learning models were optimized for the 3Q only, the Innsbruck RBD-I and summary question only, and all questions together. In case of “don’t know” answers, a value of 0.5 was imputed for the boosted decision trees. The decision trees are invariant to the exact imputation value if it is between 0 and 1.
2.4.1 Validation
The classifiers were fitted and evaluated in a leave-one-out cross validation setup. Iteratively, all nights of actigraphy from a participant were held out while fitting, subsequently, the model was validated in the held-out data. Hence, each model was fitted as many times as the number of participants, resulting in an unbiased performance estimate in all subjects.
Once both actigraphy and questionnaire models were developed and validated, a two-dimensional model was created. This 3-class classifier would output iRBD, non-iRBD, or “undetermined diagnosis”, in case of concordant positive, concordant negative, or discordant prediction, respectively.
2.5 Statistical Tests
Statistical analyses were performed in Matlab (R2019b, The Mathworks). Group comparisons were carried out using chi-squared test for nominal variables and Mann-Whitney tests for numerical variables. Classifiers were evaluated using sensitivity, specificity, precision, and accuracy. Additionally, receiver-operating-curves (ROC) analyses were performed to calculate the area under the ROC curve (AUC). Performance confidence intervals were estimated using Clopper-Pearson’s method.
Correlations were evaluated between individual actigraphy features as well as overall actigraphy prediction score and RSWAi, PLMi, AHI, sleep diagnoses, and RBD treatments.
Performance was evaluated as a function of number of nights recorded with actigraphy. Due to the wide range of nights recorded, we restricted this analysis to recordings with at least 14 nights. Thereby, the sample we consider in this analysis do not change depending on the number of nights analyzed. Confidence bounds were estimated using bootstrapping in 1000 iterations, i.e., for each participant, the performance was evaluated in subsets of nights that were randomly sampled.
3 Results
Out of the 108 participants (52 iRBD cases, 56 CL) enrolled, 25 were excluded: 8 due to an uncertain diagnosis, 3 with clinical RBD but RSWA index below cutoff and 4 with clinical RBD without recent vPSG, one with definite RBD reportedly symptom-free since receiving sodium oxybate. Additionally, 7 did not comply with instructions, 4 wore defective or incorrectly configured devices, 4 withdrew from the study, and 2 never received a study packet. The final dataset used for the analysis comprised 83 participants: 41 iRBD cases and 42 CL, including 21 clinic and 21 community CL (Table 1). No between-group differences were observed for age, sex, and BMI. Clinic CL included 5 with NREM parasomnias, 7 untreated OSA (4 moderate, 3 severe) and 6 with RLS/PLM. Cases and CL sets were similar in terms of usage of PAP device and AHI. As expected, iRBD cases had higher PLMi and were more likely to use melatonin and clonazepam than CL. Other RBD treatments included transdermal rivastigmine and pramipexole. Antidepressant use was more frequent in iRBD.
On average, 17.5 ± 8.1 and 25.5 ± 8.0 nights (SD) were recorded in the CL and iRBD groups, respectively. Due to the heterogeneity within each group, the classification performances were evaluated as a function of number of nights.
3.1 Actigraphy features and questionnaire
Two features were found to have higher p-values than prior feature SIB: LIB, TA (see eTable 1 for details).
All questions of the RBD-I and 3Q showed significance (p-value < 0.05). Among all questions, the most accurate was RBD-I item 3 (sleep-related extensive movements) with 93.5 % accuracy (eTable 2).
3.2 iRBD Classification
The multi-feature actigraphy classifier achieved best performance using 1-second windows, with 94 % (95% CI = 86.5 - 98.0) accuracy and 92.9% (95% CI = 84.9 - 97.3) precision (AUC 0.979) analyzing all nights (see Fig. 2 and Table 2). Optimizing threshold (eTable 3) improved accuracy to 95.2 % using all nights. Analysis in 15- or 30-second windows both yielded a lower accuracy of 91.6 % accuracy (eTable 4 and 5).
The ROC curves for classification using actigraphy are generated per subject and per night for the boosted decision trees (multi-feature) and SIB (single-feature). ROC: receiver-operator-characteristics; iRBD: isolated REM sleep behavior disorder; SIB: short immobile bouts; AUC: area under the ROC curve; RBD-I: Innsbruck RBD inventory; 3Q: 3 prodromal symptom questions.
In comparison, the single-feature SIB classifier achieved 83. 1% accuracy and 78.7 % precision (AUC 0.922) with all nights, after optimization using the parameter (x = 5) and a median cut-off 0.379 across cross-validation folds.
The machine learning classifier using all questionnaire data (6-item RBD-I + prodromal 3Q) performed with 91.6 % accuracy and 92.5 % precision (AUC 0.954). Restricting the analysis to either the RBD-I or 3Q achieved 89.2 % and 81.9 % accuracy, respectively.
The two-dimensional model combining actigraphy and questionnaires achieved 87.8 % sensitivity and 100 % precision (see Table 2 and Figure 3). Predictions were concordant in 90.2 % of cases and 85.7 % of CL. In 9.8 % (n = 4) of cases, the output diagnosis was undetermined, and 2.4 % (n = 1) of cases were misdiagnosed as non-iRBD. In 14.3 % (n = 6) of CL, the output diagnosis was undetermined. No CL was misdiagnosed as RBD.
The dashed lines indicate the default decision lines for each classifier and the gray area signifies “undetermined diagnosis”. The size of each marker is scaled to the number of nights recorded with actigraphy. The iRBD prediction score from questionnaire used the Innsbruck RBD-I and 3 prodromal symptom questions.
Analysis of incremental performance over multiple nights showed gain in sensitivity and specificity over 7 and 10 nights, respectively, after which performance only marginally improved (eFig. 1).
3.3 Interpretation of models
In the iRBD group, OSA treated with PAP therapy correlated with lower actigraphy RBD prediction score (r = -0.42, p = 0.022) (eTable 6). In the CL group, hypersomnia and insomnia correlated with higher RBD prediction score (hypersomnia: r = 0.36, p = 0.019; insomnia: r = 0.35, p = 0.018). No correlation was found between RSWAi or number of RBD episodes, and prediction score.
TA and LIB were the two features with highest importance in the multi-feature actigraphy model (eFig. 2). None of the feature masks had more importance than the whole-night window.
In the questionnaire model, the RBD-I item 1 (violent/aggressive dream content), showed the highest importance (eFig. 3).
4 Discussion
IRBD is rarely diagnosed before injuries to self or bed partners occur or the disorder converts to a defined synucleinopathy. Even for sleep experts, diagnosing RBD based on history alone can be challenging, and vPSG is not easily accessible or accepted by patients. In this study, we developed a multimodal iRBD detection framework combining a 9-item questionnaire screening for signs of RBD and prodromal symptoms of synucleinopathy with ambulatory actigraphy, in a dataset of cases and CL from the sleep clinic and community.
Using an ensemble of decision trees, actigraphy detected iRBD with 95.1% sensitivity and 92.9% specificity. Concordant iRBD prediction between actigraphy and questionnaire models had 87.7 % sensitivity and 100 % precision with performance plateauing out after 7 to 10 nights. Analysis in 1-second windows raised accuracy to 94.0 %, from 91.6 % in 15-and 30-seconds windows.
We found that single-feature actigraphy using SIB had lower accuracy close to 80 %, consistent with prior report18. We also designed two actigraphy features with higher discriminative value, LIB and TA. Video-based studies show that in RBD a majority of movements are simple, non-purposeful twitches more prominent in the arms22–25. We found that the periodicity of movements detected at the wrist is variable, unlike in other pathologies.
No relation was found between RSWAi or clinical RBD activity, and iRBD prediction scores. This could be due to the small sample size, or to a more complex relation between RSWA, covert (small twitches), and overt (clinical episodes) manifestations in RBD. If the detection model truly operates independent of the RSWA burden and clinical severity, it could potentially identify individuals with borderline RSWA, or with RSWA but no overt manifestation. This entity coined “prodromal RBD” is of great importance as it carries a high lifetime risk of synucleinopathy but is rarely detected26. The 3 CL misclassified by actigraphy included one with insomnia and OSA on PAP therapy, one with untreated REM-related OSA (apnea-hypopnea index of 19), history of leg kicking in sleep with PLM index of 8, and one community CL with occasional sleep talking but no vPSG data. It shows that actigraphy has possible RBD mimics that would be difficult to exclude with this modality alone.
The comprehensive questionnaire model achieved 91.6 % accuracy, outperforming RBD-I and 3Q prodromal questionnaire alone (89.2 % and 81.9 %, respectively). RBD-I’s performance was similar to a prior study17. Finding that RBD-I item 3 (sleep-related extensive movements) alone exceeded full RBD-I’s performance was not expected and may be due to greater insight and disease severity in iRBD patients established at the Stanford sleep clinic. Twelve percent of non-RBD individuals endorsed at least one synucleinopathy prodrome, which is consistent with population studies27,28.
Altogether, our findings suggest that a comprehensive iRBD questionnaire that includes disease prodromes other than RBD improves current clinical diagnostic procedure. A two-step approach using questionnaire first, followed in those who screened positive by ambulatory actigraphy, could select individuals at highest risk of iRBD, and thereby result in higher positive predictive value (100 % in this small sample of 83 subjects), before confirmatory vPSG. Although performance may be lower in subjects with a high prevalence of sleep disorders, it is expected to perform as well or better in non-clinical populations expected to have less sleep comorbidities than patients at a sleep clinic. Future studies are needed to test this ambulatory screening procedure in large sample of the general population.
Limitations in this study include its small sample size, which could reduce the robustness of our findings. Clinic CL had a relatively low representation of “pseudo-RBD” secondary to OSA, and post-traumatic stress disorder (PTSD), which can also mimic RBD. The study was not powered to thoroughly evaluate performance across sleep disorders.
The RBD set, in turn, was well-characterized and consistent with the classic description of “idiopathic” RBD, associated in 85 % of cases with other disease prodromes, and vPSG finding of prominent upper extremity and chin EMG augmentation. One misclassified patient was a female with no other prodromes of synucleinopathy and near-complete absence of arm EMG elevation on vPSG, raising the possibility that upper extremity RSWA is necessary for actigraphy detection. The second was a man with borderline RSWA. Studies are needed to explore the relation between severity and topographic distribution of RSWA, and performance of wrist actigraphy. Studies should also be conducted to test the utility of the actigraphy model for detection of RBD associated with overt synucleinopathy, or other RBD phenotypes such as RBD associated with narcolepsy and post-traumatic stress disorder.
5 Conclusion
We found that ambulatory wrist actigraphy complements screening questionnaires for iRBD, and that their combination predicts RBD with high accuracy and precision. We described several actigraphy markers of RBD, which could be calculated using any wearable containing high-frequency accelerometers. Our detection paradigm has the potential to be used in a staged approach, using questionnaires first, followed by actigraphy in positive screens, to select individuals with high pretest probability before confirmatory vPSG. This algorithm could be implemented fully remotely in outpatient clinics. The study also lays the ground for future population studies aiming at developing precise and cost-effective screening strategies for iRBD in the general population.
Data Availability
All data produced in the present study are available upon reasonable request to the authors-
6 Acknowledgements
This study was supported by Feldman Foundation CA. Additional support to Andreas Brink-Kjaer was provided by the Stibo, Oberstloejtnant Max Noergaard & Hustru Magda Noergaards, Otto Moensted, Augustinus, Knud Hoejgaard, William Demant, Vera & Carl Johan Michaelsens, Tranes, Marie & M.B. Richters Fond, and IDAs & Berg-Nielsens foundations.
6 References
