Abstract
Prosthetics and orthotics have been recognized for decades as a potential means to restore hand function and independence to individuals living with impairment due to stroke. However, 75% of stroke survivors, caregivers, and health care professionals (HCP) believe current practices are insufficient, specifically calling out upper extremity as an area where innovation is needed to develop highly usable prosthetics/orthotics for the stroke population. A promising method for controlling upper limb prosthetics is to infer movement intent non-invasively from electromyography (EMG) activity. While this approach has garnered significant attention in the literature, existing technologies are often limited to research settings and struggle to meet stated user needs. To address these limitations, we have developed the NeuroLife® EMG System which consists of a wearable garment, similar to a compression sleeve, worn on the forearm with 150 embedded electrodes spread across the forearm, and associated hardware and software to record and decode high-resolution surface EMG. We demonstrate that the NeuroLife EMG System can accurately decode 12 functional hand, wrist, and forearm movements, including multiple types of grasps from participants with varying levels of impairment, with an overall accuracy of 77.1±5.6% in ideal scenarios and 74.7±5.0% in simulated real-time situations. Importantly, we demonstrate the ability to decode movements that participants are unable to overtly perform, showing potential as a control mechanism for assistive technologies. Feedback from stroke survivors who tested the system indicates that the design of the sleeve meets various user needs including being comfortable, portable, and lightweight. The sleeve is in a form factor such that it can be used at home without an expert technician, can be donned with the help of a caretaker, and can be worn for multiple hours without discomfort. Taken together, the NeuroLife EMG System represents a platform technology to record and decode high-definition electromyography for the real-time control of assistive devices in a form factor designed to meet user needs.
Introduction
Stroke is a leading cause of long-term physical disability in the United States, affecting more than 800,000 people per year [1]. Unilateral paralysis (hemiparesis) affects up to 80% of stroke survivors, leaving many to struggle with activities of daily living involving manipulating objects such as doors, utensils, and clothing due to decreased upper-limb muscle coordination and weakness [2]. Restoration of hand and arm function to improve independence and overall quality of life is a top priority for stroke survivors and caregivers [3]. Intensive physical rehabilitation is the current gold standard for improving motor function after stroke. Unfortunately, 75% of stroke survivors, caregivers, and health care providers report that current upper extremity training practice is insufficient [4]. The development of user-centric neurotechnologies to restore motor function in stroke survivors could address these unmet clinical needs through a range of different mechanisms such as improving motivation, enhancing neuroplasticity in damaged sensorimotor networks, and enabling at-home therapy.
Assistive technologies (AT) have been recognized for decades as a potential means to restore hand function and independence to individuals with paralysis [5]. ATs, including exoskeletons and functional electrical stimulation (FES), can assist with opening the hand and also evoke grips strong enough to hold and manipulate objects [6]. Additionally, these systems have been used therapeutically during rehabilitation to strengthen damaged neural connections to restore function [7]. A wide variety of mechanisms to control ATs have been investigated including voice [8], switch [9], position sensors [10], electroencephalography (EEG) [11], electrocorticography (ECoG) [12], intracortical microelectrode arrays (MEA) [13], and electromyography (EMG) [14]. Unfortunately, no single system has simultaneously delivered naturalistic use paired with an intuitive, user-friendly, and high degree-of-freedom (DoF) control that people can use functionally in real-world settings.
Recent advances in portable, high-definition EMG-based (HDEMG) systems have the potential to overcome several of these barriers and deliver an intuitive and fully non-invasive AT control solution. While there are various EMG-based ATs currently available, these systems use a small number of electrodes and rely on threshold-based triggering, consequently limiting the DoF of the system [15]. Conversely, HDEMG systems leveraging machine learning approaches to infer complex movement intention can provide high DoF control. Currently, these systems are largely confined to the research setting [16]. As such, these systems have usability limitations including being difficult to set up, requiring manual placement of electrodes, and are non-portable, and bulky which can hinder successful translation of effective and useful technologies.
To address these limitations, we have recently developed the NeuroLife EMG System to decode complex forearm motor intention in chronic stroke survivors while simultaneously addressing end user needs. The NeuroLife EMG System was designed to be used as a control device for various end effectors such as FES systems and exoskeletons. Additionally, the system was designed to meet user needs in the following domains that have previously been identified as high-value for stroke survivors: donning/doffing simplicity, device setup and initialization, portability, robustness, comfortability, size and weight, and intuitive usage [4]. The sleeve is a wearable garment consisting of up to 150 embedded electrodes that measure muscle activity in the forearm to decode the user’s motor intention. A single zipper on one edge of the sleeve allows for a simplified and streamlined donning and doffing by the user and/or a caregiver. The sleeve design facilitates an intuitive setup process as embedded electrodes are consistently placed on the arm eliminating the need for manual electrode placement on specific muscles. The lightweight stretchable fabric, similar to a compression sleeve, was chosen to enhance comfort for long-term use. Overall, these design features of the sleeve help address critical usability factors for ATs [4].
In this work, we demonstrate that the NeuroLife EMG System can extract task-specific myoelectric activity at high temporal and spatial resolution to resolve individual movements. Based on EMG data collected from seven individuals with upper extremity hemiparesis due to stroke, trained neural network machine learning models can accurately decode muscle activity in the forearm to infer movement intention even in the absence of overt motion. We assess the viability of this technique for real time decoding, demonstrating the feasibility of controlling ATs based on motor intention using the NeuroLife EMG System. Finally, we present usability data collected from the study participants that highlight the user-centric design of the sleeve. This data will be used to inform future developments to ultimately deliver an effective EMG-based neural interface that meets end user needs. Together, we present a capable, user-centric EMG-based neural interface for the detection of motor intention after stroke, with usability guidelines that will inform future device improvements.
Methods
Participants
Eight individuals with a history of stroke were recruited to participate in a set of experiments to record EMG using the NeuroLife EMG System while attempting various hand and wrist movements. Additionally, data was collected from six able-bodied individuals to serve as a general comparison of EMG data, and to benchmark decoding algorithms. Data was collected as part of an ongoing clinical study being conducted at Battelle Memorial Institute and approved by the Battelle Memorial Institute Institutional Review Board (IRB No. 0779-100135968 and IRB No. 0773, respectively). Informed consent was obtained from all participants prior to any experimental procedures. Demographics on study participants with stroke are provided in Table 1, and for able-bodied participants in Supplementary Table 1. Specific inclusion and exclusion criteria are listed in the Supplementary Methods.
During the first session prior to EMG data collection, standardized clinical assessments were performed by a licensed occupational therapist in all participants with stroke. These included the upper extremity section of the Fugl-Meyer (UE-FM) to assess upper extremity motor impairment, the Box and Blocks test to assess manual dexterity, and the Modified Ashworth test to assess spasticity of the finger, wrist, and elbow flexors. Based on predetermined exclusion criteria, one participant was removed from data analysis due to hemispatial neglect affecting their ability to consistently follow movement cues.
Experimental Setup
Participants were seated facing a computer monitor with their arms placed on a table, and the sleeve was placed on the paretic arm for participants with stroke, and the right arm for able-bodied participants, regardless of handedness (Figure 1). The sleeve comprises a stretchable fabric with an embedded array of electrodes (Supplementary Figure 1). Depending on the forearm size of the participant, a small, medium, or large sized sleeve was used containing 128 electrodes (64 channel pairs), 142 electrodes (71 channel pairs), or 150 electrodes (75 channel pairs) respectively. Each electrode is 12mm diameter, spaced 25mm apart, and wrap the forearm from elbow to wrist. With a flexible and lightweight nylon-Lycra hybrid material, the sleeve wears like a compression sleeve and weighs 180, 195, and 220 grams for the small, medium, and large sleeves, respectively. A zipper on the ulnar edge of the sleeve allows for easy donning and doffing. Prior to donning, an electrode solution spray (Signaspray, Parker Laboratories, Fairfield, NJ) was applied to the participant’s forearm to improve signal quality. Bipolar EMG signals were sampled at 3KHz using an Intan Recording Controller (Intan Technologies, Los Angeles, CA). An embedded electrode in the sleeve near the elbow was used as a reference for all bipolar amplifiers.
The participants were instructed to attempt a series of hand, wrist, and forearm movements. A series of images of the desired hand movement was presented on a computer monitor, and the participants were instructed to attempt to match each movement shown, to the best of their ability. Additionally, participants were instructed to attempt the movement at 25-50% maximal effort to minimize co-contraction and muscle fatigue throughout the session.
The following movements were collected during the session: Hand Flexion, Hand Extension, Index Extension, Thumb Flexion, Thumb Extension, Thumb Abduction, Forearm Supination, Forearm Pronation, Wrist Flexion, Wrist Extension, Two Point Pinch, and Key Pinch. These movements were identified by a licensed occupational therapist as highly relevant functional movements for dexterous hand use, and these movements have been used in similar studies [17]. Recording blocks consisted of a single movement repeated 10 times (referred to as “single blocks”), or multiple movements repeated within a single recording session (referred to as “mixed blocks”). Every block began with an 8s rest period, followed by alternating movement and rest periods. During mixed blocks, a collection of movements (e.g., hand flexion, hand extension, forearm supination) were randomly presented to the participant. Before beginning the block, participants were shown the movement(s) in the upcoming block. For participants with stroke, the time for each movement was randomly selected from a uniform distribution between 4-6s, and rest time was randomly selected between 4-6s. For able-bodied participants, the movement and rest times were both set randomly between 2-3s. The cue and rest times were shortened in able-bodied participants due to faster movement times and simpler decoding compared to the participants with stroke. In the last recording session, a usability questionnaire assessing user needs (adapted from [4]) was given to subjects to evaluate the usability of the current sleeve design (responses from participants are presented in Supplementary Table 3).
Data was collected across 3-4 sessions with each participant with stroke, and in 1 session with able-bodied participants. Each session was <2 hours. For participants with stroke, data from all sessions except the last half of the final session were used to train the classifiers. Total amount of training data per movement for participants with stroke are shown in Supplementary Figure 3. For able-bodied experiments, data was collected in a single session with a total of 10 repetitions for each movement with the first 5 repetitions used for training, and the last 5 repetitions used for testing corresponding to 7.5 seconds of data used for both training and testing.
Each movement category for all participants were scored based on a scoring scheme adapted from the Action Research Arm Test (ARAT) [18], and an “observed movement score” was assigned. The scale had the following categories: 0=no movement; 1=incomplete range of motion; 2=complete range of motion but impaired; 3=normal.
Pre-processing, windowing, and feature extraction
The EMG data was bandpass filtered between 20-400Hz using a 10th order Butterworth filter, and a 60Hz notch filter was applied similar to previous studies [19]. Following pre-processing, the root mean square (RMS) was extracted using 100ms data windows with no overlap. An example of the RMS extraction for multiple movement sets are shown in Figure 2A,B. For decoding of movement intent during a given time window, the current window and three preceding windows were used, totaling 400ms of RMS data used for each prediction. Next, the training data was normalized (mean=0, variance=1) and the testing data was normalized using the mean and variance from the training data.
Classification of movement intention in stroke participants was performed in two different ways: (1) using the 2.5s center during a cue or rest period, or (2) on the continuous timeseries data. For the 2.5s center window method, the middle 2.5s of each cue and rest period during a block was extracted (Figure 3A). This resulted in a total of 22 predictions of 100ms binned RMS data per cue (2.5s with the first three 100ms bins removed for containing out of window data at the beginning of the cue). This method was applied to both the training and testing datasets to reduce noise by removing the transition periods, similar to previous studies [20]. This dataset was used to compare across models. In able-bodied subjects, classification was performed as described above but using the 1.5s center during a cue or rest period, resulting in a total of 12 predictions. These data are presented in Figure 3 for participants with stroke, and Supplementary Figure 5 for able-bodied participants.
For decoding of continuous timeseries data, we performed a dynamic cue shifting technique to account for cue onset and offset variability. Latency between cue onset and the onset of EMG activity is a persistent problem within decoding that can lead to significant deficits in algorithm performance and is exacerbated in data recorded from participants with neurological impairments such as stroke. Traditionally, these onset and offset variabilities are handled by shifting cues a predetermined amount of time by observing an average latency for a participant or can be dynamically assigned for each cue manually [21]. However, these methods still fail to capture the full distribution of onset and offset variability. Here we use an automated approach to dynamically shift cue labels to match the EMG activity. The average EMG signal was aligned with the intended cue times, and residuals were calculated between the EMG signal and the signal mean for each cue segment. Cue timing was then iteratively optimized to minimize the sum of squared residuals (Supplementary Figure 4). Cue timings were delayed up to a maximum time of 2s beyond the intended cue time. These data are presented in Figure 4.
Classification
Classification was performed using all recording blocks (single and mixed). Importantly, the testing consisted of the final 4 recording blocks (consisting of 5 repeats of each of the 12 movements) of data collected for that subject. In other words, none of the training set occurred later in time than the testing set to prevent data leakage of time dependent signal fluctuations that can significantly influence decoding performance.
Three classifiers were compared, including a logistic regression (LR) model, support vector machine (SVM), and a neural network (NN). For the LR and SVM models, data was additionally preprocessed using principal component analysis for dimensionality reduction, keeping components that accounted for >95% of the variance. LR and SVM models were trained using the sci-kit learn toolbox in Python 3.8. To optimize hyperparameters for both LR and SVM, a grid search on the training data with 5-fold cross validation was applied to tailor a specific model for each subject. Hyperparameter C was varied from 1e-4 to 1e4 for LR, and hyperparameters C and Gamma were varied from 1e-4 and 1e4 for SVM. The best performing model hyperparameter combinations were used for eventual testing.
The NN was developed in Python 3.8 using the FastAI package [22]. FastAI defaults were used for training except where noted. The model architecture takes an input of a flattened N channels x 4 array from the N channels of the sleeve and 4, 100ms windows of mean RMS signal. The input layer connects to two fully-connected dense layers, with size 1000 and 500 respectively, with batch normalization and the ReLU activation function between layers. The final layer had 13 classes corresponding to the 12 cued movements and rest. Finally, a Softmax activation function was applied to the model outputs to provide prediction probabilities for each of our movements. The predicted movement for a given time point was the movement with the greatest prediction probability. The training procedure used label smoothing cross entropy loss (p=0.9) and the Adam optimizer. During training, dropout was applied to each layer with 20% probability to prevent overfitting. The learning rate was optimized using the FastAI learning rate finder tool [23]. Each model was trained for 400 epochs with early stopping criterion, using the one cycle training policy from FastAI.
Accuracy is defined as the percentage of RMS bins predicted by the classifier to be the same as ground truth. In addition to accuracy, we also present success rate as a decoding performance metric. An informative metric for assistive device use is the percentage of correctly predicted events, and success rate represents an approximation for an observer rating each predicted movement as a success or failure. An event is considered successful and contributes to success rate if a minimum of 1s continuous period within a cue that is decoded as the intended movement.
Statistics
All comparisons were planned in the experimental design a priori. Significant differences were determined using paired t-tests (Figure 3D, 4A) and unpaired t-tests where appropriate. Significant differences for multiple comparisons were determined using one-way ANOVAs followed by Tukey HSD tests (Figure 3D). Alpha of 0.05 were used for single comparisons. Statistical tests for each comparison are noted in the text. Statistical analysis was performed in Python 3.8 using SciPy and Statsmodels. In all figures, * indicates p<0.05, ** indicates p<0.01, and *** indicates p<0.001. Error bars indicate mean ± SEM in all figures.
Results
Decoding hand gestures using the NeuroLife EMG System
To demonstrate the utility of the NeuroLife EMG System to sense and interpret muscle activity in the forearm, we first assessed the ability to decode hand, wrist, and forearm movement intention. Participants were guided through various blocks of movements and EMG data was recorded. To extract the most stable segment of the signal, the middle 2.5-seconds of each cue and rest period was isolated to remove transition periods (Figure 3A). By removing the transition periods and focusing on periods of consistent activity we are left with a standardized dataset to compare performance of various models.
Heatmaps of EMG activity across the sleeve are shown for one participant with stroke (Figure 3B). These heatmaps highlight the visual differences between forearm EMG activity across the various movements. Importantly, EMG activity is less localized in the heatmaps of participants with stroke compared to the able-bodied (Supplementary Figure 2). These results are consistent with previous reports of lack of independent muscle control in stroke [24].
Decoding of EMG activity can be performed using a variety of different techniques, we therefore evaluated 3 commonly used machine learning decoding approaches including a logistic regression (LR) [25], support vector machine (SVM) [26], and neural network (NN) [27]. To validate our decoding pipeline, we tested decoding performance in able-bodied subjects with the expectation of highly accurate decoding. Overall, the NN obtained 96.8±0.5% accuracy and outperformed LR and SVM models which had 91.5±0.8% and 90.8±1.2% accuracy respectively (Supplementary Figure 5; Able-Bodied Model: paired t-test NN vs. LR, p=5.8 × 10−5; NN vs. SVM, p= 1.6 × 10−3). These decoding results are consistent in the dataset comprised of participants with stroke, where the NN obtained 77.1±5.6% accuracy, and outperforms the LR (69.0±5.4%) and SVM models (66.6±6.9%) (Figure 3D; Stroke Model: paired t-test NN vs. LR, p=9.1 × 10−4; NN vs. SVM, p=9.3 × 10−3). In summary, the NN outperforms the LR and SVM when decoding forearm EMG activity to infer movement intention.
Next, we investigated the influence of movement quality on decoding performance.Generally, decoding performance improved as the observed movement score increased (Figure 3E; Observed Movement Score, One-way ANOVA, Accuracy (%): F[3, 80] = 13.38, p= 3.7 × 10−7). A comparison of decoding accuracy based on movement score was computed using a Tukey HSD test (Supplementary Table 2). Separating movements with visible motion (score >= 1), the overall decoding accuracy increases to 85.7±3.2% with the no movement accuracy at 27.3±3.2% (Movement Ability: Movement score=0 vs. Movement score=1-3: unpaired t-test, p=3.9 × 10−9).
Next, we investigated decoding performance of individual movements in the dataset without transition periods. The confusion matrix with individual movements for one subject is shown in Figure 3C. The best performing movements across subjects were Rest, Wrist Flexion, and Index Extension with an average accuracy of 93.3±6.5% (Figure 3C, right column). On average across subjects, the worst performing movements were Forearm Supination and Thumb Abduction, with an average accuracy of 39.4±9.9% (Supplementary Figure 6).
We also investigated using a success metric to provide a more intuitive metric for evaluating eventual control of assistive devices. The success rate metrics are meant to approximate an observer scoring each movement cue as a binary success or failure. An event is considered successful if a continuous 1s period is correctly identified within the cue or rest period. The success rate per movement type for one subject is presented in the right column of the confusion matrix (Figure 3C). Overall grand average success rate across all movements achieved 75.9±4.2%. The top three movements had an average success rate of 93.2±7.6% (Successes/Attempts; Rest: 409/425, Index Extension: 23/30, Wrist Flexion: 20/30), with the bottom two movements obtaining a much lower success rate of 36.0±11.5 (Successes/Attempts; Forearm Supination: 14/43, Thumb Abduction: 17/43).
Decoding continuous data in simulated real-time
To demonstrate the utility of the NeuroLife EMG System to interpret muscle activity from the forearm to act as a control signal for assistive devices, we next tested our decoding algorithms in simulated real-time scenarios. Following a stroke, the ability to contract and relax muscle groups is slowed and highly variable [28], which consequently makes automated labeling of cues using a static time shift (e.g., 300ms) for training machine learning models imprecise. To account for this cue onset and offset variability, we first performed a dynamic cue shifting technique to automatically shift cue labels to match EMG activity (Supplementary Figure 4). To verify this method, we compared decoding performance with and without cue shifting. Cue shifting significantly improved decoding performance from 65.6±5.9% to 74.7±5.0%% accuracy (Figure 4A; Cue Shift: Static vs. Dynamic: paired t-test, p=0.023). This improvement is likely due to 1) improved accuracy of the timing of cue onset and offsets in the training data which gives a better representation of each movement and thus better decoding performance, and 2) more accurate testing alignment and better testing parameters. These results suggest that cue labeling can substantially affect overall decoding performance in real-time decoders, and intelligent cue labeling can improve overall performance.
We investigated decoding performance of individual movements in the simulated real-time dataset. The confusion matrix with individual movements for a single subject is shown in Figure 4C. The best performing movements across subjects were Rest, Wrist Flexion, and Wrist Extension, with an average accuracy of 89.7.7±7.3% for these movements. The worst performing movements across subjects were Forearm Supination and Thumb Abduction, with an average accuracy of 29.5±9.0%. A continuous time series plot of movement probabilities is shown in Figure 4D. Shaded regions indicate the cued movement with the probability of the movement type decoded based on motor intention.
Usability
Usability is a critical factor in the long-term adoption of an AT. Inconveniences of setup and comfort, as well as frustrations with reliability can often lead to eventual device abandonment. Therefore, in our final EMG recording session with each participant, we collected initial usability data of the NeuroLife Sleeve for use in chronic stroke survivors to help guide future development efforts. The questions posed to participants here were adapted to investigate overarching themes mentioned by stroke survivors, caregivers, and HCPs for the use of an assistive technology [4]. Questions were targeted at the following categories: simple to apply, comfort for long-term use, freedom of movement during use, functionality / lightweightness and portability, potential for clinical and home use, and overall aesthetic design of the device (Supplementary Table 5). Summary data from the usability questionnaire is presented in Figure 5. For usability metrics with more than one question (e.g. simple to apply), the mean value was scored for that assessment.
In general, participants were optimistic that they could don and doff the NeuroLife Sleeve with the help of a caretaker in their home (3.60±0.28). Concerns were generally centered around the pre-application of the conductive spray and relative positioning of the system, which we are actively addressing in our next design iteration. During sessions, participants had the sleeve donned for >1.5 hours, and all participants reported general satisfaction with the overall comfort of the device (4.57±0.20). ATs should not hinder movement so that the user can successfully perform rehabilitation exercises or complete normal daily activities while using the device. The sleeve was designed with a lightweight stretchable fabric, and participants were generally satisfied with the ability to move their arm while the sleeve was donned (4.07±0.32). Participants were highly confident (4.07±0.22) that they could wear the sleeve doing functional light activities around their home, suggesting that the sleeve is non-restrictive, lightweight, portable, and promising for home use. A commonly overlooked barrier to widespread adoption of assistive technologies is user acceptance of the overall look and feel of the device [4]. Overall, all participants were extremely satisfied with the overall design of the sleeve (4.36±0.24). In general, they were all very excited for the opportunity to use the sleeve with the general favorability metric receiving the highest score of 4.79±0.15. In summary, the usability results from the current study provide promising early data that the NeuroLife Sleeve meets end user needs with directions on where to improve for future iterations.
Discussion
In this study, we demonstrate decoding of motor intention using the NeuroLife EMG System in people with upper-limb hemiparesis due to chronic stroke. Based on high-density surface EMG data collected from the forearm,12 functional hand, wrist and forearm movements were classified with high accuracy. Overall decoding accuracy was associated with the participant’s ability to perform the movement (quantified here as observed movement score), with greater functional movement corresponding with higher decoding accuracy. Even in movements with little to no movement capacity (movement score <= 1), the system was able to accurately differentiate movement intent, albeit with some decrease in performance. This demonstrates the NeuroLife EMG System’s ability to infer movement intention in stroke survivors with severe motor impairments.
Usability is an important factor for clinical technologies to assist with stroke rehabilitation by supporting motivation for consistent and active training. While existing AT solutions show promising results, these systems tend to focus on the technology and often fall short in the user-centric designs. Most clinical ATs involve manual placement of patch electrodes and long calibration procedures which limits the amount of practice that can be achieved within a given rehabilitation session. Furthermore, many systems are bulky and lack portability, which can limit patient adoption for use outside of rehabilitation training and into the home [29]. Here, we demonstrate that the NeuroLife EMG System can address many usability concerns of current technologies while providing robust decoding of motor intention. In combination with soft exoskeletons or FES, the sleeve can drive intention-based training coupled with functional movements in a user-centric form factor.
We collected and report user feedback to quantify features of the system that end-users were satisfied with, and identify areas for further development efforts. Based on user feedback from the current study, the sleeve design meets various end user needs. The design allows for use on either arm, and the stretchable, lightweight fabric design was reported by participants to be comfortable and does not limit natural arm movements. Aesthetically, subjects were pleased with the sleeve design and advocated that they would use the system at home for rehabilitation and activities of daily living given the opportunity. Participants mostly agreed that the sleeve was straightforward to don and doff during the study with the help of the researchers, and believed that they could apply the sleeve with the help of a caretaker. However, participants identified the simplicity to apply the sleeve as an area that is currently lacking, and participants were not confident in being able to apply the sleeve independently without assistance. This is an identified area for future development and will be the focus of next design iterations to enable at-home use. Despite this current usability limitation, participants indicated that not only would they feel comfortable performing rehabilitation therapy at home, but are excited for the possibility of using the sleeve as a therapy tool indicated by the highest score for general favorability.
Despite high decoding accuracy for most stroke participants, there were two participants (61204 and 87134) that had relatively low accuracy (<65%). In general, with increased movement ability and higher clinical scores, the decoding performance improved. With limited subjects, however, we cannot confirm this relationship, but we expect that decoding performance is dependent on the level of movement ability. To address this concern for an at-home therapeutic tool for severe stroke participants, we would likely need to decrease the number of functional movements. Focusing on a few key movements (Hand Extension/Flexion and Wrist Pronation/Supination) would be beneficial for severe stroke survivors lacking those abilities. A robust decoding paradigm with limited functional movements is likely to be more favorable for end-users. With continued rehabilitation training and data collection, additional functional movements can be added as patients functionally progress and decoders improve.
While the reported results indicate that the Neurolife EMG System can be used to decode motor intention in a package that meets end user needs, future work will focus on refining decoding algorithms, the sleeve design and related hardware, and eventual applications. Future work refining decoding algorithms will focus on overall improvements to decoding performance by leveraging many of the advancements made in recent years in the field of machine learning. We will investigate the use of more complex neural network models, including long short-term memory layers (LSTM) and convolutional neural networks (CNNs), which could improve overall decoding accuracy and specifically for participants with limited movement capability. Additionally, we will apply various machine learning techniques including transfer learning to expedite setup and calibration times for new users to address this important aspect of usability. Improvements to data quality itself can be accomplished with visual reinforcement to subjects. An online decoding system that displays the decoded intention may be more beneficial to participant engagement over the image cues used in the current study. This visual feedback, potentially incorporated into VR, may enhance rehabilitation efficacy by directing intention-based movements through an immersive experience. Lastly, our group is working to integrate FES functionality within the same EMG recording electrodes to eliminate the need for additional hardware such as exoskeleton or additional patch electrodes. With a technology that incorporates EMG and FES into a single consolidated sleeve, the system has the potential to help support motor recovery and assist in ADLs.
Conclusion
The focus of this study was to validate the NeuroLife EMG System by decoding hand, wrist, and forearm movements and collect usability data from participants with stroke. We demonstrate accurate EMG decoding of 12 different movement classes with a neural network in both able-bodied and stroke participants. Decoding accuracy was associated with the movement ability of the participant. The decoding results are consistent with similar myoelectric intention-based studies. Finally, we present data on the common usability factors of assistive devices including the simplicity, comfortability, portability, and weight of the sleeve. Overall, all participants reported good to outstanding ratings for each of the usability categories, indicating that the NeuroLife EMG System can provide accurate decoding of upper extremity motor intention while meeting the usability needs of end users.
Data Availability
Data will be made available upon request.
Supplementary Material
Supplementary Methods
Inclusion and Exclusion Criteria
For physically impaired individuals, inclusion criteria address the minimum length of time since the stroke that led to the impairment. Inclusion criteria may also pertain to meeting dimensional requirements related to interacting with the system hardware (e.g., subject’s arm dimensions must be such that they can appropriately don an existing electrode sleeve design).
For populations with potential for cognitive impairment (e.g., stroke survivors), inclusion criteria indicating ability to follow 3-step commands and communicate verbally (e.g., at least able to provide yes/no responses with accuracy) apply.
Specific Inclusion Criteria include:
Males and females ≥ 18 years old
Chronic stroke survivors who are at least 180 days post-stroke
Ability to provide appropriate consent to partake in the study
Ability to follow 3-step commands and deemed by an occupational therapist to have the capacity to complete required upper extremity movements
Ability to secure transportation to attend scheduled study sessions
Stroke-related hand impairment that interferes with ability to complete activities of daily living and is classified as Stage 1-6 on the hand subscale of the Chedoke McMaster Stroke Assessment
Persons with life-supporting or sustaining equipment or critical non-removeable implanted electronic devices are excluded for safety reasons since it is not known if the experimental systems would interfere with this equipment.
Specific exclusion criteria include:
Presence of any other clinically significant medical comorbidity for which, in the judgment of the Investigator, participation in the study would pose a safety risk to the subject
Currently participating in physical rehabilitation (e.g., occupational or physical therapy) for stroke-related upper limb impairment
Co-occurring neurological condition (e.g., Parkinson’s disease, Multiple Sclerosis) or other neuromuscular disorder (e.g., Carpal Tunnel Syndrome, neuropathy) that, in the judgment of the Investigator, may influence study results
Individuals who are immunosuppressed, have conditions that typically result in becoming immunocompromised, taking chronic steroids, or currently receiving immunosuppressive therapy
Individuals having or requiring any of the following: implanted pacemaker, life supporting/sustaining equipment, or critical non-removable implantable electronic devices such as an insulin pump or neurostimulator. An implanted Medtronic LINQ monitor does not meet this criterion (i.e., patients with a LINQ monitor may participate in this study).
Persistent pain ≥ 7/10 in impaired upper extremity, as measured by Numeric Pain Rating Scale (0-10)
Individuals whose forearm is determined to be too small or too large to fit the electrode sleeve being investigated.
Individuals who are pregnant or plan to get pregnant during the course of the study (self report).
Supplementary Figures
Acknowledgements
The authors would like to thank our development and management teams at Battelle Memorial Institute including Jesse Keckler, Nick Annetta, Sam Colachis, and Josh Branch for their engineering contributions, Charli Hooper for her assistance with data collection, and Andrew Sweeney for his contribution to the manuscript graphics. Financial support for this study came from Battelle Memorial Institute.