A checklist for assessing the methodological quality of studies using transcranial magnetic stimulation to study the motor system: An international consensus study
Highlights
► A TMS methodological checklist was developed for the reporting and interpreting of TMS studies of the motor system. ► An international expert panel participated in a Delphi study to establish consensus on items in the TMS methodological checklist. ► Use of the TMS methodological checklist should improve the quality of data collection and reporting in TMS studies of the motor system.
Introduction
Over the last 20 years, single and paired pulse transcranial magnetic stimulation (TMS) has been used extensively to study the neurophysiological bases of excitability and plasticity in different cortical regions (Rossini and Rossi, 2007, Edwards et al., 2008, Horvath et al., 2011, Groppa et al., 2012). Most experimental work using TMS has been performed on the motor system where stimulation of the motor cortex produces a visible muscle response known as a motor evoked potential (MEP), when recorded as electromyographic activity (Rothwell, 2011).
As a non-invasive brain stimulation tool with few safety risks, applications are rapidly diversifying such that the responses to TMS are being used increasingly as an outcome measure in clinical trials (Ferreri et al., 2011, Rijntjes et al., 2011) as a predictor of recovery (Manganotti et al., 2012) and as a diagnostic tool (Chen et al., 2008, Edwards et al., 2008). However, variability in MEP responses to TMS has been noted. Wassermann (2002), in a review of a large data set, estimated that 40–50% of the variation in MEP thresholds could be the result of subject variability (between and within) and experimental differences. Such high variability is likely to impact on the reliability and interpretation of research findings.
There is evidence, albeit conflicting, that between subject factors, such as age (Pitcher et al., 2003, Oliviero et al., 2006, Smith et al., 2011), gender (Wassermann, 2002, Inghilleri et al., 2004) and genotype (Cheeran et al., 2008, Voti et al., 2011) and within subject factors, such as caffeine use (Cerqueira et al., 2006) and time of day (Sale et al., 2007) may influence the MEP response to TMS. Similarly, evidence is building for the impact of different experimental techniques on MEP responses including the use of TMS pulses of different waveforms (Sommer et al., 2006), the orientation of the current induced by TMS in the motor cortex (Hill et al., 2000), coil positioning (Conforto et al., 2004), stimulation (trial) frequency, and, in the resting state, sub-threshold activation of corticospinal outputs (Wassermann, 2002). Given the variability in TMS responses and the potential for methodological and physiological differences to influence TMS responses, there is a clear need to report and control as many of these factors as possible.
To ensure rigorous research findings and facilitate comparison of data from separate studies, factors that impact on MEP responses should be excluded or controlled as a covariate in multivariate analyses (Wunsch, 2007). Yet, studies using TMS have small sample sizes that preclude covariate analysis. A large body of work investigating a broad range of factors is necessary before all those that impact on TMS responses are identified. In the absence of large datasets, the traditional approach to ensure findings from studies are interpreted and presented correctly is by peer review and editorial decisions, yet the quality of these processes is not guaranteed (Cobo et al., 2011). To assist peer review and critical appraisal of study methodology, reporting checklists and guidelines are being used with increasing frequency. This not only improves the transparency of research reporting and the quality of data collection but also reduces design and reporting biases. These reporting checklists specify “a minimum set of items required for a clear and transparent account of what was done and what was found in a research study, reflecting in particular, issues that might introduce bias into the research” (EQUATOR Network, 2011).
Checklists are available for specific research methodologies including randomised controlled trials (Moher et al., 2010), observational studies (Vandenbroucke et al., 2009), systematic reviews and meta-analyses (Moher et al., 1999) and on-line surveys (Eysenbach, 2004). The most well known is the CONSORT statement which was developed to provide an evidence-based minimum set of recommendations to prepare reports of findings from randomised controlled trials in order to facilitate complete and transparent reporting while aiding critical appraisal and interpretation (Moher et al., 2010). Given the potential for variability in the parameters of the MEP responses elicited by TMS and the growth in research applications for TMS, guidelines on factors that should be reported and/or controlled in single or paired pulse TMS studies of the motor system are essential to ensure research findings are correctly interpreted.
Section snippets
Methods
An observational approach using an on-line Delphi technique was employed. The Delphi technique is a widely used and accepted method for achieving consensus of opinion on a topic area solicited from experts within a field (Keeney et al., 2001, Hsu and Sandford, 2007, Vernon, 2009). Delphi methodology uses a series of sequential questionnaires (commonly referred to as rounds) interspersed by controlled feedback to gain the most reliable consensus of opinion (Thompson, 2009, Vernon, 2009).
Results
From the 78 experts invited to participate in the first round, 42 (53.8%) responses were received. The characteristics of the expert panel are presented in Table 1. The expert panel had a mean (SD) of 13.3 (6.5) years experience using TMS (single or paired pulse) to study the motor system. In total, the panel had 520 years of experience. Seventy-four per cent of participants held PhDs with over 60% working in universities in various research roles. From the 42 respondents from round 1, 39
Discussion
The goal of this study was to design a checklist to facilitate evaluation of methodology and reporting of studies that use single or paired pulse TMS to evaluate the motor system. The final TMS methodological checklist was developed through an on-line survey using the Delphi technique and consists of 27 items for single pulse studies and 30 items for paired pulse studies. Eight items relate to subjects (e.g. age, gender), 21 to methodology (e.g. coil type, stimulus intensity) and two to
Acknowledgements
We are grateful to all panel members who participated in both rounds of this Delphi consensus study: Abbruzzese G., Byblow W., Cantello R., Celnik P., Chen R., Chipchase L., Classen J., Di Lazzaro V., Dimyan M., Epstein C., Floel A., Galea M., Golaszewski S., Hamada M., Khedr E., Lefaucheu J., Macefield V., Mall V., McDonnell M., Orth M., Paulus W., Perez M., Pitcher J., Quartarone A., Rossi S., Rothkegel H., Rothwell J., Sale M., Sandrini M., Schabrun S., Schambra H., Semmler J., Taylor J.,
References (43)
- et al.
Comparison of “standard” and “navigated” procedures of TMS coil positioning over motor, premotor and prefrontal targets in patients with chronic pain and depression
Clin Neurophysiol
(2010) - et al.
Does caffeine modify corticomotor excitability?
Clin Neurophysiol
(2006) - et al.
The clinical diagnostic utility of transcranial magnetic stimulation: report of an IFCN committee
Clin Neurophysiol
(2008) - et al.
Impact of coil position and electrophysiological monitoring on determination of motor thresholds to transcranial magnetic stimulation
Clin Neurophysiol
(2004) - et al.
Development of a quality checklist using Delphi methods for prescriptive clinical prediction rules: the QUADCPR
J Manipulative Physiol Ther
(2010) - et al.
Clinical applications of transcranial magnetic stimulation in patients with movement disorders
Lancet Neurol
(2008) - et al.
Motor cortex excitability in Alzheimer’s disease: a transcranial magnetic stimulation follow-up study
Neurosci Lett
(2011) - et al.
Delphi as a method to establish consensus for diagnostic criteria
J Clin Epidemiol
(2003) - et al.
A practical guide to diagnostic transcranial magnetic stimulation: report of an IFCN committee
Clin Neurophysiol
(2012) - et al.
Ovarian hormones and cortical excitability. An rTMS study in humans
Clin Neurophysiol
(2004)