Elsevier

Clinical Neurophysiology

Volume 123, Issue 9, September 2012, Pages 1698-1704
Clinical Neurophysiology

A checklist for assessing the methodological quality of studies using transcranial magnetic stimulation to study the motor system: An international consensus study

https://doi.org/10.1016/j.clinph.2012.05.003Get rights and content

Abstract

In the last decade transcranial magnetic stimulation (TMS) has been the subject of more than 20,000 original research articles. Despite this popularity, TMS responses are known to be highly variable and this variability can impact on interpretation of research findings. There are no guidelines regarding the factors that should be reported and/or controlled in TMS studies. This study aimed to develop a checklist to be recommended to evaluate the methodology and reporting of studies that use single or paired pulse TMS to study the motor system. A two round international web-based Delphi study was conducted. Panellists rated the importance of a number of subject, methodological and analytical factors to be reported and/or controlled in studies that use single or paired pulse TMS to study the motor system. Twenty-seven items for single pulse studies and 30 items for paired pulse studies were included in the final checklist. Eight items related to subjects (e.g. age, gender), 21 to methodology (e.g. coil type, stimulus intensity) and two to analysis (e.g. size of the unconditioned motor evoked potential). The checklist is recommended for inclusion when submitting manuscripts for publication to ensure transparency of reporting and could also be used to critically appraise previously published work. It is envisaged that factors could be added and deleted from the checklist on the basis of future research. Use of the TMS methodological checklist should improve the quality of data collection and reporting in TMS studies of the motor system.

Highlights

► A TMS methodological checklist was developed for the reporting and interpreting of TMS studies of the motor system. ► An international expert panel participated in a Delphi study to establish consensus on items in the TMS methodological checklist. ► Use of the TMS methodological checklist should improve the quality of data collection and reporting in TMS studies of the motor system.

Introduction

Over the last 20 years, single and paired pulse transcranial magnetic stimulation (TMS) has been used extensively to study the neurophysiological bases of excitability and plasticity in different cortical regions (Rossini and Rossi, 2007, Edwards et al., 2008, Horvath et al., 2011, Groppa et al., 2012). Most experimental work using TMS has been performed on the motor system where stimulation of the motor cortex produces a visible muscle response known as a motor evoked potential (MEP), when recorded as electromyographic activity (Rothwell, 2011).

As a non-invasive brain stimulation tool with few safety risks, applications are rapidly diversifying such that the responses to TMS are being used increasingly as an outcome measure in clinical trials (Ferreri et al., 2011, Rijntjes et al., 2011) as a predictor of recovery (Manganotti et al., 2012) and as a diagnostic tool (Chen et al., 2008, Edwards et al., 2008). However, variability in MEP responses to TMS has been noted. Wassermann (2002), in a review of a large data set, estimated that 40–50% of the variation in MEP thresholds could be the result of subject variability (between and within) and experimental differences. Such high variability is likely to impact on the reliability and interpretation of research findings.

There is evidence, albeit conflicting, that between subject factors, such as age (Pitcher et al., 2003, Oliviero et al., 2006, Smith et al., 2011), gender (Wassermann, 2002, Inghilleri et al., 2004) and genotype (Cheeran et al., 2008, Voti et al., 2011) and within subject factors, such as caffeine use (Cerqueira et al., 2006) and time of day (Sale et al., 2007) may influence the MEP response to TMS. Similarly, evidence is building for the impact of different experimental techniques on MEP responses including the use of TMS pulses of different waveforms (Sommer et al., 2006), the orientation of the current induced by TMS in the motor cortex (Hill et al., 2000), coil positioning (Conforto et al., 2004), stimulation (trial) frequency, and, in the resting state, sub-threshold activation of corticospinal outputs (Wassermann, 2002). Given the variability in TMS responses and the potential for methodological and physiological differences to influence TMS responses, there is a clear need to report and control as many of these factors as possible.

To ensure rigorous research findings and facilitate comparison of data from separate studies, factors that impact on MEP responses should be excluded or controlled as a covariate in multivariate analyses (Wunsch, 2007). Yet, studies using TMS have small sample sizes that preclude covariate analysis. A large body of work investigating a broad range of factors is necessary before all those that impact on TMS responses are identified. In the absence of large datasets, the traditional approach to ensure findings from studies are interpreted and presented correctly is by peer review and editorial decisions, yet the quality of these processes is not guaranteed (Cobo et al., 2011). To assist peer review and critical appraisal of study methodology, reporting checklists and guidelines are being used with increasing frequency. This not only improves the transparency of research reporting and the quality of data collection but also reduces design and reporting biases. These reporting checklists specify “a minimum set of items required for a clear and transparent account of what was done and what was found in a research study, reflecting in particular, issues that might introduce bias into the research” (EQUATOR Network, 2011).

Checklists are available for specific research methodologies including randomised controlled trials (Moher et al., 2010), observational studies (Vandenbroucke et al., 2009), systematic reviews and meta-analyses (Moher et al., 1999) and on-line surveys (Eysenbach, 2004). The most well known is the CONSORT statement which was developed to provide an evidence-based minimum set of recommendations to prepare reports of findings from randomised controlled trials in order to facilitate complete and transparent reporting while aiding critical appraisal and interpretation (Moher et al., 2010). Given the potential for variability in the parameters of the MEP responses elicited by TMS and the growth in research applications for TMS, guidelines on factors that should be reported and/or controlled in single or paired pulse TMS studies of the motor system are essential to ensure research findings are correctly interpreted.

Section snippets

Methods

An observational approach using an on-line Delphi technique was employed. The Delphi technique is a widely used and accepted method for achieving consensus of opinion on a topic area solicited from experts within a field (Keeney et al., 2001, Hsu and Sandford, 2007, Vernon, 2009). Delphi methodology uses a series of sequential questionnaires (commonly referred to as rounds) interspersed by controlled feedback to gain the most reliable consensus of opinion (Thompson, 2009, Vernon, 2009).

Results

From the 78 experts invited to participate in the first round, 42 (53.8%) responses were received. The characteristics of the expert panel are presented in Table 1. The expert panel had a mean (SD) of 13.3 (6.5) years experience using TMS (single or paired pulse) to study the motor system. In total, the panel had 520 years of experience. Seventy-four per cent of participants held PhDs with over 60% working in universities in various research roles. From the 42 respondents from round 1, 39

Discussion

The goal of this study was to design a checklist to facilitate evaluation of methodology and reporting of studies that use single or paired pulse TMS to evaluate the motor system. The final TMS methodological checklist was developed through an on-line survey using the Delphi technique and consists of 27 items for single pulse studies and 30 items for paired pulse studies. Eight items relate to subjects (e.g. age, gender), 21 to methodology (e.g. coil type, stimulus intensity) and two to

Acknowledgements

We are grateful to all panel members who participated in both rounds of this Delphi consensus study: Abbruzzese G., Byblow W., Cantello R., Celnik P., Chen R., Chipchase L., Classen J., Di Lazzaro V., Dimyan M., Epstein C., Floel A., Galea M., Golaszewski S., Hamada M., Khedr E., Lefaucheu J., Macefield V., Mall V., McDonnell M., Orth M., Paulus W., Perez M., Pitcher J., Quartarone A., Rossi S., Rothkegel H., Rothwell J., Sale M., Sandrini M., Schabrun S., Schambra H., Semmler J., Taylor J.,

References (43)

  • S. Keeney et al.

    A critical review of the Delphi technique as a research methodology for nursing

    Int J Nurs Stud

    (2001)
  • P. Manganotti et al.

    Effect of median-nerve electrical stimulation on BOLD activity in acute ischemic stroke patients

    Clin Neurophysiol

    (2012)
  • D. Moher et al.

    Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement

    Lancet

    (1999)
  • D. Moher et al.

    CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials

    Clin Epidemiol

    (2010)
  • A. Oliviero et al.

    Effects of aging on motor cortex excitability

    Neurosci Res

    (2006)
  • V.N. Palter et al.

    Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology

    Am J Surg

    (2011)
  • J.C. Rothwell

    Using transcranial magnetic stimulation methods to probe connectivity between motor areas of the brain

    Hum Mov Sci

    (2011)
  • M. Sommer et al.

    Half sine, monophasic and biphasic transcranial magnetic stimulation of the human motor cortex

    Clin Neurophysiol

    (2006)
  • E.M. Wassermann

    Variation in the response to transcranial magnetic brain stimulation in the general population

    Clin Neurophysiol

    (2002)
  • B. Cheeran et al.

    A common polymorphism in the brain-derived neurotrophic factor gene (BDNF) modulates human cortical plasticity and the response to rTMS

    J Physiol

    (2008)
  • E. Cobo et al.

    Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial

    BMJ

    (2011)
  • Cited by (0)

    View full text