ABSTRACT
Objective To assess the effectiveness of the Peerspectives course on editor-judged quality of peer review
Design Single arm, pre-post interventional study
Setting Peerspectives is a peer review training course developed for doctoral students at the Charité – Universitätsmedizin Berlin, Germany
Participants Doctoral students in health research domains who enrolled in the Peerspectives course between October 2020 and August 2022 and consented to participate (N = 80). Intervention: A semester-long course (approx. 18 weeks) providing training about the structure, purpose, and conduct of peer review and editing processes in biomedical journals. The course consisted of 12 hours of interactive lectures, weekly homework assignments, and 12 hours of hands-on, small-group workshops under guidance of mentors with reviewing and editing experience. Students peer-reviewed real scientific manuscripts under review at a partnering journal, The BMJ.
Main outcome measures The overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, participants’ self-assessed knowledge and skills related to scholarly peer review (1-5 Likert scale).
Results After course completion, participants’ RQI scores were higher than before the course (median increase of 0.5 points, p<0.001; mean increase of 0.36 points, p<0.001). The RQI scores of participants’ post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Largest improvements were seen in understanding what is expected from a reviewer (increase in means from 2.9 to 4.5), confidence in own ability to review (2.5 to 3.9), and knowing what to look for in a manuscript while reviewing (2.8 to 4.2).
Conclusions Providing doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback.
Study pre-registration: https://osf.io/vndcx
Competing Interest Statement
TG and JLR received funding from the Volkswagen Foundation to further develop the Peerspectives course and create open-source train-the-trainer materials. This funding was received after data used in this evaluation study were collected and analyzed, but supports the ongoing development and organization of the course and openly available course materials. TK has received research grants from the German Federal Joint Committee (G-BA) and personal compensation from Eli Lilly & Company, Novartis, the BMJ, and Frontiers. He is also a consultant methods editor at The BMJ and BMJ Open. SS is a full time employee at BMJ. NW, KR, HG, IA, MLS, and MP report no disclosures.
Clinical Protocols
Funding Statement
Partial funding for this study was obtained by TG and JLR via the SPOKES Wellcome Trust Funded Translational Partnership Fellowship program (granted to the QUEST Center at the Berlin Institute of Health; Nr. 218358). A grant from the Volkswagen Foundation (Pioneer Projects - Impetus for the German Research System; Nr. 9C872) has supported the continuation of the course beyond the study period, including part of the position of JLR, and for open access fees. These funders had no role in study design, data collection, analyses, interpretation of data, writing of the report, or decision to submit the article for publication.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This study was approved by the Charite - Universitatsmedizin Berlin ethics committee (EA4/190/20). All participants provided written informed consent.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
The corresponding pseudonymised datasets used to generate our results and corresponding analytical code will be made accessible via our OSF project page upon publication.
