Authors : Jessica L. Rohmann, Nadja Wülk, Kerstin Rubarth, Hannah Grillmaier, Iman Abdikarim, Mariana Lopes Simões, Sara Schroter, Marco Piccininni, Tobias Kurth, Toivo Glatz
Background
Peer review remains a cornerstone of scientific knowledge dissemination, yet comprehensive, practically relevant training is limited. This inspired us to develop Peerspectives, a peer review training course for doctoral students in the biomedical sciences in Berlin, Germany. We aimed to assess the effectiveness of the Peerspectives course on editor-judged quality of peer review reports.
Methods
Doctoral students in health research fields who enrolled in the Peerspectives course between October 2020 and August 2022 were invited to participate in the study, and 80 consented. The ~18 week-long course provided training on the structure, purpose, and conduct of peer review and editorial processes in biomedical journals. It included 12 h of lectures, homework assignments, and 12 h of hands-on, small-group workshops, during which students reviewed original research manuscripts currently under consideration at The BMJ under the guidance of experienced mentors.
The primary outcome was the overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, we compared participants’ post-course scores with those of actual BMJ reviewers. We also compared participants’ self-assessed knowledge and skills related to scholarly peer review (1–5 Likert scale) before and after the course.
Results
After course completion, the editor-assessed overall quality of the participants’ peer review reports was higher than before the course (median increase of 0.5 points, p < 0.001; mean increase of 0.36 points, p < 0.001). The RQI scores of participants’ post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Greatest improvements were seen in understanding reviewer expectations (increase in means from 2.9 to 4.5), confidence in reviewing (2.5 to 3.9), and knowing what to look for while reviewing (2.8 to 4.2).
Conclusions
Providing doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback.