Étiquette : Sara Schroter

Catégories
EN

Effectiveness of the researcher-led “Peerspectives” peer review training course on review quality, knowledge, and skills among doctoral students in the biomedical sciences: a pre-post study

Authors : Jessica L. Rohmann, Nadja Wülk, Kerstin Rubarth, Hannah Grillmaier, Iman Abdikarim, Mariana Lopes Simões, Sara Schroter, Marco Piccininni, Tobias Kurth, Toivo Glatz

Background

Peer review remains a cornerstone of scientific knowledge dissemination, yet comprehensive, practically relevant training is limited. This inspired us to develop Peerspectives, a peer review training course for doctoral students in the biomedical sciences in Berlin, Germany. We aimed to assess the effectiveness of the Peerspectives course on editor-judged quality of peer review reports.

Methods

Doctoral students in health research fields who enrolled in the Peerspectives course between October 2020 and August 2022 were invited to participate in the study, and 80 consented. The ~18 week-long course provided training on the structure, purpose, and conduct of peer review and editorial processes in biomedical journals. It included 12 h of lectures, homework assignments, and 12 h of hands-on, small-group workshops, during which students reviewed original research manuscripts currently under consideration at The BMJ under the guidance of experienced mentors.

The primary outcome was the overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, we compared participants’ post-course scores with those of actual BMJ reviewers. We also compared participants’ self-assessed knowledge and skills related to scholarly peer review (1–5 Likert scale) before and after the course.

Results

After course completion, the editor-assessed overall quality of the participants’ peer review reports was higher than before the course (median increase of 0.5 points, p < 0.001; mean increase of 0.36 points, p < 0.001). The RQI scores of participants’ post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Greatest improvements were seen in understanding reviewer expectations (increase in means from 2.9 to 4.5), confidence in reviewing (2.5 to 3.9), and knowing what to look for while reviewing (2.8 to 4.2).

Conclusions

Providing doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback.

URL : Effectiveness of the researcher-led “Peerspectives” peer review training course on review quality, knowledge, and skills among doctoral students in the biomedical sciences: a pre-post study

DOI : https://doi.org/10.1186/s41073-026-00220-3

Catégories
EN

Improving peer review of systematic reviews and related review types by involving librarians and information specialists as methodological peer reviewers: a randomised controlled trial

Authors : Melissa L Rethlefsen, Sara Schroter, Lex M Bouter, Jamie J Kirkham,  David Moher, Ana Patricia Ayala, David Blanco, Tara J Brigham, Holly K Grossetta Nardini,  Shona Kirtley, Kate Nyhan, Whitney Townsend, Maurice Zeegers

Objective

To evaluate the impact of adding librarians and information specialists (LIS) as methodological peer reviewers to the formal journal peer review process on the quality of search reporting and risk of bias in systematic review searches in the medical literature.

Design

Pragmatic two-group parallel randomised controlled trial.

Setting

Three biomedical journals.

Participants

Systematic reviews and related evidence synthesis manuscripts submitted to The BMJ, BMJ Open and BMJ Medicine and sent out for peer review from 3 January 2023 to 1 September 2023. Randomisation (allocation ratio, 1:1) was stratified by journal and used permuted blocks (block size=4). Of 2670 manuscripts sent to peer review during study enrollment, 400 met inclusion criteria and were randomised (62 The BMJ, 334 BMJ Open, 4 BMJ Medicine). 76 manuscripts were revised and resubmitted in the intervention group and 90 in the control group by 2 January 2024.

Interventions

All manuscripts followed usual journal practice for peer review, but those in the intervention group had an additional (LIS) peer reviewer invited.

Main outcome measures

The primary outcomes are the differences in first revision manuscripts between intervention and control groups in the quality of reporting and risk of bias. Quality of reporting was measured using four prespecified PRISMA-S items. Risk of bias was measured using ROBIS Domain 2. Assessments were done in duplicate and assessors were blinded to group allocation. Secondary outcomes included differences between groups for each individual PRISMA-S and ROBIS Domain 2 item. The difference in the proportion of manuscripts rejected as the first decision post-peer review between the intervention and control groups was an additional outcome.

Results

Differences in the proportion of adequately reported searches (4.4% difference, 95% CI: −2.0% to 10.7%) and risk of bias in searches (0.5% difference, 95% CI: −13.7% to 14.6%) showed no statistically significant differences between groups. By 4 months post-study, 98 intervention and 70 control group manuscripts had been rejected after peer review (13.8% difference, 95% CI: 3.9% to 23.8%).

Conclusions

Inviting LIS peer reviewers did not impact adequate reporting or risk of bias of searches in first revision manuscripts of biomedical systematic reviews and related review types, though LIS peer reviewers may have contributed to a higher rate of rejection after peer review.

URL : Improving peer review of systematic reviews and related review types by involving librarians and information specialists as methodological peer reviewers: a randomised controlled trial

DOI : https://doi.org/10.1136/bmjebm-2024-113527

Catégories
EN

Biomedical authors’ awareness of publication ethics: an international survey

Authors : Sara Schroter, Jason Roberts, Elizabeth Loder, Donald B Penzien, Sarah Mahadeo, Timothy T Houle

Objective

The extent to which biomedical authors have received training in publication ethics, and their attitudes and opinions about the ethical aspects of specific behaviours, have been understudied. We sought to characterise the knowledge and attitudes of biomedical authors about common issues in publication ethics.

Design

Cross-sectional online survey.

Setting and participants

Corresponding authors of research submissions to 20 journals.

Main outcome measure(s)

Perceived level of unethical behaviour (rated 0 to 10) presented in five vignettes containing key variables that were experimentally manipulated on entry to the survey and perceived level of knowledge of seven ethical topics related to publishing (prior publication, author omission, self-plagiarism, honorary authorship, conflicts of interest, image manipulation and plagiarism).

Results

4043/10 582 (38%) researchers responded. Respondents worked in 100 countries and reported varying levels of publishing experience. 67% (n=2700) had received some publication ethics training from a mentor, 41% (n=1677) a partial course, 28% (n=1130) a full course and 55% (n=2206) an online course; only a small proportion rated training received as excellent.

There was a full range (0 to 10 points) in ratings of the extent of unethical behaviour within each vignette, illustrating a broad range of opinion about the ethical acceptability of the behaviours evaluated, but these opinions were little altered by the context in which it occurred.

Participants reported substantial variability in their perceived knowledge of seven publication ethics topics; one-third perceived their knowledge to be less than ‘some knowledge’ for the sum of the seven ethical topics and only 9% perceived ‘substantial knowledge’ of all topics.

Conclusions

We found a large degree of variability in espoused training and perceived knowledge, and variability in views about how ethical or unethical scenarios were. Ethical standards need to be better articulated and taught to improve consistency of training across institutions and countries.

URL : Biomedical authors’ awareness of publication ethics: an international survey

DOI : http://dx.doi.org/10.1136/bmjopen-2017-021282