Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

Authors : Clarissa F. D. Carneiro, Victor G. S. Queiroz, Thiago C. Moulin, Carlos A. M. Carvalho, Clarissa B. Haas, Danielle Rayêe, David E. Henshall, Evandro A. De-Souza, Felippe Espinelli, Flávia Z. Boos, Gerson D. Guercio, Igor R. Costa, Karina L. Hajdu, Martin Modrák, Pedro B. Tan, Steven J. Burgess, Sylvia F. S. Guerra, Vanessa T. Bortoluzzi, Olavo B. Amara

Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings.

In this observational study, we compared random samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. We found that peer-reviewed articles had, on average, higher quality of reporting than preprints, although this difference was small.

We found larger differences favoring PubMed in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information.

Interestingly, an exploratory analysis showed that preprints with figures and legends embedded within text had reporting scores similar to PubMed articles.

These differences cannot be directly attributed to peer review or editorial processes, as manuscripts might already differ before submission due to greater uptake of preprints by particular research communities.

Nevertheless, our results show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

An ongoing second phase of the project is comparing preprints to their own published versions in order to more directly assess the effects of peer review.

URL : Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature


arXiv and the Symbiosis of Physics Preprints and Journal Review Articles: A Model

Author : Brian Simboli

This paper recommends a publishing model that can help achieve the goal of reforming physics publishing. It distinguishes two complementary needs in scholarly communication.

Preprints, increasingly important in science, are properly the vehicle for claiming priority of discovery and for eliciting feedback that will help with versioning.

Traditional journal publishing, however, should focus on providing synthesis in the form of overlay journals that play the same role as review articles.


Preprints in Scholarly Communication: Re-Imagining Metrics and Infrastructures

Authors : B. Preedip Balaji, M. Dhanamjaya

Digital scholarship and electronic publishing among the scholarly communities are changing when metrics and open infrastructures take centre stage for measuring research impact. In scholarly communication, the growth of preprint repositories over the last three decades as a new model of scholarly publishing has emerged as one of the major developments.

As it unfolds, the landscape of scholarly communication is transitioning, as much is being privatized as it is being made open and towards alternative metrics, such as social media attention, author-level, and article-level metrics. Moreover, the granularity of evaluating research impact through new metrics and social media change the objective standards of evaluating research performance.

Using preprint repositories as a case study, this article situates them in a scholarly web, examining their salient features, benefits, and futures. Towards scholarly web development and publishing on semantic and social web with open infrastructures, citations, and alternative metrics—how preprints advance building web as data is discussed.

We examine that this will viably demonstrate new metrics and in enhancing research publishing tools in scholarly commons facilitating various communities of practice.

However, for the preprint repositories to sustain, scholarly communities and funding agencies should support continued investment in open knowledge, alternative metrics development, and open infrastructures in scholarly publishing.

URL : Preprints in Scholarly Communication: Re-Imagining Metrics and Infrastructures


On the value of preprints: an early career researcher perspective

Authors : Sarvenaz Sarabipour​, Humberto J Debat, Edward Emmott, Steven Burgess, Benjamin Schwessinger, Zach Hensel

Peer-reviewed journal publication is the main means for academic researchers in the life sciences to create a permanent, public record of their work. These publications are also the de facto currency for career progress, with a strong link between journal brand recognition and perceived value.

The current peer-review process can lead to long delays between submission and publication, with cycles of rejection, revision and resubmission causing redundant peer review.

This situation creates unique challenges for early career researchers (ECRs), who rely heavily on timely publication of their work to gain recognition for their efforts. ECRs face changes in the academic landscape including the increased interdisciplinarity of life sciences research, expansion of the researcher population and consequent shifts in employer and funding demands.

The publication of preprints, publicly available scientific manuscripts posted on dedicated preprint servers prior to journal managed peer-review, can play a key role in addressing these ECR challenges.

Preprinting benefits include rapid dissemination of academic work, open access, establishing priority or concurrence, receiving feedback and facilitating collaborations. While there is a growing appreciation for and adoption of preprints, a minority of all articles in life sciences and medicine are preprinted.

The current low rate of preprint submissions in life sciences and ECR concerns regarding preprinting needs to be addressed.

We provide a perspective from an interdisciplinary group of early career researchers on the value of preprints and advocate the wide adoption of preprints to advance knowledge and facilitate career development.

URL : On the value of preprints: an early career researcher perspective


Citation Count Analysis for Papers with Preprints

Authors : Sergey Feldman, Kyle Lo, Waleed Ammar

We explore the degree to which papers prepublished on arXiv garner more citations, in an attempt to paint a sharper picture of fairness issues related to prepublishing. A paper’s citation count is estimated using a negative-binomial generalized linear model (GLM) while observing a binary variable which indicates whether the paper has been prepublished.

We control for author influence (via the authors’ h-index at the time of paper writing), publication venue, and overall time that paper has been available on arXiv. Our analysis only includes papers that were eventually accepted for publication at top-tier CS conferences, and were posted on arXiv either before or after the acceptance notification.

We observe that papers submitted to arXiv before acceptance have, on average, 65\% more citations in the following year compared to papers submitted after. We note that this finding is not causal, and discuss possible next steps.


The evolving preprint landscape: Introductory report for the Knowledge Exchange working group on preprints

Authors : Jonathan Tennant, Serge Bauin, Sarah James, Juliane Kant

In 1961, the USA National Institutes of Health (NIH) launched a program called Information Exchange Groups, designed for the circulation of biological preprints, but this shut down in 1967 (Confrey, 1996; Cobb, 2017).

In 1991, the arXiv repository was launched for physics, computer science, and mathematics, which is when preprints (or ‘e-prints’) began to increase in popularity and attention (Wikipedia ArXiv#History; Jackson, 2002). The Social Sciences Research Network (SSRN) was launched in 1994, and in 1997 Research Papers in Economics (Wikipedia RePEc) was launched.

In 2008, the research network platforms and ResearchGate were both launched and allowed sharing of research papers at any stage. In 2013, two new biological preprint servers were launched, bioRxiv (by Cold Spring Harbor Laboratory) and PeerJ Preprints (by PeerJ) (Wikipedia BioRxiv; Wikipedia PeerJ).

Between these major ongoing initiatives were various, somewhat less-successful attempts to launch preprint servers, including Nature Precedings (folded in April 2012) and Netprints from the British Medical Journal (Wikipedia Nature Precedings; BMJ, 1999).

Now, a range of innovative services, organisations, and platforms are rapidly developing around preprints, prompting this overview of the present ecosystem on behalf of Knowledge Exchange.

URL : The evolving preprint landscape: Introductory report for the Knowledge Exchange working group on preprints


Prepublication disclosure of scientific results: Norms, competition, and commercial orientation

Authors : Jerry G. Thursby, Carolin Haeussler, Marie C. Thursby, Lin Jiang

On the basis of a survey of 7103 active faculty researchers in nine fields, we examine the extent to which scientists disclose prepublication results, and when they do, why? Except in two fields, more scientists disclose results before publication than not, but there is significant variation in their reasons to disclose, in the frequency of such disclosure, and in withholding crucial results when making public presentations.

They disclose results for feedback and credit and to attract collaborators. Particularly in formulaic fields, scientists disclose to attract new researchers to the field independent of collaboration and to deter others from working on their exact problem.

A probability model shows that 70% of field variation in disclosure is related to differences in respondent beliefs about norms, competition, and commercialization. Our results suggest new research directions—for example, do the problems addressed or the methods of scientific production themselves shape norms and competition?

Are the levels we observe optimal or simply path-dependent? What is the interplay of norms, competition, and commercialization in disclosure and the progress of science?

URL : Prepublication disclosure of scientific results: Norms, competition, and commercial orientation