A multi-disciplinary perspective on emergent and future innovations in peer review

Authors : Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Kat, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O’Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure?

Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research.

With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review.

We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation.

Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages.

We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system.

Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

URL : A multi-disciplinary perspective on emergent and future innovations in peer review

DOI : http://dx.doi.org/10.12688/f1000research.12037.1

 

Openness in Scholarship: A Return to Core Values?

Author : Cameron Neylon

The debate over the meaning, and value, of open movements has intensified. The fear of co-option of various efforts from Open Access to Open Data is driving a reassessment and re-definition of what is intended by “open”.

In this article I apply group level models from cultural studies and economics to argue that the tension between exclusionary group formation and identity and aspirations towards inclusion and openness are a natural part of knowledge-making.

Situating the traditional Western Scientific Knowledge System as a culture-made group, I argue that the institutional forms that support the group act as economic underwriters for the process by which groups creating exclusive knowledge invest in the process of making it more accessible, less exclusive, and more public-good-like, in exchange for receiving excludable goods that sustain the group.

A necessary consequence of this is that our institutions will be conservative in their assessment of what knowledge-goods are worth of consideration and who is allowed within those institutional systems. Nonetheless the inclusion of new perspectives and increasing diversity underpins the production of general knowledge.

I suggest that instead of positioning openness as new, and in opposition to traditional closed systems, it may be more productive to adopt a narrative in which efforts to increase inclusion are seen as a very old, core value of the academy, albeit one that is a constant work in progress.

URL : Openness in Scholarship: A Return to Core Values?

On the origin of nonequivalent states: How we can talk about preprints

Authors : Cameron Neylon, Damian Pattinson, Geoffrey Bilder, Jennifer Lin

Increasingly, preprints are at the center of conversations across the research ecosystem. But disagreements remain about the role they play. Do they “count” for research assessment?

Is it ok to post preprints in more than one place? In this paper, we argue that these discussions often conflate two separate issues, the history of the manuscript and the status granted it by different communities.

In this paper, we propose a new model that distinguishes the characteristics of the object, its “state”, from the subjective “standing” granted to it by different communities.

This provides a way to discuss the difference in practices between communities, which will deliver more productive conversations and facilitate negotiation, as well as sharpening our focus on the role of different stakeholders on how to collectively improve the process of scholarly communications not only for preprints, but other forms of scholarly contributions.

URL : On the origin of nonequivalent states: How we can talk about preprints

DOI : http://dx.doi.org/10.12688/f1000research.11408.1