Catégorie : EN

Catégories
EN

A multi-disciplinary perspective on emergent and future innovations in peer review

Authors : Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Kat, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O’Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure?

Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research.

With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review.

We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation.

Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages.

We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system.

Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

URL : A multi-disciplinary perspective on emergent and future innovations in peer review

DOI : http://dx.doi.org/10.12688/f1000research.12037.1

 

Catégories
EN

Searching Data: A Review of Observational Data Retrieval Practices

Authors : Kathleen Gregory, Paul Groth, Helena Cousijn, Andrea Scharnhorst, Sally Wyatt

A cross-disciplinary examination of the user behaviours involved in seeking and evaluating data is surprisingly absent from the research data discussion. This review explores the data retrieval literature to identify commonalities in how users search for and evaluate observational research data.

Two analytical frameworks rooted in information retrieval and science technology studies are used to identify key similarities in practices as a first step toward developing a model describing data retrieval.

URL : https://arxiv.org/abs/1707.06937

 

Catégories
EN

Peer Evaluation of Teaching in an Online Information Literacy Course

Authors : Susan A . Vega García, Kristine Stacy-Bates, Jeff Alger, Rano Marupova

This paper reports on the development and implementation of a process of peer evaluation of teaching to assess librarian instruction in a high-enrollment online information literacy course for undergraduates.

This paper also traces a shift within libraries from peer coaching to peer evaluation models. One common model for peer evaluation, using pre- and post-observation meetings between instructor and evaluator, as well as a formal summative report, has been adapted to focus attention on key aspects of online teaching.

The paper also discusses the need for evaluating librarians’ online teaching performance, as distinct from online course design.

URL : http://lib.dr.iastate.edu/refinst_pubs/66/

 

Catégories
EN

Research Output Availability on Academic Social Networks : Implications for Stakeholders in Academic Publishing

Authors : Mikael Laakso, Juho Lindman, Cenyu Shen, Linus Nyman, Bo-Christer Björk

A recent disruption in academic publishing are Academic Social Networks (ASN), i.e. web platforms such as ResearchGate and Academia.edu that have provided new ways for researchers to disseminate, search for, and retrieve research articles.

ASNs are still a grey area in terms of implications for involved stakeholders, and research on them has so far been scarce.

In an effort to map out factors related to ASN use this article provides a multi-method case study of one business school (Hanken School of Economics, Finland) that incorporates 1) a bibliometric analysis on the full-text availability of research output on ASNs for research published 2012–2014 by Hanken affiliated authors, 2) semi-structured interviews with faculty active in publishing in order to gain insight into motivations for use and use patterns, and 3) a survey distributed to all research-active faculty and doctoral students in order to gain a wider perspective on ASN use.

ASNs have for many become the primary way to provide access to one’s research output, outpacing all other types of online locations such as personal websites and repositories.

Based on the case study findings, earlier research, and recent industry developments, the article concludes with a discussion about the implications that the current trajectory of ASN use has on major stakeholders in academic publishing.

URL : http://mikaellaakso.com/Laakso_et_al_2017_ASN.pdf

Catégories
EN

DuEPublicA: Automated bibliometric reports based on the University Bibliography and external citation data

Author : Eike T. Spielberg

This paper describes a web application to generate bibliometric reports based on the University Bibliography and the Scopus citation database. Our goal is to offer an alternative to easy-to-prepare automated reports from commercial sources.

These often suffer from an incomplete coverage of publication types and a difficult attribution to people, institutes and universities. Using our University Bibliography as the source to select relevant publications solves the two problems.

As it is a local system, maintained and set up by the library, we can include every publication type we want. As the University Bibliography is linked to the identity management system of the university, it enables an easy selection of publications for people, institutes and the whole university.

The program is designed as a web application, which collects publications from the University Bibliography, enriches them with citation data from Scopus and performs three kinds of analyses:
1. A general analysis (number and type of publications, publications per year etc.),
2. A citation analysis (average citations per publication, h-index, uncitedness), and
3. An affiliation analysis (home and partner institutions)

We tried to keep the code highly generic, so that the inclusion of other databases (Web of Science, IEEE) or other bibliographies is easily feasible. The application is written in Java and XML and uses XSL transformations and LaTeX to generate bibliometric reports as HTML pages and in pdf format.

Warnings and alerts are automatically included if the citation analysis covers only a small fraction of the publications from the University Bibliography. In addition, we describe a small tool that helps to collect author details for an analysis.

URL : http://journal.code4lib.org/articles/12549

Catégories
EN

Journal Data Sharing Policies and Statistical Reporting Inconsistencies in Psychology

Authors : Michele Nuijten, Jeroen Borghuis, Coosje Veldkamp, Linda Alvarez, Marcel van Assen, Jelte Wicherts

In this paper, we present three studies that investigate the relation between data sharing and statistical reporting inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance (Wicherts, Bakker, & Molenaar, 2011).

We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies.

In Study 2, we compared reporting inconsistencies in articles published in PLOS (with a data sharing policy) and Frontiers in Psychology (without a data sharing policy). In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors.

Overall, we found no relationship between data sharing and reporting inconsistencies. We did find that journal policies on data sharing are extremely effective in promoting data sharing.

We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature.

DOI : https://dx.doi.org/10.17605/OSF.IO/SGBTA

Catégories
EN

A Proposed Currency System for Academic Peer Review Payments Using the BlockChain Technology

Author : Michael Spearpoint

Peer review of scholarly papers is seen to be a critical step in the publication of high quality outputs in reputable journals.

However, it appears that there are few incentives for researchers to agree to conduct suitable reviews in a timely fashion and in some cases unscrupulous practices are occurring as part of the production of academic research output. Innovations in internet-based technologies mean that there are ways in which some of the challenges can be addressed.

In particular, this paper proposes a new currency system using the BlockChain as its basis that provides a number of solutions. Potential benefits and problems of using the technology are discussed in the paper and these will need further investigation should the idea develop further.

Ultimately, the currency could be used as an alternative publication metric for authors, institutions and journals.

URL : A Proposed Currency System for Academic Peer Review Payments Using the BlockChain Technology

Alternative location : http://www.mdpi.com/2304-6775/5/3/19