The Social Structure of Consensus in Scientific Review

Posté le par

Authors : Misha Teplitskiy, Daniel Acuna, Aida Elamrani-Raoult, Konrad Kording, James Evans

Personal connections between creators and evaluators of scientific works are ubiquitous, and the possibility of bias ever-present. Although connections have been shown to bias prospective judgments of (uncertain) future performance, it is unknown whether such biases occur in the much more concrete task of assessing the scientific validity of already completed work, and if so, why.

This study presents evidence that personal connections between authors and reviewers of neuroscience manuscripts are associated with biased judgments and explores the mechanisms driving the effect.

Using reviews from 7,981 neuroscience manuscripts submitted to the journal PLOS ONE, which instructs reviewers to evaluate manuscripts only on scientific validity, we find that reviewers favored authors close in the co-authorship network by ~0.11 points on a 1.0 – 4.0 scale for each step of proximity.

PLOS ONE’s validity-focused review and the substantial amount of favoritism shown by distant vs. very distant reviewers, both of whom should have little to gain from nepotism, point to the central role of substantive disagreements between scientists in different “schools of thought.”

The results suggest that removing bias from peer review cannot be accomplished simply by recusing the closely-connected reviewers, and highlight the value of recruiting reviewers embedded in diverse professional networks.

URL : https://arxiv.org/abs/1802.01270

Gathering the Needles: Evaluating the Impact of Gold Open Access Content With Traditional Subscription Journals

Posté le par

Authors : Alison Boba, Jill Emery

Utilizing the Project COUNTER Release 4 JR1-GOA report, two librarians explore these data in comparison to journal package subscriptions represented via the JR1 reports. This paper outlines the methodology and study undertaken at the Portland State University Library and the University of Nebraska Medical Center Library using these reports for the first time.

The initial outcomes of the study are provided in various Tables for 2014 and 2015. The intent of the study was to provide both institutions with a baseline from which to do further study. In addition, some ideas are given for how these reports can be used in vendor negotiations going forward.

URL : Gathering the Needles: Evaluating the Impact of Gold Open Access Content With Traditional Subscription Journals

DOI : http://dx.doi.org/10.1629/uksg.291

Scaling Research Data Management Services Along the Maturity Spectrum: Three Institutional Perspectives

Posté le par

Authors : Cinthya Ippoliti, Amy Koshoffer, Renaine Julian, Micah Vandegrift, Devin Soper, Sophie Meridien

Research data services promise to advance many academic libraries’ strategic goals of becoming partners in the research process and integrating library services with modern research workflows. Academic librarians are well positioned to make an impact in this space due to their expertise in managing, curating, and preserving digital information, and a history of engaging with scholarly communications writ large.

Some academic libraries have quickly developed infrastructure and support for every activity ranging from data storage and curation to project management and collaboration, while others are just beginning to think about addressing the data needs of their researchers.

Regardless of which end of the spectrum they identify with, libraries are still seeking to understand the research landscape and define their role in the process.

This article seeks to blend both a general perspective regarding these issues with actual case studies derived from three institutions, University of Cincinnati, Oklahoma State University, and Florida State University, all of which are at different levels of implementation, maturity, and campus involvement.

URL : Scaling Research Data Management Services Along the Maturity Spectrum: Three Institutional Perspectives

DOI : https://dx.doi.org/10.17605/OSF.IO/WZ8FN

 

Harvesting the Academic Landscape: Streamlining the Ingestion of Professional Scholarship Metadata into the Institutional Repository

Posté le par

Authors : Jonathan Bull, Teresa Auch Schultz

INTRODUCTION

Although librarians initially hoped institutional repositories (IRs) would grow through researcher self-archiving, practice shows that growth is much more likely through library-directed deposit. Libraries must then find efficient ways to ingest material into their IR to ensure growth and relevance.

DESCRIPTION OF PROGRAM

Valparaiso University developed and implemented a workflow that was semiautomated to help cut down on the time needed to ingest articles into its IR, ValpoScholar. The workflow, which continues to be refined, makes use of practices and ideas used by other repositories to more efficiently collect metadata for items and upload them to the repository.

NEXT STEPS

The article discusses the pros and cons of this workflow and areas of ingesting that still need to be addressed, including adding full-text items, checking copyright policies, managing student staffing, and dealing with hurdles created by the repository’s software.

URL : Harvesting the Academic Landscape: Streamlining the Ingestion of Professional Scholarship Metadata into the Institutional Repository

DOI : http://doi.org/10.7710/2162-3309.2201

Allegation of scientific misconduct increases Twitter attention

Posté le par

Authors : Lutz Bornmann, Robin Haunschild

The web-based microblogging system Twitter is a very popular altmetrics source for measuring the broader impact of science. In this case study, we demonstrate how problematic the use of Twitter data for research evaluation can be, even though the aspiration of measurement is degraded from impact to attention measurement.

We collected the Twitter data for the paper published by Yamamizu et al. (2017). An investigative committee found that the main figures in the paper are fraudulent.

URL : https://arxiv.org/abs/1802.00606

A scoping review of comparisons between abstracts and full reports in primary biomedical research

Posté le par

Authors : Guowei Li, Luciana P. F. Abbade, Ikunna Nwosu, Yanling Jin, Alvin Leenus, Muhammad Maaz, Mei Wang, Meha Bhatt, Laura Zielinski, Nitika Sanger, Bianca Bantoto, Candice Luo, Ieta Shams, Hamnah Shahid, Yaping Chang, Guangwen Sun, Lawrence Mbuagbaw, Zainab Samaan, Mitchell A. H. Levine, Jonathan D. Adachi, Lehana Thabane

Background

Evidence shows that research abstracts are commonly inconsistent with their corresponding full reports, and may mislead readers.

In this scoping review, which is part of our series on the state of reporting of primary biomedical research, we summarized the evidence from systematic reviews and surveys, to investigate the current state of inconsistent abstract reporting, and to evaluate factors associated with improved reporting by comparing abstracts and their full reports.

Methods

We searched EMBASE, Web of Science, MEDLINE, and CINAHL from January 1st 1996 to September 30th 2016 to retrieve eligible systematic reviews and surveys. Our primary outcome was the level of inconsistency between abstracts and corresponding full reports, which was expressed as a percentage (with a lower percentage indicating better reporting) or categorized rating (such as major/minor difference, high/medium/low inconsistency), as reported by the authors.

We used medians and interquartile ranges to describe the level of inconsistency across studies. No quantitative syntheses were conducted. Data from the included systematic reviews or surveys was summarized qualitatively.

Results

Seventeen studies that addressed this topic were included. The level of inconsistency was reported to have a median of 39% (interquartile range: 14% – 54%), and to range from 4% to 78%. In some studies that separated major from minor inconsistency, the level of major inconsistency ranged from 5% to 45% (median: 19%, interquartile range: 7% – 31%), which included discrepancies in specifying the study design or sample size, designating a primary outcome measure, presenting main results, and drawing a conclusion.

A longer time interval between conference abstracts and the publication of full reports was found to be the only factor which was marginally or significantly associated with increased likelihood of reporting inconsistencies.

Conclusions

This scoping review revealed that abstracts are frequently inconsistent with full reports, and efforts are needed to improve the consistency of abstract reporting in the primary biomedical community.

URL : A scoping review of comparisons between abstracts and full reports in primary biomedical research

DOI : https://doi.org/10.1186/s12874-017-0459-5

Exploring the feasibility of applying data mining for library reference service improvement : a case study of Turku Main Library

Posté le par

Author : Ming Zhan

Data mining, as a heatedly discussed term, has been studied in various fields. Its possibilities in refining the decision-making process, realizing potential patterns and creating valuable knowledge have won attention of scholars and practitioners. However, there are less studies intending to combine data mining and libraries where data generation occurs all the time.

Therefore, this thesis plans to fill such a gap. Meanwhile, potential opportunities created by data mining are explored to enhance one of the most important elements of libraries: reference service. In order to thoroughly demonstrate the feasibility and applicability of data mining, literature is reviewed to establish a critical understanding of data mining in libraries and attain the current status of library reference service.

The result of the literature review indicates that free online data resources other than data generated on social media are rarely considered to be applied in current library data mining mandates. Therefore, the result of the literature review motivates the presented study to utilize online free resources. Furthermore, the natural match between data mining and libraries is established.

The natural match is explained by emphasizing the data richness reality and considering data mining as one kind of knowledge, an easy choice for libraries, and a wise method to overcome reference service challenges. The natural match, especially the aspect that data mining could be helpful for library reference service, lays the main theoretical foundation for the empirical work in this study.

Turku Main Library was selected as the case to answer the research question: whether data mining is feasible and applicable for reference service improvement. In this case, the daily visit from 2009 to 2015 in Turku Main Library is considered as the resource for data mining.

In addition, corresponding weather conditions are collected from Weather Underground, which is totally free online. Before officially being analyzed, the collected dataset is cleansed and preprocessed in order to ensure the quality of data mining.

Multiple regression analysis is employed to mine the final dataset. Hourly visits are the independent variable and weather conditions, Discomfort Index and seven days in a week are dependent variables. In the end, four models in different seasons are established to predict visiting situations in each season.

Patterns are realized in different seasons and implications are created based on the discovered patterns. In addition, library-climate points are generated by a clustering method, which simplifies the process for librarians using weather data to forecast library visiting situation. Then the data mining result is interpreted from the perspective of improving reference service.

After this data mining work, the result of the case study is presented to librarians so as to collect professional opinions regarding the possibility of employing data mining to improve reference services. In the end, positive opinions are collected, which implies that it is feasible to utilizing data mining as a tool to enhance library reference service.

URL : http://www.doria.fi/handle/10024/124215