Peer review’s irremediable flaws: Scientists’ perspectives on grant evaluation in Germany

Authors : Eva Barlösius, Laura Paruschke, Axel Philipps

Peer review has developed over time to become the established procedure for assessing and assuring the scientific quality of research. Nevertheless, the procedure has also been variously criticized as conservative, biased, and unfair, among other things. Do scientists regard all these flaws as equally problematic?

Do they have the same opinions on which problems are so serious that other selection procedures ought to be considered? The answers to these questions hints at what should be modified in peer review processes as a priority objective. The authors of this paper use survey data to examine how members of the scientific community weight different shortcomings of peer review processes.

Which of those processes’ problems do they consider less relevant? Which problems, on the other hand, do they judge to be beyond remedy? Our investigation shows that certain defects of peer review processes are indeed deemed irreparable: (1) legitimate quandaries in the process of fine-tuning the choice between equally eligible research proposals and in the selection of daring ideas; and (2) illegitimate problems due to networks. Science-policy measures to improve peer review processes should therefore clarify the distinction between field-specific remediable and irremediable flaws than is currently the case.

URL : Peer review’s irremediable flaws: Scientists’ perspectives on grant evaluation in Germany


Platformisation of Science. Conceptual Foundations and Critical Perspectives for the Science System

Authors : Benedikt Fecher, Raffaela Kunz, Nataliia Sokolovska, Marcel Wrzesinski

The digital platforms we are dealing with in this article are auxiliary tools that do not produce anything themselves but provide an infrastructure for service providers and users to meet. They have potentially unlimited scaling potential and have become the central places of exchange.

In academia we can also observe that research and its communication becomes more digital and that digital services are aiming to become platforms. In this article we explore the concept of digital platforms and their potential impact on academic research, firstly addressing the question: To what extent can digital platforms be understood as a specific type of research infrastructure?

We draw from recent literature on platforms and platformisation from different streams of scholarship and relate them to the science studies concept of research infrastructures, to eventually arrive at a framework for science platforms. Secondly, we aim to assess how science platforms may affect scholarly practice. To this end, we relate common platform practices to scientific practice. Thirdly, we aim to assess to what extent science is platformized and how this interferes with scientific understandings of quality and autonomy.

In the end of this article, we argue that the potential benefits of platform infrastructure for academic pursuits cannot be ignored, but the commercialization of the infrastructure for scholarly communication is a cause for concern. Ultimately, a nuanced and well-informed perspective on the impact of platformisation on academia is necessary to ensure that the academic community can maximize the benefits of digital infrastructures while mitigating negative consequences.

URL : Platformisation of Science. Conceptual Foundations and Critical Perspectives for the Science System


How transformative are transformative agreements? Evidence from Germany across disciplines

Author : W. Benedikt Schmal

Research institutions across the globe attempt to change the academic publishing system as digitization opens up new opportunities, and subscriptions to the large journal bundles of the leading publishers put library budgets under pressure. One approach is the negotiation of so-called transformative agreements.

I study the ‘DEAL’ contracts between nearly all German research institutions and Springer Nature and Wiley. I investigate 6.1 million publications in 5,862 journals covering eight fields in the years 2016–2022 and apply a causal difference-in-differences design to identify whether the likelihood of a paper appearing in an eligible journal increases. The effect strongly depends on the discipline.

While material science, chemistry, and economics s tend to hift towards these journals, all other disciplines in my sample do not react. Suggestive evidence hints at the market position of the encompassed publishers before the ‘DEAL’ was established: Springer Nature and Wiley appear to benefit more from the contracts in disciplines in which they possessed a higher market share ex ante.

The transformative vigor of these agreements in terms of publication behavior seems to be limited. It and highlights that the developments in this intertwined market require further examination.

URL : How transformative are transformative agreements? Evidence from Germany across disciplines


Scholar Metrics Scraper (SMS): automated retrieval of citation and author data

Authors : Yutong Cao, Nicole A. Cheung, Dean Giustini, Jeffrey LeDue, Timothy H. Murphy

Academic departments, research clusters and evaluators analyze author and citation data to measure research impact and to support strategic planning. We created Scholar Metrics Scraper (SMS) to automate the retrieval of bibliometric data for a group of researchers.

The project contains Jupyter notebooks that take a list of researchers as an input and exports a CSV file of citation metrics from Google Scholar (GS) to visualize the group’s impact and collaboration. A series of graph outputs are also available. SMS is an open solution for automating the retrieval and visualization of citation data.

URL : Scholar Metrics Scraper (SMS): automated retrieval of citation and author data


Bibliometric analysis of Sci-Hub downloads by Egyptian researchers

Authors : Ismail Ragab Osman, Hendy Abdullah Hendy Ahmed

In this study we present an in-depth bibliometric analysis of Sci-Hub downloads by Egyptian researchers based on the 2017 download log file. The study reveals that Egyptian researchers heavily rely on Sci-Hub, generating a substantial 1,357,526 download requests in 2017, with 65% of these occurring outside regular working hours. Cairo emerges as a central hub for this activity, contributing 81.58% of total downloads.

Journal articles constitute the majority of downloads at 82.36%, followed by conference papers (12.89%). A discernible trend shows a preference for recent papers published between 2012 and 2017, highlighting the demand for up-to-date research. The analysis also highlights prominent publishers, including IEEE, Elsevier, Wiley, and Springer, as preferred sources for Egyptian researchers. “Journal of the American Chemical Society” and “Journal of Applied Physics” stand out among accessed journals, while IEEE-associated conferences, notably “IEEE Power and Energy Society General Meeting,” dominate conference paper downloads. Examining journal accessibility via the Egyptian Knowledge Bank (EKB) reveals that 62.84% of journals are accessible, with Science Direct as the leading provider (28.37%).

However, a significant gap emerges as 87.39% of downloaded conference papers remain inaccessible through EKB. Furthermore, a semantic analysis highlights recurring themes such as “systems,” “review,” “analysis,” “treatment,” “power,” and “energy,” reflecting the key research areas of Egyptian researchers. Overall, this study offers valuable insights into Sci-Hub’s role in supplementing Egyptian researchers’ resource access and underscores the need for comprehensive resource coverage and accessibility enhancements.

URL : Bibliometric analysis of Sci-Hub downloads by Egyptian researchers


The Nexus of Open Science and Innovation: Insights from Patent Citations

Author : Abdelghani Maddi

This paper aims to analyze the extent to which inventive activity relies on open science. In other words, it investigates whether inventors utilize Open Access (OA) publications more than subscription-based ones, especially given that some inventors may lack institutional access.

To achieve this, we utilized the (Marx, 2023) database, which contains citations of patents to scientific publications (Non-Patent References-NPRs). We focused on publications closely related to invention, specifically those cited solely by inventors within the body of patent texts. Our dataset was supplemented by OpenAlex data.

The final sample comprised 961,104 publications cited in patents, of which 861,720 had a DOI. Results indicate that across all disciplines, OA publications are 38% more prevalent in patent citations (NPRs) than in the overall OpenAlex database.

In biology and medicine, inventors use 73% and 27% more OA publications, respectively, compared to closed-access ones. Chemistry and computer science are also disciplines where OA publications are more frequently utilized in patent contexts than subscription-based ones.


Beyond journals and peer review: towards a more flexible ecosystem for scholarly communication

Author : Michael Wood

This article challenges the assumption that journals and peer review are essential for developing,evaluating and disseminating scientific and other academic knowledge. It suggests a more flexible ecosystem, and examines some of the possibilities this might facilitate. The market for academic outputs should be opened up by encouraging the separation of the dissemination service from the evaluation service.

Publishing research in subject-specific journals encourages compartmentalising research into rigid categories. The dissemination of knowledge would be better served by an open access, web-based repository system encompassing all disciplines. There would then be a role for organisations to assess the items in this repository to help users find relevant, high-quality work.

There could be a variety of such organisations which could enable reviews from peers to be supplemented with evaluation by non-peers from a variety of different perspectives: user reviews, statistical reviews, reviews from the perspective of different disciplines, and so on. This should reduce the inevitably conservative influence of relying on two or three peers, and make the evaluation system more critical, multi-dimensional and responsive to the requirements of different audience groups, changing circumstances, and new ideas.

Non-peer review might make it easier to challenge dominant paradigms, and expanding the potential audience beyond a narrow group of peers might encourage the criterion of simplicity to be taken more seriously – which is essential if human knowledge is to continue to progress.

Arxiv :