The role of non-scientific factors vis-a-vis the quality of publications in determining their scholarly impact

Authors : Giovanni Abramo, Ciriaco Andrea D’Angelo, Leonardo Grilli

In the evaluation of scientific publications’ impact, the interplay between intrinsic quality and non-scientific factors remains a subject of debate. While peer review traditionally assesses quality, bibliometric techniques gauge scholarly impact. This study investigates the role of non-scientific attributes alongside quality scores from peer review in determining scholarly impact.

Leveraging data from the first Italian Research Assessment Exercise (VTR 2001-2003) and Web of Science citations, we analyse the relationship between quality scores, non-scientific factors, and publication short- and long-term impact.

Our findings shed light on the significance of non-scientific elements overlooked in peer review, offering policymakers and research management insights in choosing evaluation methodologies. Sections delve into the debate, identify non-scientific influences, detail methodologies, present results, and discuss implications.

Arxiv : https://arxiv.org/abs/2404.05345

Sentiment Analysis of Citations in Scientific Articles Using ChatGPT: Identifying Potential Biases and Conflicts of Interest

Author : Walid Hariri

Scientific articles play a crucial role in advancing knowledge and informing research directions. One key aspect of evaluating scientific articles is the analysis of citations, which provides insights into the impact and reception of the cited works. This article introduces the innovative use of large language models, particularly ChatGPT, for comprehensive sentiment analysis of citations within scientific articles.

By leveraging advanced natural language processing (NLP) techniques, ChatGPT can discern the nuanced positivity or negativity of citations, offering insights into the reception and impact of cited works. Furthermore, ChatGPT’s capabilities extend to detecting potential biases and conflicts of interest in citations, enhancing the objectivity and reliability of scientific literature evaluation.

This study showcases the transformative potential of artificial intelligence (AI)-powered tools in enhancing citation analysis and promoting integrity in scholarly research.

Arxiv : https://arxiv.org/abs/2404.01800

Text mining arXiv: a look through quantitative finance papers

Author : Michele Leonardo Bianchi

This paper explores articles hosted on the arXiv preprint server with the aim to uncover valuable insights hidden in this vast collection of research. Employing text mining techniques and through the application of natural language processing methods, we examine the contents of quantitative finance papers posted in arXiv from 1997 to 2022.

We extract and analyze crucial information from the entire documents, including the references, to understand the topics trends over time and to find out the most cited researchers and journals on this domain. Additionally, we compare numerous algorithms to perform topic modeling, including state-of-the-art approaches.

Arxiv : https://arxiv.org/abs/2401.01751

Authorship conflicts in academia: an international cross‑discipline survey

Authors : Elizaveta Savchenko, Ariel Rosenfeld

Collaboration among scholars has emerged as a significant characteristic of contemporary science. As a result, the number of authors listed in publications continues to rise steadily. Unfortunately, determining the authors to be included in the byline and their respective order entails multiple difficulties which often lead to conflicts. Despite the large volume of literature about conflicts in academia, it remains unclear how exactly these are distributed over the main socio-demographic properties, as well as the different types of interactions academics experience.

To address this gap, we conducted an international and cross-disciplinary survey answered by 752 academics from 41 fields of research and 93 countries that statistically well-represent the overall academic workforce. Our findings are concerning and suggest that conflicts over authorship credit arise very early in one’s academic career, even at the level of Master and Ph.D., and become increasingly common over time.

URL : Authorship conflicts in academia: an international cross‑discipline survey

DOI : https://doi.org/10.1007/s11192-024-04972-x

The impact of researchers’ perceived pressure on their publication strategies

Authors : David Johann, Jorg Neufeld, Kathrin Thomas, Justus Rathmann, Heiko Rauhut

This article investigates researchers’ publication strategies and how their perceived pressure to publish and to obtain external funding are related to these strategies. The analyses rely on data from the Zurich Survey of Academics (ZSoA), an online survey representative of academics working at higher education institutions in Germany, Austria, and Switzerland. The results suggest that academics pursue both instrumental and normative publication strategies.

The main finding is that academics who perceive high pressure to publish tend to employ instrumental publication strategies rather than normative ones: they are more likely to focus on the journal’s reputation and the speed of publication when selecting an outlet for peer review. Publishing results in open-access outlets or in native languages other than English is less important for those under pressure.

However, the extent to which researchers’ perceived pressure affects publication strategies also depends on other factors, such as the discrepancy between the time available for research and the time actually desired for research.

URL : The impact of researchers’ perceived pressure on their publication strategies

DOI : https://doi.org/10.1093/reseval/rvae011

Evolution of Peer Review in Scientific Communication

Author : Dmitry Kochetkov

It is traditionally believed that peer review is the backbone of an academic journal and scientific communication, ensuring high quality and trust in the published materials. However, peer review only became an institutionalized practice in the second half of the 20th century, although the first scientific journals appeared three centuries earlier. By the beginning of the 21st century, there emerged an opinion that the traditional model of peer review is in deep crisis.

The aim of this article is to formulate a perspective model of peer review for scientific communication. The article discusses the evolution of the institution of scientific peer review and the formation of the current crisis. The author analyzed the modern landscape of innovations in peer review and scientific communication. Based on this analysis, three main peer review models in relation to editorial workflow were identified: pre-publication peer review (traditional model),  registered reports, and post-publication (peer) review (including preprints (peer) review).

The author argues that the third model offers the best way to implement the main functions of scientific communication.

URL : Evolution of Peer Review in Scientific Communication

DOI : https://doi.org/10.31235/osf.io/b2ra3