Étiquette : scientific communication

Catégories
EN

Navigating the ethical landscape of scholarly publishing: a comparative evaluation of Gemini and DeepSeek LLMs in addressing authorship and contributorship disputes

Authors : Kannan Sridharan, Sivarama Krishnan

Background:

The rising complexity of publication ethics, particularly authorship disputes, necessitates exploring Large Language Models (LLMs) as potential evaluative tools. This study compares the performance of Google Gemini 2.5 Flash and DeepSeek-V3.2 against expert Committee on Publication Ethics (COPE) forum responses.

Methods:

A cross-sectional analysis including 12 COPE authorship and contributorship cases was conducted using three prompting strategies: Minimal, Deterministic, and Stochastic. Responses were scored across seven domains on a 5-point Likert scale (1 = poor, 5 = excellent) by independent raters.

Results:

Both LLMs achieved perfect scores (5 ± 0) in Actionability of Recommendations and high marks in Safety and Avoidance of Hallucination (4.88 ± 0.33). In the Consistency with COPE Principles domain, DeepSeek performed slightly better than Gemini (4.45 ± 1.0 vs. 4.12 ± 1.29), while Gemini showed a better Overall Appropriateness (4.03 ± 0.98 vs. 3.82 ± 1.29) but they were not statistically significant. Both models struggled most with Identification of Ethical Issues (Gemini: 3.91 ± 1.33; DeepSeek: 3.82 ± 1.29). Under Minimal prompts, Gemini’s ethical identification was lower (3.55 ± 1.44) compared to Deterministic/Stochastic prompts (4.09 ± 1.3). Qualitatively, Gemini recorded an 8% major disagreement rate with COPE, while DeepSeek had a 16% combined (minor and major) disagreement rate. Mean similarity scores to COPE forum experts were approximately 4 for both models. Both models missed specific legal/copyright nuances but provided unique “value-add” strategies, such as author disassociation statements and editorial de-escalation training, not present in original COPE forum advice.

Conclusion:

LLMs demonstrated high degree of alignment with COPE expert ethical reasoning. While they possess a “legal blind spot,” their ability to provide actionable and clear guidance, optimized through structured prompting, makes them valuable supplementary tools for journal editors.

URL : Navigating the ethical landscape of scholarly publishing: a comparative evaluation of Gemini and DeepSeek LLMs in addressing authorship and contributorship disputes

DOI : https://doi.org/10.3389/frma.2026.1781697

Catégories
EN

Library Publishing in Practice: A Case Study in Open Course Publications

Authors : Ioana Liuta, Jennifer Zerkee

Introduction: Open course publications provide students with real-world experience of the scholarly publishing process, engaging students as information creators rather than consumers. Open course publications, an example of open pedagogy in action, can be journals or monographs created as an assignment in a credit bearing course. Supporting open assignments is one of the most impactful activities undertaken by Digital Publishing units in academic libraries, educating the next generation of scholars about the value of open access. This article describes Simon Fraser University Library’s approach to supporting in-class publication projects, focusing on in-class open monographs.

Description of the service: The Digital Publishing Librarian and Copyright Specialist collaborate with an instructor to plan support for their course publication. This includes working with the instructor to plan the project; providing an in-class workshop on key scholarly publishing topics, including an introduction to open access and Creative Commons licences; ongoing support as needed through the semester; and production and publication of the finalized monograph.

Next steps: The Library is currently addressing long-term sustainability needs for these publications. The authors are considering further opportunities for outreach to instructors beyond the humanities and social sciences, as well as potential connections to undergraduate research activities, while recognizing the capacity required to provide and expand this service.

URL : Library Publishing in Practice: A Case Study in Open Course Publications

DOI : https://doi.org/10.31274/jlsc.21364

Catégories
EN

Diverse roles of twitter in research evaluation: original tweets and retweets capture different types of engagements with scholarly articles

Authors :  Ashraf Maleki, Kim Holmberg

Altmetrics need to be more critically assessed in terms of the extent to which they reflect impact and quality of research compared to popularity or mere attention. Twitter (now rebranded as X) is a popular platform to, among other things, discuss and share scientific articles.

Earlier altmetric studies have often focused on investigating whether the number of tweets mentioning scientific articles could be used as an indicator of scientific impact or attention, with results showing weak to moderate correlations with citation counts. But all tweets may not be equal, as original tweets and retweets may reflect different levels of engagement and impact. Using a dataset of over 330,000 PLOS publications, this study explores whether these two forms of Twitter activity correlate differently with traditional citation metrics and how these relationships vary across disciplines.

The findings showed the correlation between citations and original tweets was consistently higher than that between citations and retweets and significant weak or moderate, but higher in Social Science and Humanities than in Natural Science, Engineering and Medicine fields. Also, including zero citation counts improved the correlation coefficients for original tweets, but reduced that of retweets.

This indicates that original tweets may be more aligned with citation counts as an indicator of scholarly impact, whereas retweets might reflect broader dissemination and popularity. In conclusion, tweets and retweets are different altmetric indicators and should be considered as two different metrics and analysed separately.

URL : Diverse roles of twitter in research evaluation: original tweets and retweets capture different types of engagements with scholarly articles

DOI : https://doi.org/10.1093/reseval/rvag014

Catégories
EN

Evaluating Open Access Advantages for Citations and Altmetrics (2011-21): A Dynamic and Evolving Relationship

Author : Mike Taylor

Differences between the impacts of Open Access (OA) and non-OA research have been observed over a range of citation and altmetric indicators, usually finding an Open Access Advantage (OAA). However, science-wide analyses covering multiple years, indicators and disciplines are lacking. Using citations and six altmetrics for 33.3M articles published 2011-21, we compare OA and non-OA papers.

The results show that there is no universal OAA across all disciplines or impact indicators: the OAA for citations tends to be lower for recent papers, whereas the OAAs for news, blogs and Twitter are consistent across years and unrelated to volume of OA publications. Wikipedia OAAs are consistently pronounced for all subjects except Humanities (HU) and Social Sciences. Patent OAAs for are strongest for Medical & Health Sciences (MHS) and Life Sciences (LS).

Uniquely, the OAAs for Policy citations is stronger for recently published research. These results support different hypotheses for different subjects and indicators. The evidence is consistent with OA accelerating research impact in MHS, LS and HU; increased visibility/discoverability being a factor in promoting the socio-economic impact; and that OA is a factor in growing online engagement with research. OAAs are therefore complex, dynamic, multi-factorial and require considerable analysis to understand.

URL : Evaluating Open Access Advantages for Citations and Altmetrics (2011-21): A Dynamic and Evolving Relationship

DOI : Serendipity and Scientific Styles: An Ordinary

Catégories
EN

The independence paradox in scientific careers

Authors : Yanmeng Xing, Ye Sun, Tongxin Pan, Giacomo Livan, Yifang Ma

Establishing an independent academic identity is a central yet insufficiently understood challenge for early-career researchers. However, limited resources and mentor-driven research agendas often constrain early efforts toward autonomy.

To provide large-scale quantitative evidence on how junior researchers develop independence, we introduce a framework that traces how mentees diverge from their mentors in both research topics and collaboration networks, and how these divergences relate to long-term scientific impact.

Analyzing over 500,000 mentee-mentor pairs in Chemistry, Neuroscience, and Physics across six decades, we find that high-impact scientists often initiate work in secondary areas of their mentors’ expertise while adaptively establishing distinct research trajectories. This pattern is most pronounced among mentees who eventually surpass their mentors’ impact.

We identify an inverted U-shaped relationship between topic divergence and mentees’ enduring impact, with moderate divergence yielding the highest scientific impact, revealing an independence paradox in scientific careers.

This pattern holds whether topic divergence is measured by citation network or semantic thematic distance. We further reveal that excessive direct mentor-mentee collaborations correlate with lower mentee impact, whereas expanding professional networks to include mentors’ collaborators is beneficial.

These findings not only offer actionable guidance for early-career researchers navigating independence but also inform institutional policies that promote mentorship structures supporting intellectual innovation and recognizing original contributions in promotion evaluations.

DOI : https://doi.org/10.48550/arXiv.2408.16992

Catégories
EN

Determining quality dimensions for peer review reports using a Delphi approach

Authors : Amanda Sizo, Adriano Lino, Álvaro Rocha, Luis Paulo Reis

The quality of peer review reports is essential to the integrity and effectiveness of scholarly communication. Yet review reports are often criticized for being vague, biased, or unconstructive, which limits their usefulness for both authors and editors. Existing frameworks for assessing review quality remain fragmented and are rarely validated through expert consensus.

This study aims to define and validate a comprehensive set of quality dimensions for peer review reports, encompassing comments addressed to both authors and editors. We employed a two-phase design combining a thematic analysis of the literature with a Delphi study involving 43 scientific editors, primarily from journals in Computer Science and Engineering.

Consensus was reached after two Delphi rounds, resulting in 62 validated statements organized into eight quality dimensions: Helpfulness, Specificity, Fairness, Thoroughness, Courteousness, Readability, Consistency, and Relevance. These findings provide an empirically grounded framework to inform the development of clearer standards for peer review practice.

URL : Determining quality dimensions for peer review reports using a Delphi approach

DOI : https://doi.org/10.1007/s11192-026-05603-3

Catégories
EN

Evaluating Multilingual Metadata Quality in Crossref

Authors : Dennis Donathan, Mike Nason, Marco Tullney, Julie Shi, Juan Pablo Alperin

Introduction

Scholarly research spans multiple languages, making multilingual metadata crucial for organizing and accessing knowledge across linguistic boundaries. These multilingual metadata already exist and are propagated throughout the scholarly publishing infrastructure, but the extent to which they are correctly recorded, or how they affect metadata quality more broadly, is little understood.

Methods

Our study quantifies the prevalence of multilingual records across a sample of publisher metadata and offers an understanding of their completeness, quality, and alignment with metadata standards.

Utilizing the Crossref API to generate a random sample of 519,665 journal article records, we categorize each record into four distinct language types: English monolingual, non-English monolingual, multilingual, and uncategorized. We then investigate the prevalence of programmatically detectable errors and the prevalence of multilingual records within the sample to determine whether multilingualism influences the quality of article metadata.

Results

We find that English-only records are still in the vast majority among metadata found in Crossref, but that, while non-English and multilingual records present unique challenges, they are not a source of significant metadata quality issues and, in a few instances, are more complete or correct than English monolingual records.

Discussion & Conclusion

Our findings contribute to discussions surrounding multilingualism in scholarly communication, serving as a resource for researchers, publishers, and information professionals seeking to enhance the global dissemination of knowledge and foster inclusivity in the academic landscape.

URL : Evaluating Multilingual Metadata Quality in Crossref

DOI : https://doi.org/10.31274/jlsc.19779