Archives des mots-clés : research impact

Forecasting the publication and citation outcomes of COVID-19 preprints

Posté le par

Authors : Michael Gordon, Michael Bishop, Yiling Chen, Anna Dreber, Brandon Goldfedder, Felix Holzmeister, Magnus Johannesson, Yang Liu, Louisa Tran, Charles Twardy, Juntao Wang, Thomas Pfeiffer

Many publications on COVID-19 were released on preprint servers such as medRxiv and bioRxiv. It is unknown how reliable these preprints are, and which ones will eventually be published in scientific journals.

In this study, we use crowdsourced human forecasts to predict publication outcomes and future citation counts for a sample of 400 preprints with high Altmetric score. Most of these preprints were published within 1 year of upload on a preprint server (70%), with a considerable fraction (45%) appearing in a high-impact journal with a journal impact factor of at least 10.

On average, the preprints received 162 citations within the first year. We found that forecasters can predict if preprints will be published after 1 year and if the publishing journal has high impact. Forecasts are also informative with respect to Google Scholar citations within 1 year of upload on a preprint server.

For both types of assessment, we found statistically significant positive correlations between forecasts and observed outcomes. While the forecasts can help to provide a preliminary assessment of preprints at a faster pace than traditional peer-review, it remains to be investigated if such an assessment is suited to identify methodological problems in preprints.

URL : Forecasting the publication and citation outcomes of COVID-19 preprints

DOI : https://doi.org/10.1098/rsos.220440

Impact factions: assessing the citation impact of different types of open access repositories

Posté le par

Authors : Jonathan Wheeler, Ngoc‑Minh Pham, Kenning Arlitsch, Justin D. Shanks

Institutional repositories (IR) maintained by research libraries play a central role in providing open access to taxpayer-funded research products. It is difficult to measure the extent to which IR contribute to new scholarship because publisher self-archiving policies typically require researchers to cite the “version of record” of a manuscript even when an IR copy is accessed to conduct the research.

While some studies report an open access (OA) citation advantage resulting from the availability of self-archived or “green” OA manuscripts, few have sought to measure an OA citation effect of IR separately from disciplinary repositories, including arXiv and PubMed Central.

In this study, the authors present a bibliometric analysis examining correlations between search engine performance of items in IR, OA availability from different types of repositories, and citations. The analysis uses a novel, open dataset of IR access and usage derived from five months of Google search engine results pages (SERP) data, which were aggregated by the Repository Analytics and Metrics Portal (RAMP) web service.

Findings indicate that making OA copies of manuscripts available in self-archiving or “green” repositories results in a positive citation effect, although the disciplinary repositories within the sample significantly outperform the other types of OA services analyzed. Also evident is an increase in citations when a single manuscript is available in multiple OA sources.

URL : Impact factions: assessing the citation impact of different types of open access repositories

DOI : https://doi.org/10.1007/s11192-022-04467-7

Does Open Access Really Increase Impact? A Large-Scale Randomized Analysis

Posté le par

Authors : Abdelghani Maddi, David Sapinho

The Open Access Citation Advantage (OACA) has been a major topic of discussion in the literature over the past twenty years. In this paper, we propose a method to constitute a control group to isolate the OACA effect. Thus, we compared citation impact (MNCS) of 2,458,378 publications in fully OA journals to that (weighted MNCS) of a control group of non-OA publications (#10,310,842).

Similarly, we did the same exercise for OA publications in hybrid journals (#1,024,430) and their control group (#11,533,001), over the period 2010-2020. The results showed that there is no OACA for publications in fully OA journals, and that there is rather a disadvantage. Conversely, the OACA seems to be a reality in hybrid journals, suggesting that a better accessibility tends to improve the impact of publications.

The lack of OACA for publications in fully OA journals is to be expected, as a great proportion of OA journals are newly created and less attractive to high-impact senior researchers. Another striking result of this paper is the fall of the OACA from 2016.

The citation advantage fell from 70% to 9% between 2016 and 2020 (for publications in hybrid journals). We wonder if this fall is linked to the increase in the notoriety of pirate sites (eg Sci-Hub) from 2016.

In other words, the democratization of pirate sites instantly cancels the positive effect of OA publication insofar as the question of access to scientific content no longer arises.

URL: https://hal.archives-ouvertes.fr/hal-03694666

Biosecurity in an age of open science

Posté le par

Authors : James Andrew Smith, Jonas B. Sandbrink

The risk of accidental or deliberate misuse of biological research is increasing as biotechnology advances. As open science becomes widespread, we must consider its impact on those risks and develop solutions that ensure security while facilitating scientific progress.

Here, we examine the interaction between open science practices and biosecurity and biosafety to identify risks and opportunities for risk mitigation. Increasing the availability of computational tools, datasets, and protocols could increase risks from research with misuse potential.

For instance, in the context of viral engineering, open code, data, and materials may increase the risk of release of enhanced pathogens. For this dangerous subset of research, both open science and biosecurity goals may be achieved by using access-controlled repositories or application programming interfaces. While preprints accelerate dissemination of findings, their increased use could challenge strategies for risk mitigation at the publication stage.

This highlights the importance of oversight earlier in the research lifecycle. Preregistration of research, a practice promoted by the open science community, provides an opportunity for achieving biosecurity risk assessment at the conception of research.

Open science and biosecurity experts have an important role to play in enabling responsible research with maximal societal benefit.

URL : Biosecurity in an age of open science

DOI : https://doi.org/10.1371/journal.pbio.3001600

Free access to scientific literature and its influence on the publishing activity in developing countries: The effect of Sci-Hub in the field of mathematics

Posté le par

Authors : Kilian Buehling, Matthias Geissler, Dorothea Strecker

This paper investigates whether free access to scientific literature increases the participation of under-represented groups in scientific discourse. To this end, we aggregate and match data tracing access to Sci-Hub, a widely used black open access (OA) repository or shadow library, and publication data from the Web of Science (WoS).

We treat the emergence of Sci-Hub as an exogenous event granting relatively unrestricted access to publications, which are otherwise hidden behind a paywall. We analyze changes in the publication count of researchers from developing countries in a given journal as a proxy for general participation in scientific discourse.

Our results indicate that in the exemplary field of mathematics, free access to academic knowledge is likely to improve the representation of authors from developing countries in international journals.

Assuming the desirability of greater international diversity in science (e.g., to generate more original work, reproduce empirical findings in different settings, or shift the research focus toward topics that are overlooked by researchers from more developed countries), our findings lend evidence to the claim of the OA movement that scientific knowledge should be free and widely distributed.

URL : Free access to scientific literature and its influence on the publishing activity in developing countries: The effect of Sci-Hub in the field of mathematics

DOI : https://doi.org/10.1002/asi.24636

Reshaping How Universities Can Evaluate the Research Impact of Open Humanities for Societal Benefit

Posté le par

Authors : Paul Longley Arthur, Lydia Hearn

During the twenty-first century, for the first time, the volume of digital data has surpassed the amount of analog data. As academic practices increasingly become digital, opportunities arise to reshape the future of scholarly communication through more accessible, interactive, open, and transparent methods that engage a far broader and more diverse public.

Yet despite these advances, the research performance of universities and public research institutes remains largely evaluated through publication and citation analysis rather than by public engagement and societal impact.

This article reviews how changes to bibliometric evaluations toward greater use of altmetrics, including social media mentions, could enhance uptake of open scholarship in the humanities.

In addition, the article highlights current challenges faced by the open scholarship movement, given the complexity of the humanities in terms of its sources and outputs that include monographs, book chapters, and journals in languages other than English; the use of popular media not considered as scholarly papers; the lack of time and energy to develop digital skills among research staff; problems of authority and trust regarding the scholarly or non-academic nature of social media platforms; the prestige of large academic publishing houses; and limited awareness of and familiarity with advanced digital applications.

While peer review will continue to be a primary method for evaluating research in the humanities, a combination of altmetrics and other assessment of research impact through different data sources may provide a way forward to ensure the increased use, sustainability, and effectiveness of open scholarship in the humanities.

DOI : https://doi.org/10.3998/jep.788

Article Processing Charges, Altmetrics and Citation Impact: Is there an economic rationale?

Posté le par

Authors : Abdelghani Maddi, David Sapinho

The present study aims to analyze 1) the relationship between Citation Normalized Score of scientific publications and Article Processing Charges (APCs) of Gold Open Access (OA) publications 2) the determinants of APCs.

To do so, we used APCs information provided by the OpenAPC database, citation scores of publications from the WoS database and, for Altmetrics, data from Altmetrics.com database, over the period from 2006 to 2019 for 83,752 articles published in 4751 journals belonging to 267 distinct publishers.

Results show that contrary to common belief, paying high APCs does not necessarily increase the impact of publications. First, large publishers with high impact are not the most expensive. Second, publishers with the highest APCs are not necessarily the best in terms of impact.

Correlation between APCs and impact is moderate. Regarding the determinants, results indicate that APCs are on average 50% higher in hybrid journals than in full OA journals. The results also suggest that Altmetrics do not have a great impact: OA articles that have garnered the most attention on internet are articles with relatively low APCs.

Another interesting result is that the “number of readers” indicator is more effective as it is more correlated with classic bibliometrics indicators than the Altmetrics score.

URL : https://hal.archives-ouvertes.fr/hal-03576911