Étiquette : scientific communication

Catégories
EN

Publish and Who Should Perish: You or Science?

Author : Ádám Kun

Something is wrong with science as there is an increasing amount of unreliable, manipulated and outright faked results appearing in the literature. Here I argue that this is a direct consequence of the pay-structure and the assessment system employed in academia and it could be remedied by changing hiring, advancement, and funding criteria.

Scientists are paid below average relative to their level of education, unless they are at the top or can secure grants that allow for higher salaries. Positions and grants are mostly awarded based on bibliometric numbers.

Consequently, there is a strong competition to accumulate numbers of papers, impact factors, and citations. Those who can increase their value efficiently will be rewarded and the accumulation of higher values will become easier (the Matthew effect).

Higher bibliometric numbers can be obtained by unethical or questionable practices, which might tempt some people. If assessments did not employ bibliometric numbers, then these practices would not have a benefit, and would fade out.

Throughout the text, data from Hungary, which are similar to data from elsewhere, supplement the argument.

URL : Publish and Who Should Perish: You or Science?

Alternative location : http://www.mdpi.com/2304-6775/6/2/18

Catégories
EN

Assessment of potential bias in research grant peer review in Canada

Authors : Robyn Tamblyn, Nadyne Girard, Christina J. Qian, James Hanley

BACKGROUND

Peer review is used to determine what research is funded and published, yet little is known about its effectiveness, and it is suspected that there may be biases. We investigated the variability of peer review and factors influencing ratings of grant applications.

METHODS

We evaluated all grant applications submitted to the Canadian Institutes of Health Research between 2012 and 2014. The contribution of application, principal applicant and reviewer characteristics to overall application score was assessed after adjusting for the applicant’s scientific productivity.

RESULTS

Among 11 624 applications, 66.2% of principal applicants were male and 64.1% were in a basic science domain. We found a significant nonlinear association between scientific productivity and final application score that differed by applicant gender and scientific domain, with higher scores associated with past funding success and h-index and lower scores associated with female applicants and those in the applied sciences.

Significantly lower application scores were also associated with applicants who were older, evaluated by female reviewers only (v. male reviewers only, −0.05 points, 95% confidence interval [CI] −0.08 to −0.02) or reviewers in scientific domains different from the applicant’s (−0.07 points, 95% CI −0.11 to −0.03).

Significantly higher application scores were also associated with reviewer agreement in application score (0.23 points, 95% CI 0.20 to 0.26), the existence of reviewer conflicts (0.09 points, 95% CI 0.07 to 0.11), larger budget requests (0.01 points per $100 000, 95% CI 0.007 to 0.02), and resubmissions (0.15 points, 95% CI 0.14 to 0.17).

In addition, reviewers with high expertise were more likely than those with less expertise to provide higher scores to applicants with higher past success rates (0.18 points, 95% CI 0.08 to 0.28).

INTERPRETATION

There is evidence of bias in peer review of operating grants that is of sufficient magnitude to change application scores from fundable to nonfundable. This should be addressed by training and policy changes in research funding.

URL : https://doi.org/10.1503/cmaj.170901

Catégories
EN

Is Predatory Publishing a Real Threat? Evidence from a Large Database Study

Authors : Marcelo Perlin, Takeyoshi Imasato, Denis Borenstein

Using a database of potential, possible, or probable predatory scholarly open-access journals, the objective of this research is to study the penetration of predatory publications in the Brazilian academic system and the profile of authors in a cross-section empirical study.

Based on a massive amount of publications from Brazilian researchers of all disciplines during the 2000 to 2015 period, we were able to analyze the extent of predatory publications using an econometric modeling.

Descriptive statistics indicate that predatory publications represent a small overall proportion, but grew exponentially in the last 5 years. Departing from prior studies, our analysis shows that experienced researchers with a high number of non-indexed publications and PhD obtained locally are more likely to publish in predatory journals.

Further analysis shows that once a journal regarded as predatory is listed in the local ranking system, the Qualis, it starts to receive more publications than non-predatory ones.

DOI : https://dx.doi.org/10.2139/ssrn.3067958

Catégories
EN

The Journal Impact Factor: A brief history, critique, and discussion of adverse effects

Authors : Vincent Lariviere, Cassidy R. Sugimoto

The Journal Impact Factor (JIF) is, by far, the most discussed bibliometric indicator. Since its introduction over 40 years ago, it has had enormous effects on the scientific ecosystem: transforming the publishing industry, shaping hiring practices and the allocation of resources, and, as a result, reorienting the research activities and dissemination practices of scholars.

Given both the ubiquity and impact of the indicator, the JIF has been widely dissected and debated by scholars of every disciplinary orientation. Drawing on the existing literature as well as on original research, this chapter provides a brief history of the indicator and highlights well-known limitations-such as the asymmetry between the numerator and the denominator, differences across disciplines, the insufficient citation window, and the skewness of the underlying citation distributions.

The inflation of the JIF and the weakening predictive power is discussed, as well as the adverse effects on the behaviors of individual actors and the research enterprise. Alternative journal-based indicators are described and the chapter concludes with a call for responsible application and a commentary on future developments in journal indicators

URL : https://arxiv.org/abs/1801.08992

Catégories
EN

Comparing scientific and technological impact of biomedical research

Author : Qing Ke

Traditionally, the number of citations that a scholarly paper receives from other papers is used as the proxy of its scientific impact. Yet citations can come from domains outside the scientific community, and one such example is through patented technologies—paper can be cited by patents, achieving technological impact.

While the scientific impact of papers has been extensively studied, the technological aspect remains largely unknown. Here we aim to fill this gap by presenting a comparative study on how 919 thousand biomedical papers are cited by U.S. patents and by other papers over time.

We observe a positive correlation between citations from patents and from papers, but there is little overlap between the two domains in either the most cited papers, or papers with the most delayed recognition.

We also find that the two types of citations exhibit distinct temporal variations, with patent citations lagging behind paper citations for a median of 6 years for the majority of papers. Our work contributes to the understanding of the technological, and societal in general, impact of papers.

URL : https://arxiv.org/abs/1804.04105

Catégories
EN

Tweet success? Scientific communication correlates with increased citations in Ecology and Conservation

Authors : Clayton T. Lamb​, Sophie L. Gilbert, Adam T. Ford

Science communication is seen as critical for the disciplines of ecology and conservation, where research products are often used to shape policy and decision making. Scientists are increasing their online media communication, via social media and news.

Such media engagement has been thought to influence or predict traditional metrics of scholarship, such as citation rates. Here, we measure the association between citation rates and the Altmetric Attention Score—an indicator of the amount and reach of the attention an article has received—along with other forms of bibliometric performance (year published, journal impact factor, and article type).

We found that Attention Score was positively correlated with citation rates. However, in recent years, we detected increasing media exposure did not relate to the equivalent citations as in earlier years; signalling a diminishing return on investment.

Citations correlated with journal impact factors up to ∼13, but then plateaued, demonstrating that maximizing citations does not require publishing in the highest-impact journals. We conclude that ecology and conservation researchers can increase exposure of their research through social media engagement and, simultaneously, enhance their performance under traditional measures of scholarly activity.

URL : Tweet success? Scientific communication correlates with increased citations in Ecology and Conservation

DOI : https://doi.org/10.7717/peerj.4564

Catégories
EN

Pubpeer: vigilante science, journal club or alarm raiser? The controversies over anonymity in post-publication peer review

Author : Didier Torny

The more journal peer review (JPR) became a scientific topic, the more it has been the subject of criticisms and controversies. Repeated fake reports, confirmed reviewers’ biases, lack of reproducibility, and a recurrent inability to detect fraud and misconduct have apparently condemned JPR in its supposedly traditional form.

In fact, just like previous historical reforms and inventions, JPR has again been the object of many proposals to “fix it” since the beginning of the 21st century. Though these proposals are very diverse, two main directions have been identified: open peer review on one side, post-publication peer review (PPPR) on the other.

These two “fixes” share a common device, the open commenting of published articles, which is both an open peer review practice as it is visible to all readers and PPPR as it comes after the publication and often the certification of articles. At their intersection, it should thus thrive and indeed many journals have proposed this feature, but with no success.

Nevertheless, there is an exception to the disappointment with open commentary in PPPR, which is the empirical case for this presentation: PubPeer, where commentators come in herds and comments flourish. The only explanation given for this peculiar success is the possibility, largely used, to publish anoymized comments on the platform.

So, how can you embrace the openness of discussion and, at the same time, enable anonymous commentators? What kind of PPPR practices is it connected with? Does it inform our views on traditional peer review and how?

To answer these questions, we will first describe how the platform has been built and works, then to what kind of dynamics it leads as far as anonymity is concerned, then typify the arguments used for and against anonymity in PPPR, discuss its effects on published papers, before concluding on the way debates could be organized in PPPR.

These first results are based on the systematic qualitative analysis of both threads on PubPeer, articles on specialized websites on PubPeer and anonymity (Scholarly Kitchen, RetractionWatch…) and on editorials from scientific journals that have commented on anonymity in PPPR.

URL : Pubpeer: vigilante science, journal club or alarm raiser? The controversies over anonymity in post-publication peer review

Alternative location : https://halshs.archives-ouvertes.fr/halshs-01700198