status

Longitudinal Trends in the Performance of Scientific Peer…

Posted on by Hans Dillaerts

Longitudinal Trends in the Performance of Scientific Peer Reviewers :

« Study objective : We characterize changes in review quality by individual peer reviewers over time.

Methods : Editors at a specialty journal in the top 11% of Institute of Scientific Information journals rated the quality of every review, using a validated 5-point quality score. Linear mixed-effect models were used to analyze rating changes over time, calculating within-reviewer trends plus predicted slope of change in score for each reviewer. Reviewers at this journal have been shown comparable to those at other journals.

Results : Reviews (14,808) were performed by 1,499 reviewers and rated by 84 editors during the 14-year study. Ninety-two percent of reviewers demonstrated very slow but steady deterioration in their scores (mean –0.04 points [–0.8%] per year). Rate of deterioration was unrelated to duration of reviewing but moderately correlated with mean reviewer quality score (R=0.52). The mean score of each reviewer’s first 4 reviews predicted subsequent performance with a sensitivity of 75% and specificity of 47%. Scores of the group stayed constant over time despite deterioration because newly recruited reviewers initially had higher mean quality scores than their predecessors.

Conclusion : This study, one of few tracking expert performance longitudinally, demonstrates that most journal peer reviewers received lower quality scores for article assessment over the years. This could be due to deteriorating performance (caused by either cognitive changes or competing priorities) or, to a partial degree, escalating expectations; other explanations were ruled out. This makes monitoring reviewer quality even more crucial to maintain the mission of scientific journals. »

URL : http://www.annemergmed.com/article/S0196-0644(10)01266-7/fulltext

status

Toward a new model of scientific publishing discussion…

Posted on by Hans Dillaerts

Toward a new model of scientific publishing: discussion and a proposal :

« The current system of publishing in the biological sciences is notable for its redundancy, inconsistency, sluggishness, and opacity. These problems persist, and grow worse, because the peer review system remains focused on deciding whether or not to publish a paper in a particular journal rather than providing (1) a high-quality evaluation of scientific merit and (2) the information necessary to organize and prioritize the literature. Online access has eliminated the need for journals as distribution channels, so their primary current role is to provide authors with feedback prior to publication and a quick way for other researchers to prioritize the literature based on which journal publishes a paper. However, the feedback provided by reviewers is not focused on scientific merit but on whether to publish in a particular journal, which is generally of little use to authors and an opaque and noisy basis for prioritizing the literature. Further, each submission of a rejected manuscript requires the entire machinery of peer review to creak to life anew. This redundancy incurs delays, inconsistency, and increased burdens on authors, reviewers, and editors. Finally, reviewers have no real incentive to review well or quickly, as their performance is not tracked, let alone rewarded. One of the consistent suggestions for modifying the current peer review system is the introduction of some form of post-publication reception, and the development of a marketplace where the priority of a paper rises and falls based on its reception from the field (see other articles in this special topics). However, the information that accompanies a paper into the marketplace is as important as the marketplace’s mechanics. Beyond suggestions concerning the mechanisms of reception, we propose an update to the system of publishing in which publication is guaranteed, but pre-publication peer review still occurs, giving the authors the opportunity to revise their work following a mini pre-reception from the field. This step also provides a consistent set of rankings and reviews to the marketplace, allowing for early prioritization and stabilizing its early dynamics. We further propose to improve the general quality of reviewing by providing tangible rewards to those who do it well. »
URL : http://www.frontiersin.org/computational_neuroscience/10.3389/fncom.2011.00055/full

status

Sharing and the Creative Economy Culture in the…

Posted on by Hans Dillaerts

Sharing and the Creative Economy: Culture in the Internet Age :

« In the past fifteen years, file sharing of digital cultural works between individuals has been at the center of a number of debates on the future of culture itself. To some, sharing constitutes piracy, to be fought against and eradicated. Others see it as unavoidable, and table proposals to compensate for its harmful effects. Meanwhile, little progress has been made towards addressing the real challenges facing culture in a digital world. Sharing starts from a radically different viewpoint, namely that the non-market sharing of digital works is both legitimate and useful. It supports this premise with empirical research, demonstrating that non-market sharing leads to more diversity in the attention given to various works. Taking stock of what we have learnt about the cultural economy in recent years, Sharing sets out the conditions necessary for valuable cultural functions to remain sustainable in this context. »
URL : http://paigrain.debatpublic.net/docs/internet_creation_1-3.pdf
URL : http://www.amazon.com/Sharing-Culture-Economy-Internet-Age/dp/9089643850

status

Mathematicians’ Views on Current Publishing Issues A Survey…

Posted on by Hans Dillaerts

Mathematicians’ Views on Current Publishing Issues: A Survey of Researchers :

This article reports research mathematicians’ attitudes about and activity in specific scholarly communication areas, as captured in a 2010 survey of more than 600 randomly-selected mathematicians worldwide. Key findings include:

  • Most mathematicians have papers in the arXiv, but posting to their own web pages remains more common;
  • A third of mathematicians have published papers in open access (OA) journals, with speed of publication being seen as the primary advantage over traditional journals, but there is substantial philosophical opposition to OA journal models that charge author fees;
  • Tenure and promotion criteria influence publishing decisions even among most tenured faculty members;
  • Mathematicians want to keep more rights to their publications than they have been allowed, but they have a high success rate in negotiating with publishers for more;
  • Online collaboration tools, such as Google Groups, are not yet widely used for research but their use is expected to rise in the near future.

Reasons behind the mathematics culture of openness were also explored. »

URL : http://www.istl.org/11-fall/refereed4.html

EN

Citations to Wikipedia in Chemistry Journals: A Preliminary Study

Posted on by Hans Dillaerts

Wikipedia has been the subject of an increasing number of studies. Many of these have focused on the quality of Wikipedia articles and the use of Wikipedia by students. Little research has focused on the use of Wikipedia by scholars. This study helps to fill that gap by examining citations to Wikipedia in chemistry journals from three major publishers over a five year period.

The study reports the number of citations to Wikipedia and describes how Wikipedia is being cited. The results show that, while only a small percentage of all articles contained a citation to Wikipedia, it is in fact being cited as a credible information source in articles in major chemistry journals.

URL : http://www.istl.org/11-fall/refereed2.html

status

National Open Access and Preservation Policies in Europe…

Posted on by Hans Dillaerts

National Open Access and Preservation Policies in Europe :

« The present report is the analysis of the answers to the questionnaire that the European Commission prepared on open access and preservation policies in Europe, with a view to taking stock in 2011 of the status of implementation of the 2007 Council conclusions on scientific information in the digital age. »

URL : http://ec.europa.eu/research/science-society/document_library/pdf_06/open-access-report-2011_en.pdf