Beyond Fact Checking: Reconsidering the Status of Truth of Published Articles

Authors : David Pontille, Didier Torny

Since the 17th century, scientific knowledge has been produced through a collective process, involving specific technologies used to perform experiments, to regulate modalities for participation of peers or lay people, and to ensure validation of the facts and publication of major results.

In such a world guided by the quest for a new kind of truth against previous beliefs various forms of misconduct – from subtle plagiarism to the entire fabrication of data and results – have largely been considered as minimal, if not inexistent.

Yet, some “betrayers of the truth” have been alleged in many fraudulent cases at least from the 1970s onward and the phenomenon is currently a growing concern in many academic corners. Facing numerous alerts, journals have generalized dedicated editorial formats to notify their readers of the emerging doubts affecting articles they had published.

This short piece is exclusively focused on these formats, which consists in “flagging” some articles to mark their problematic status.The visibility given to these flags and policies undermine the very basic components of the economy of science: How long can we collectively pretend that peer-reviewed knowledge should be the anchor to face a “post-truth” world?

URL : https://halshs.archives-ouvertes.fr/halshs-01576348

Les classements à l’international des revues en SHS

Auteurs/Authors : David Pontille, Didier Torny

Bien que plusieurs classements de revues aient été élaborés dès les années 1970, le caractère inédit de ceux qui ont émergé au cours des années 2000 réside dans leur statut d’instrument de politique publique. C’est le cas de l’Australie, du Brésil, de la France, de la Flandre, de la Norvège, et des Pays-Bas où cette technologie d’évaluation est en vigueur pour certains domaines – notamment en sciences humaines et sociales (SHS).

Dans cet article, nous analysons les modes d’existence de cette technologie d’évaluation spécifique. Bien que la formule générique du « classement de revues » se propage au plan international , différentes versions se développent parallèlement : leurs modalités de production, les valeurs défendues par leurs promoteurs et leurs usagers, aussi bien que leurs formes concrètes sont extrêmement variées.

Nous montrons que l’espace de variations des classements de revues en SHS est toujours bordé par deux options : favoriser une « bonne recherche » qui, sous l’effet d’avantages cumulatifs, risque de conduire à une science (hyper)normale soutenant des dispositions de conformité sociale chez les chercheurs ; encourager l’émergence des communautés minoritaires (linguistiques, disciplinaires, interdisciplinaires) et promouvoir la diversité des méthodes, théories et objets, au risque de mener à des formes de relativisme ou d’archipelisation de la recherche.

URL : https://hal-mines-paristech.archives-ouvertes.fr/hal-01256027

Pubpeer: vigilante science, journal club or alarm raiser? The controversies over anonymity in post-publication peer review

Author : Didier Torny

The more journal peer review (JPR) became a scientific topic, the more it has been the subject of criticisms and controversies. Repeated fake reports, confirmed reviewers’ biases, lack of reproducibility, and a recurrent inability to detect fraud and misconduct have apparently condemned JPR in its supposedly traditional form.

In fact, just like previous historical reforms and inventions, JPR has again been the object of many proposals to “fix it” since the beginning of the 21st century. Though these proposals are very diverse, two main directions have been identified: open peer review on one side, post-publication peer review (PPPR) on the other.

These two “fixes” share a common device, the open commenting of published articles, which is both an open peer review practice as it is visible to all readers and PPPR as it comes after the publication and often the certification of articles. At their intersection, it should thus thrive and indeed many journals have proposed this feature, but with no success.

Nevertheless, there is an exception to the disappointment with open commentary in PPPR, which is the empirical case for this presentation: PubPeer, where commentators come in herds and comments flourish. The only explanation given for this peculiar success is the possibility, largely used, to publish anoymized comments on the platform.

So, how can you embrace the openness of discussion and, at the same time, enable anonymous commentators? What kind of PPPR practices is it connected with? Does it inform our views on traditional peer review and how?

To answer these questions, we will first describe how the platform has been built and works, then to what kind of dynamics it leads as far as anonymity is concerned, then typify the arguments used for and against anonymity in PPPR, discuss its effects on published papers, before concluding on the way debates could be organized in PPPR.

These first results are based on the systematic qualitative analysis of both threads on PubPeer, articles on specialized websites on PubPeer and anonymity (Scholarly Kitchen, RetractionWatch…) and on editorials from scientific journals that have commented on anonymity in PPPR.

URL : Pubpeer: vigilante science, journal club or alarm raiser? The controversies over anonymity in post-publication peer review

Alternative location : https://halshs.archives-ouvertes.fr/halshs-01700198