Catégories
EN

A framework for assessing the trustworthiness of scientific research findings

Authors : Brian A. Nosek, David B. Allison, Kathleen Hall Jamieson, Marcia McNutt, A. Beau Nielsen, Susan M. Wolf

Vigorous debate has erupted over the trustworthiness of scientific research findings in a number of domains. The question “what makes research findings trustworthy?” elicits different answers depending on whether the emphasis is on research integrity and ethics, research methods, transparency, inclusion, assessment and peer review, or scholarly communication. Each provides partial insight.

We offer a systems approach that focuses on whether the research is accountable, evaluable, well-formulated, has been evaluated, controls for bias, reduces error, and whether the claims are warranted by the evidence. We tie each of these components to measurable indicators of trustworthiness for evaluating the research itself, the researchers conducting the research, and the organizations supporting the research.

Our goals are to offer a framework that can be applied across methods, approaches, and disciplines and to foster innovation in development of trustworthiness indicators. Developing valid indicators will improve the conduct and assessment of research and, ultimately, public understanding and trust.

URL : A framework for assessing the trustworthiness of scientific research findings

DOI : https://doi.org/10.1073/pnas.2536736123

Catégories
EN

A simple proposal for the publication of journal citation distributions

Authors : Vincent Larivière, Véronique Kiermer, Catriona J. MacCallum, Marcia McNutt, Mark Patterson, Bernd Pulverer, Sowmya Swaminathan, Stuart Taylor, Stephen Curry

Although the Journal Impact Factor (JIF) is widely acknowledged to be a poor indicator of the quality of individual papers, it is used routinely to evaluate research and researchers. Here, we present a simple method for generating the citation distributions that underlie JIFs.

Application of this straightforward protocol reveals the full extent of the skew of distributions and variation in citations received by published papers that is characteristic of all scientific journals.

Although there are differences among journals across the spectrum of JIFs, the citation distributions overlap extensively, demonstrating that the citation performance of individual papers cannot be inferred from the JIF.

We propose that this methodology be adopted by all journals as a move to greater transparency, one that should help to refocus attention on individual pieces of work and counter the inappropriate usage of JIFs during the process of research assessment.

URL : A simple proposal for the publication of journal citation distributions

Alternative location : http://www.biorxiv.org/content/early/2016/07/05/062109.abstract