Archives des mots-clés : transparency

Reproducible research practices and transparency across linguistics

Posté le par

Authors : Agata Bochynska, Liam Keeble, Caitlin Halfacre, Joseph V. Casillas, Irys-Amélie Champagne, Kaidi Chen, Melanie Röthlisberger, Erin M. Buchanan, Timo B. Roettger

Scientific studies of language span across many disciplines and provide evidence for social,  cultural, cognitive, technological, and biomedical studies of human nature and behavior. As it becomes increasingly empirical and quantitative, linguistics has been facing challenges and limitations of the scientific practices that pose barriers to reproducibility and replicability.

One of the  proposed solutions to the widely acknowledged reproducibility and replicability crisis has been the implementation of transparency practices,  e.g., open access publishing, preregistrations, sharing study materials, data, and analyses, performing study replications, and declaring conflicts of interest.

Here, we have assessed the prevalence of these practices in 600 randomly sampled journal articles from linguistics across two time points. In line with similar studies in other disciplines, we found that 35% of the articles were published open access and the rates of sharing materials, data, and protocols were below 10%. None of the articles reported preregistrations, 1% reported replications, and 10% had conflict of interest statements.

These rates have not increased noticeably between 2008/2009 and 2018/2019, pointing to remaining barriers and the slow adoption of open and reproducible research practices in linguistics.

To facilitate adoption of these practices, we provide a range of recommendations and solutions for implementing transparency and improving reproducibility of research in linguistics.

URL : Reproducible research practices and transparency across linguistics

DOI : https://doi.org/10.5070/G6011239

Transparency in conducting and reporting research: A survey of authors, reviewers, and editors across scholarly disciplines

Posté le par

Authors : Mario Malički, IJsbrand Jan Aalbersberg, Lex Bouter, Adrian Mulligan, Gerben ter Riet

Calls have been made for improving transparency in conducting and reporting research, improving work climates, and preventing detrimental research practices. To assess attitudes and practices regarding these topics, we sent a survey to authors, reviewers, and editors. We received 3,659 (4.9%) responses out of 74,749 delivered emails.

We found no significant differences between authors’, reviewers’, and editors’ attitudes towards transparency in conducting and reporting research, or towards their perceptions of work climates. Undeserved authorship was perceived by all groups as the most prevalent detrimental research practice, while fabrication, falsification, plagiarism, and not citing prior relevant research, were seen as more prevalent by editors than authors or reviewers.

Overall, 20% of respondents admitted sacrificing the quality of their publications for quantity, and 14% reported that funders interfered in their study design or reporting. While survey respondents came from 126 different countries, due to the survey’s overall low response rate our results might not necessarily be generalizable.

Nevertheless, results indicate that greater involvement of all stakeholders is needed to align actual practices with current recommendations.

URL : Transparency in conducting and reporting research: A survey of authors, reviewers, and editors across scholarly disciplines

DOI : https://doi.org/10.1371/journal.pone.0270054

Exploring enablers and barriers to implementing the Transparency and Openness Promotion Guidelines: a theory-based survey of journal editors

Posté le par

Authors : Kevin Naaman, Sean Grant, Sina Kianersi, Lauren Supplee, Beate Henschel, Evan Mayo-Wilson

The Transparency and Openness Promotion (TOP) Guidelines provide a framework to help journals develop open science policies. Theories of behaviour change can guide understanding of why journals do (not) implement open science policies and the development of interventions to improve these policies.

In this study, we used the Theoretical Domains Framework to survey 88 journal editors on their capability, opportunity and motivation to implement TOP. Likert-scale questions assessed editor support for TOP, and enablers and barriers to implementing TOP.

A qualitative question asked editors to provide reflections on their ratings. Most participating editors supported adopting TOP at their journal (71%) and perceived other editors in their discipline to support adopting TOP (57%). Most editors (93%) agreed their roles include maintaining policies that reflect current best practices.

However, most editors (74%) did not see implementing TOP as a high priority compared with other editorial responsibilities. Qualitative responses expressed structural barriers to implementing TOP (e.g. lack of time, resources and authority to implement changes) and varying support for TOP depending on study type, open science standard, and level of implementation.

We discuss how these findings could inform the development of theoretically guided interventions to increase open science policies, procedures and practices.

URL : Exploring enablers and barriers to implementing the Transparency and Openness Promotion Guidelines: a theory-based survey of journal editors

DOI : https://doi.org/10.1098/rsos.221093

Champions of Transparency in Education: What Journal Reviewers Can Do to Encourage Open Science Practices

Posté le par

Authors : Rachel Renbarger, Jill L. Adelson, Joshua Rosenberg, Sondra M Stegenga, Olivia Lowrey, Pamela Rose Buckley, Qiyang Zhang

As the field of education and especially gifted education gradually moves towards open science, our research community increasingly values transparency and openness brought by open science practices.

Yet, individual researchers may be reluctant to adopt open science practices due to low incentives, barriers of extra workload, or lack of support to apply these in certain areas, such as qualitative research.

We encourage and give guidelines to reviewers to champion open science practices by warmly influencing authors to consider applying open science practices to quantitative, qualitative, and mixed methods research and providing ample support to produce higher-quality publications.

Instead of imposing open science practices on authors, we advocate reviewers suggest small, non-threatening, specific steps to support authors without making them feel overwhelmed, judged, or punished.

We believe that these small steps taken by reviewers will make a difference to create a more supportive environment for researchers to adopt better practices.

DOI : https://doi.org/10.35542/osf.io/xqfwb

Adoption of Transparency and Openness Promotion (TOP) Guidelines across Journals

Posté le par

Authors : Inga Patarčić, Jadranka Stojanovski

Journal policies continuously evolve to enable knowledge sharing and support reproducible science. However, that change happens within a certain framework. Eight modular standards with three levels of increasing stringency make Transparency and Openness Promotion (TOP) guidelines which can be used to evaluate to what extent and with which stringency journals promote open science.

Guidelines define standards for data citation, transparency of data, material, code and design and analysis, replication, plan and study pre-registration, and two effective interventions: “Registered reports” and “Open science badges”, and levels of adoption summed up across standards define journal’s TOP Factor. In this paper, we analysed the status of adoption of TOP guidelines across two thousand journals reported in the TOP Factor metrics.

We show that the majority of the journals’ policies align with at least one of the TOP’s standards, most likely “Data citation” (70%) followed by “Data transparency” (19%). Two-thirds of adoptions of TOP standard are of the stringency Level 1 (less stringent), whereas only 9% is of the stringency Level 3.

Adoption of TOP standards differs across science disciplines and multidisciplinary journals (N = 1505) and journals from social sciences (N = 1077) show the greatest number of adoptions. Improvement of the measures that journals take to implement open science practices could be done: (1) discipline-specific, (2) journals that have not yet adopted TOP guidelines could do so, (3) the stringency of adoptions could be increased.

URL : Adoption of Transparency and Openness Promotion (TOP) Guidelines across Journals

DOI : https://doi.org/10.3390/publications10040046

Transparency versus anonymity: which is better to eliminate bias in peer review?

Posté le par

Authors: Faye Holst, Kim Eggleton, Simon Harris

Peer review is a critical component of the scientific process. When conducted properly by dedicated and competent reviewers, it helps to safeguard the quality, validity, authority and rigour of academic work. However, bias in peer review is well documented and can skew objectivity of the review and hinder fair assessment of research.

To mitigate against bias and enhance accountability, IOP Publishing has introduced two different, but complementary, approaches to all their peer-reviewed, open access (OA) journals: double-anonymous peer review and transparent peer review.

Double-anonymous peer review, where the reviewer and author identities are concealed, is designed to tackle inequality in the scholarly publishing process as it reduces bias with respect to gender, race, country of origin or affiliation.

Transparent peer review shows readers the full peer review history, including reviewer reports, editor decision letters and the authors’ responses alongside the published article. Making this process visible to the community increases accountability, allows reviewers to be recognized more for their work and can aid the training of aspiring reviewers.

IOP Publishing is the first physics publisher to adopt both of these approaches portfolio wide. In this article we discuss how applying these methods has altered different elements of the publishing process. Early indicators show that there may be a marked difference in acceptance rates across regions.

URL : Transparency versus anonymity: which is better to eliminate bias in peer review?

DOI : http://doi.org/10.1629/uksg.584

Improving evidence-based practice through preregistration of applied research: Barriers and recommendations

Posté le par

Authors : Thomas Rhys Evans, Peter Branney, Andrew Clements, Ella Hatton

Preregistration is the practice of publicly publishing plans on central components of the research process before access to, or collection, of data. Within the context of the replication crisis, open science practices like preregistration have been pivotal in facilitating greater transparency in research.

However, such practices have been applied nearly exclusively to basic academic research, with rare consideration of the relevance to applied and consultancy-based research. This is particularly problematic as such research is typically reported with very low levels of transparency and accountability despite being disseminated as influential gray literature to inform practice.

Evidence-based practice is best served by an appreciation of multiple sources of quality evidence, thus the current review considers the potential of preregistration to improve both the accessibility and credibility of applied research toward more rigorous evidence-based practice.

The current three-part review outlines, first, the opportunities of preregistration for applied research, and second, three barriers – practical challenges, stakeholder roles, and the suitability of preregistration.

Last, this review makes four recommendations to overcome these barriers and maximize the opportunities of preregistration for academics, industry, and the structures they are held within – changes to preregistration templates, new types of templates, education and training, and recognition and structural changes.

URL : Improving evidence-based practice through preregistration of applied research: Barriers and recommendations

DOI : https://doi.org/10.1080/08989621.2021.1969233