EN

Attitudes toward open peer review among stakeholders of a scholar-led journal in Brazil

Posted on by Hans Dillaerts

Authors : Leonardo Ferreira Fontenelle, Thiago Dias Sarti

Scholarly journals should consider the attitudes of their communities before adopting any of the seven traits of open peer review. Unfortunately, surveys from the Global North might not apply to the Global South, where double-blind peer review is commonplace even among natural sciences and medicine journals.

This paper reports the findings of a survey on attitudes toward open peer review among four stakeholder groups of a scholar-led medical journal in Brazil: society members, journal readers, authors, and reviewers.

Compared to a previous survey, which mostly recruited natural sciences researchers from Europe, this survey found similar support for open peer review in general and for most of its traits.

One important exception was open identities, which were considered detrimental by most participants, even more in this survey than in the previous one. Interestingly, participants were more dismissive of open identities as a whole than of statements about its specific consequences.

Because preprints are increasingly popular but incompatible with double-blind review, future research should examine the effects of transitioning from double-blind to open identities, especially on gender bias.

Meanwhile, scholarly journals with double-blind review might prefer to begin by adopting other traits of open review or to make open identities optional at first.

URL : Attitudes toward open peer review among stakeholders of a scholar-led journal in Brazil

DOI : https://doi.org/10.1590/2318-0889202133e200072

EN

Will Academic Library Publishing Break OER? A Diffusion of Innovations Study

Posted on by Hans Dillaerts

Authors : Kathy Essmiller, Tutaleni Asino

Academic libraries are among the organizations advocating for open educational resources (OER), often playing a key campus role in education, advocacy, and support of their creation and publication. Publication of OER resonates with the role of the academic library.

Because “incongruence in perceptions” (Chtena 2019: 24) can cause difficulties and unforeseen challenges with implementation and use of OER, organizations involved in OER initiatives need familiarity with how OER and organizational values align.

The goal of this exploration was to investigate how academic libraries enact academic library publishing programs and the ramification that has in the diffusion process of OER in higher education. Data collected in this single case study research project was analyzed through the lens of Diffusion of Innovations Theory.

The findings from the study suggest that, if academic libraries are to enact the creation and publication of OER in ways appropriate to their conception, those involved will need to be intentional about ensuring enactment of the values foundational to OER.

Future suggested research includes a multiple-case study comparative research study looking at academic library publication of OER, exploration of how opinion leaders and attributes of innovations impact academic library publication of OER, and investigation into the impact of organizational structure on the diffusion of OER creation and publication.

URL : Will Academic Library Publishing Break OER? A Diffusion of Innovations Study

DOI : http://doi.org/10.5334/jime.673

EN

Journals in Beall’s list perform as a group less well than other open access journals indexed in Scopus but reveal large differences among publishers

Posted on by Hans Dillaerts

Authors : Henk F. Moed, Carmen Lopez-Illescas, Vicente P. Guerrero-Bote, Felix de Moya-Anegon

The list of potential, possible or probable predatory scholarly open access (OA) publishers compiled by Jeffrey Beall was examined to determine the effect of their inclusion upon authors, and a possible bias against OA journals.

Manually collected data from the publication archives of a sample of 250 journals from Beall publishers reveals a strong tendency towards a decline in their article output during 2012–2020. A comparison of the subset of 506 Beall journals indexed in Scopus with a benchmark set of other OA journals in Scopus with similar characteristics shows that Beall journals reveal as a group a strong decline in citation impact over the years, and reached an impact level far below that of their benchmarks.

The Beall list of publishers was found to be heterogeneous in terms of bibliometric indicators but to be clearly differentiated from OA journals not included in the list. The same bibliometric comparison against comparable non-OA journals reveal similar, but less marked, differences in citation and publication growth.

URL : Journals in Beall’s list perform as a group less well than other open access journals indexed in Scopus but reveal large differences among publishers

DOI : https://doi.org/10.1002/leap.1428

EN

Promoting a culture of openness: Institutional open access policy development and evaluation at a Canadian university

Posted on by Hans Dillaerts

Authors : Alison J. Moore, Jennifer Zerkee, Kate Shuttleworth, Rebecca Dowson, Gwen Bird

Institutional open access (OA) policies can act as a solid foundation on which to build university-wide support for open access. This is the first paper to reflect on the entire process of developing, implementing, and reviewing an institutional open access policy at a Canadian post-secondary institution.

Simon Fraser University (SFU) is one of a few Canadian universities with an institutional open access policy. As a leader in open access, SFU is well positioned to share observations of our experiences in the first three years of our OA policy.

Throughout this paper, we reflect on the role that the policy plays in the broader culture of openness at SFU and on the OA resources and supports provided to SFU researchers.

Other institutions may find our observations and adoption of the SOAR (strengths, opportunities, aspirations, results) appreciative inquiry framework useful as they explore future policy development or review and work to promote a culture of open access within their university community.

URL: Promoting a culture of openness: Institutional open access policy development and evaluation at a Canadian university

DOI : https://doi.org/10.21083/partnership.v16i1.6150

EN

Peer Review in Law Journals

Posted on by Hans Dillaerts

Authors : Jadranka Stojanovski, Elías Sanz-Casado, Tommaso Agnoloni, Ginevra Peruginelli

The field of law has retained its distinctiveness regarding peer review to this day, and reviews are often conducted without following standardized rules and principles. External and independent evaluation of submissions has recently become adopted by European law journals, and peer review procedures are still poorly defined, investigated, and attuned to the legal science publishing landscape.

The aim of our study was to gain a better insight into current editorial policies on peer review in law journals by exploring editorial documents (instructions, guidelines, policies) issued by 119 Croatian, Italian, and Spanish law journals.

We relied on automatic content analysis of 135 publicly available documents collected from the journal websites to analyze the basic features of the peer review processes, manuscript evaluation criteria, and related ethical issues using WordStat.

Differences in covered topics between the countries were compared using the chi-square test. Our findings reveal that most law journals have adopted a traditional approach, in which the editorial board manages mostly anonymized peer review (104, 77%) engaging independent/external reviewers (65, 48%).

Submissions are evaluated according to their originality and relevance (113, 84%), quality of writing and presentation (94, 70%), comprehensiveness of literature references (93, 69%), and adequacy of methods (57, 42%).

The main ethical issues related to peer review addressed by these journals are reviewer’s competing interests (42, 31%), plagiarism (35, 26%), and biases (30, 22%). We observed statistically significant differences between countries in mentioning key concepts such as “Peer review ethics”, “Reviewer”, “Transparency of identities”, “Publication type”, and “Research misconduct”.

Spanish journals favor reviewers’ “Independence” and “Competence” and “Anonymized” peer review process. Also, some manuscript types popular in one country are rarely mentioned in other countries.

Even though peer review is equally conventional in all three countries, high transparency in Croatian law journals, respect for research integrity in Spanish ones, and diversity and inclusion in Italian are promising indicators of future development.

URL : Peer Review in Law Journals

DOI : https://doi.org/10.3389/frma.2021.787768

EN

The production of scientific and societal value in research evaluation: a review of societal impact assessment methods

Posted on by Hans Dillaerts

Authors : Jorrit P Smit, Laurens K Hessels

Over the past two decades, several methods have been developed to evaluate the societal impact of research. Compared to the practical development of the field, the conceptual development is relatively weak.

This review article contributes to the latter by elucidating the theoretical aspects of the dominant methods for evaluating societal impact of research, in particular, their presuppositions about the relationship between scientific and societal value of research. We analyse 10 approaches to the assessment of the societal impact of research from a constructivist perspective.

The methods represent different understandings of knowledge exchange, which can be understood in terms of linear, cyclical, and co-production models. In addition, the evaluation methods use a variety of concepts for the societal value of research, which suggest different relationships with scientific value.

While some methods rely on a clear and explicit distinction between the two types of value, other methods, in particular Evaluative Inquiry, ASIRPA, Contribution Mapping, Public Value Mapping, and SIAMPI, consider the mechanisms for producing societal value integral to the research process.

We conclude that evaluation methods must balance between demarcating societal value as a separate performance indicator for practical purposes and doing justice to the (constructivist) science studies’ findings about the integration of scientific and societal value of research.

Our analytic comparison of assessment methods can assist research evaluators in the conscious and responsible selection of an approach that fits with the object under evaluation. As evaluation actively shapes knowledge production, it is important not to use oversimplified concepts of societal value.

URL : The production of scientific and societal value in research evaluation: a review of societal impact assessment methods

DOI : https://doi.org/10.1093/reseval/rvab002

EN

How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries

Posted on by Hans Dillaerts

Authors : Annette Boaz, Robert Borst, Maarten Kok, Alison O’Shea

There is a growing recognition that needs more to be done to ensure that research contributes to better health services and patient outcomes. Stakeholder engagement in research, including co-production, has been identified as a promising mechanism for improving the value, relevance and utilization of research.

This article presents findings from a prospective study which explored the impact of stakeholder engagement in a 3-year European tobacco control research project. That research project aimed to engage stakeholders in the development, testing and dissemination of a return-on-investment tool across five EU countries (the Netherlands, Spain, Hungary, Germany and the UK).

The prospective study comprised interviews, observations and document review. The analysis focused on the extent to which the project team recognized, conceptualized and operationalized stakeholder engagement over the course of the research project. Stakeholder engagement in the European research project was conceptualized as a key feature of pre-designated spaces within their work programme.

Over the course of the project, however, the tool development work and stakeholder engagement activities decoupled. While the modelling and tool development became more secluded, stakeholder engagement activities subtly transformed from co-production, to consultation, to something more recognizable as research participation.

The contribution of this article is not to argue against the potential contribution of stakeholder engagement and co-production, but to show how even well-planned engagement activities can be diverted within the existing research funding and research production systems where non-research stakeholders remain at the margins and can even be seen as a threat to academic identify and autonomy.

URL : How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries

DOI : https://doi.org/10.1093/reseval/rvab013