Catégorie : EN

Publié le

Peer reviewers’ dilemmas: a qualitative exploration of decisional conflict in the evaluation of grant applications in the medical humanities and social sciences

Authors : Gaëlle Vallée-Tourangeau, Ana Wheelock, Tushna Vandrevala, Priscilla Harris

Independent evaluations of grant applications by subject experts are an important part of the peer-review system. However, little is known about the real-time experiences of peer reviewers or experts who perform reviews of a grant application independently.

This study sought to gain insight into this stage of the grant evaluation process by observing how experts conduct an independent review in near real time. Using the think aloud approach and Critical Decision Method of interviewing, in-depth interviews were conducted with 16 peer reviewers from a range of roles and disciplines within the medical humanities and social sciences.

Participants were asked to think aloud while reviewing applications to different grant schemes from a single prestigious funder. The analysis shows reviewers encountered five dilemmas during the evaluation process.

These dilemmas were related to whether or not one should (1) accept an invitation to review, (2) rely exclusively on the information presented in the application, (3) pay attention to institutional prestige, (4) offer comments about aspects that are not directly related to academics’ area of expertise, and (5) to take risks and overlook shortcomings rather than err on the side of caution.

In order to decide on the appropriate course of action, reviewers often engaged in a series of deliberations and trade-offs—varying in length and complexity.

However, their interpretation of what was ‘right’ was influenced by their values, preferences and experiences, but also by relevant norms and their understanding of the funder’s guidelines and priorities.

As a result, the way reviewers approached the identified dilemmas was idiosyncratic and sometimes diametrically opposed to other reviewers’ views, which could lead to variation in peer-review outcomes.

The dilemmas we have uncovered suggest that peer reviewers engage in thoughtful considerations during the peer-review process.

We should, therefore, be wary of reducing the absence of consensus as resulting from biased, instinctive thinking. Rather, these findings highlight the diversity of values, priorities and habits and ways of working each reviewer brings to the fore when reviewing the applicants and their project proposals and call for further reflection on, and study of, this “invisible work” to better understand and continue to improve the peer-reviewing process.

URL : Peer reviewers’ dilemmas: a qualitative exploration of decisional conflict in the evaluation of grant applications in the medical humanities and social sciences

DOI : https://doi.org/10.1057/s41599-022-01050-6

Publié le

Scientific research in news media: a case study of misrepresentation, sensationalism and harmful recommendations

Authors : Georgia Dempster, Georgina Sutherland, Louise Keogh

Accurate news media reporting of scientific research is important as most people receive their health information from the media and inaccuracies in media reporting can have adverse health outcomes.

We completed a quantitative and qualitative analysis of a journal article, the corresponding press release and the online news reporting of a scientific study.

Four themes were identified in the press release that were directly translated to the news reports that contributed to inaccuracies: sensationalism, misrepresentation, clinical recommendations and subjectivity.

The pressures on journalists, scientists and their institutions has led to a mutually beneficial relationship between these actors that can prioritise newsworthiness ahead of scientific integrity to the detriment of public health.

URL : Scientific research in news media: a case study of misrepresentation, sensationalism and harmful recommendations

DOI : https://doi.org/10.22323/2.21010206

Publié le

Editors publishing in their own journals: A systematic review of prevalence and a discussion of normative aspects

Authors : Gert Helgesson, Igor Radun, Jenni Radun, Gustav Nilsonne

Journal editors are the main gatekeepers in scientific publishing. Yet there is a concern that they may receive preferential treatment when submitting manuscripts to their own journals. The prevalence of such self-publishing is not known, nor the consequences for reliability and trustworthiness of published research.

This study aimed to systematically review the literature on the prevalence of editors publishing in their own journals and to conduct a normative ethical analysis of this practice. A systematic review was performed using the following databases: Medline, PsycInfo, Scopus and Web of Science.

Articles that provided primary data about editors publishing in own journals were included. We identified 15 studies meeting inclusion criteria. There was large variability of self-publishing across fields, journals and editors, ranging from those who never published in their own journal to those publishing extensively in their own journal.

Many studies suffered from serious methodological limitations. Nevertheless, our results show that there are settings where levels of self-publication are very high. We recommend that editors-in-chief and associate editors who have considerable power in journals refrain from publishing research articles in their own journals. Journals should have clear processes in place about the treatment of articles submitted by editorial board members.

URL : Editors publishing in their own journals: A systematic review of prevalence and a discussion of normative aspects

DOI : https://doi.org/10.1002/leap.1449

Publié le

Predatory journals and publishers: Characteristics and impact of academic spam to researchers in educational sciences

Authors : Jaume Sureda-Negre, Aina Calvo-Sastre, Rubén Comas-Forgas

This study focuses on the phenomenon of presumed predatory scientific publications in the field of Educational Sciences, and the utilization of email by editors to request manuscripts. It examined, using content analysis methods, 210 emails received by three professors of the field of Education, at a Spanish university with different research profiles over a period of 3 months.

Through analysis of the unsolicited emails a total of 139 journals and 37 publishers were identified and examined using: (a) the two main predatory journal inventories (Beall’s list and Cabells’ Predatory Reports), and (b) six of the major scientific bibliographic databases. The publishers and their websites were also analyzed, as well as the basic aspects of the emails’ content.

The majority of the unsolicited emails were from predatory journals or publishers and half of the article requests did not match the field of the recipient. In addition, it is relevant to note that more than half of the domains of predatory publishers analysed have untrustworthy security levels.

The data provided relevant information on the phenomenon of predation in scientific publications in the field of Education and, most importantly, provided evidence for developing training and preventive strategies to tackle it.

URL : Predatory journals and publishers: Characteristics and impact of academic spam to researchers in educational sciences

DOI : https://doi.org/10.1002/leap.1450

Publié le

Extraction and validation of scientific data for the identification of publications in open access journals

Authors : Patrícia Mascarenhas Dias, Thiago Magela Rodrigues Dias, Gray Farias Moita, Adilson Luiz Pinto

This work aims to identify articles published in open access journals registered in the Lattes Platform curricula. Currently, the curricular data from the Lattes Platform has been the source of several studies that adopt bibliometric metrics to understand scientific evolution in Brazil. However, when registering a publication in a curriculum, only basic information from the magazine is informed.

Therefore, to quantify the publications made in open access journals, a strategy is proposed that uses data from the Directory of Open Access Journals, validating the publications and, thus, obtaining a process that allows identifying which publications were made in this communication format.

Therefore, using the proposed methodology, which has a low computational cost, it was possible to quantify in an unprecedented way the publications of Brazilians in open access journals registered in the curricula of the Lattes Platform.

In addition, having the authors’ data of the publications, it was possible to understand how these authors are geographically distributed in the country, what their areas of operation are and how they have published the results of their research in open access.

It is noteworthy that they are generally published in Portuguese and English. Over the last few years, the representativeness of publications in open access journals has increased.

URL : Extraction and validation of scientific data for the identification of publications in open access journals

DOI : https://doi.org/10.1590/2318-0889202234e210040

Publié le

What Are the Characteristics of Canadian Independent, Scholarly Journals? Results from a Website Analysis

Authors : Jessica Lange, Sarah Severson

The dominance of commercial publishers (Larivière, Haustein, and Mongeon 2015) has led to a discussion in Canada focusing on alternative models for supporting independent, non-commercial, scholarly journals.

Although small in number, these journals represent an important contribution to Canadian and global scholarship. They also act as a counterbalance to the increasingly for-profit nature of scholarly publishing. Despite their importance, there exists no definitive list of journals of this nature in Canada, making analysis and understanding of their characteristics difficult.

In order to address this gap, the researchers undertook an analysis of the websites of 485 Canadian, independent, scholarly journals. Independent was defined as journals which are not affiliated with a commercial publisher.

The researchers gathered data for each journal on their access type (e.g., closed, open access), subject area, size and composition of the editorial team, and any affiliation(s). This data was then analyzed to create a portrait of these journals with these themes.

The researchers found that most of these journals were affiliated with at least one organization, with over half being associated with two or more. They also discovered that affiliations varied depending on the discipline and that the size of the editorial team was correlated to the access type.

Journals were predominantly in the humanities and social sciences, and the majority were open access (OA) without article processing charges (APCs).

While the focus of this study is on Canadian journals, this article provides a framework for other researchers to examine non-commercial, independent publishing in their own countries.

Its results also provide preliminary data which may inspire future avenues of research, particularly into models for non-APC, open access journals as well as the editorial board structure and size for independent journals.

DOI : https://doi.org/10.3998/jep.153

Publié le

Reshaping How Universities Can Evaluate the Research Impact of Open Humanities for Societal Benefit

Authors : Paul Longley Arthur, Lydia Hearn

During the twenty-first century, for the first time, the volume of digital data has surpassed the amount of analog data. As academic practices increasingly become digital, opportunities arise to reshape the future of scholarly communication through more accessible, interactive, open, and transparent methods that engage a far broader and more diverse public.

Yet despite these advances, the research performance of universities and public research institutes remains largely evaluated through publication and citation analysis rather than by public engagement and societal impact.

This article reviews how changes to bibliometric evaluations toward greater use of altmetrics, including social media mentions, could enhance uptake of open scholarship in the humanities.

In addition, the article highlights current challenges faced by the open scholarship movement, given the complexity of the humanities in terms of its sources and outputs that include monographs, book chapters, and journals in languages other than English; the use of popular media not considered as scholarly papers; the lack of time and energy to develop digital skills among research staff; problems of authority and trust regarding the scholarly or non-academic nature of social media platforms; the prestige of large academic publishing houses; and limited awareness of and familiarity with advanced digital applications.

While peer review will continue to be a primary method for evaluating research in the humanities, a combination of altmetrics and other assessment of research impact through different data sources may provide a way forward to ensure the increased use, sustainability, and effectiveness of open scholarship in the humanities.

DOI : https://doi.org/10.3998/jep.788