How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries

Authors : Annette Boaz, Robert Borst, Maarten Kok, Alison O’Shea

There is a growing recognition that needs more to be done to ensure that research contributes to better health services and patient outcomes. Stakeholder engagement in research, including co-production, has been identified as a promising mechanism for improving the value, relevance and utilization of research.

This article presents findings from a prospective study which explored the impact of stakeholder engagement in a 3-year European tobacco control research project. That research project aimed to engage stakeholders in the development, testing and dissemination of a return-on-investment tool across five EU countries (the Netherlands, Spain, Hungary, Germany and the UK).

The prospective study comprised interviews, observations and document review. The analysis focused on the extent to which the project team recognized, conceptualized and operationalized stakeholder engagement over the course of the research project. Stakeholder engagement in the European research project was conceptualized as a key feature of pre-designated spaces within their work programme.

Over the course of the project, however, the tool development work and stakeholder engagement activities decoupled. While the modelling and tool development became more secluded, stakeholder engagement activities subtly transformed from co-production, to consultation, to something more recognizable as research participation.

The contribution of this article is not to argue against the potential contribution of stakeholder engagement and co-production, but to show how even well-planned engagement activities can be diverted within the existing research funding and research production systems where non-research stakeholders remain at the margins and can even be seen as a threat to academic identify and autonomy.

URL : How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries

DOI : https://doi.org/10.1093/reseval/rvab013

The impact of geographical bias when judging scientific studies

Authors : Marta Kowal, Piotr Sorokowski, Emanuel Kulczycki, Agnieszka Żelaźniewicz

The beauty of science lies within its core assumption that it seeks to identify the truth, and as such, the truth stands alone and does not depend on the person who proclaims it. However, people’s proclivity to succumb to various stereotypes is well known, and the scientific world may not be exceptionally immune to the tendency to judge a book by its cover.

An interesting example is geographical bias, which includes distorted judgments based on the geographical origin of, inter alia, the given work and not its actual quality or value. Here, we tested whether both laypersons (N = 1532) and scientists (N = 480) are prone to geographical bias when rating scientific projects in one of three scientific fields (i.e., biology, philosophy, or psychology).

We found that all participants favored more biological projects from the USA than China; in particular, expert biologists were more willing to grant further funding to Americans. In philosophy, however, laypersons rated Chinese projects as better than projects from the USA. Our findings indicate that geographical biases affect public perception of research and influence the results of grant competitions.

URL : The impact of geographical bias when judging scientific studies

DOI : https://doi.org/10.1007/s11192-021-04176-7

Industry Collaborations of Research Teams: Are They Penalized or Rewarded in the Grant Evaluation Process?

Authors : Sıla Öcalan-Özel, Patrick Llerena

This paper explores the relationship between the industry collaborations of grant applicant teams and the outcomes of a multistage grant evaluation process.

We studied this relationship by focusing on two possible channels of impact of industry engagement—team diversity (or the diversity effect) and prior collaboration experience (or the experience effect)—and examined their influence on the evaluators’ decision by using the proxies of direct industry engagement (i.e., the involvement of a company-affiliated researcher in the grant applicant team) and indirect industry engagement (i.e., joint publications with a company-affiliated researcher prior to the grant application), respectively.

We analyzed data extracted from the application and reviewed materials of a multidisciplinary, pan-European research funding scheme—European Collaborative Research (EUROCORES)—for the period 2002–2010 and conducted an empirical investigation of its three consecutive grant evaluation stages at the team level.

We found that teams presenting an indirect engagement were more likely to pass the first stage of selection, whereas no significant relationships were found at any of the three evaluation stages for teams presenting a direct engagement.

Our findings point to the heterogeneity of the decision-making process within a multistage grant evaluation scheme and suggest that the policy objective of fostering university–industry collaboration does not significantly impact the funding process.

URL: Industry Collaborations of Research Teams: Are They Penalized or Rewarded in the Grant Evaluation Process?

DOI : https://doi.org/10.3389/frma.2021.707278

Science rules! A qualitative study of scientists’ approaches to grant lottery

Author : Axel Philipps

Using peer review to assess the validity of research proposals has always had its fair share of critics, including a more-than-fair-share of scholars. The debate about this method of assessing these proposals now seems trivial when compared with assessing the validity for granting funding by lottery.

Some of the same scholars have suggested that the way grant lottery was being assessed has made random allocation seem even-handed, less biased and more supportive of innovative research.

But we know little of what researchers actually think about grant lottery and even less about the thoughts of those scientists who rely on funding. This paper examines scientists’ perspectives on selecting grants by ‘lots’ and how they justify their support or opposition.

How do they approach something scientifically that is, in itself, not scientific? These approaches were investigated with problem-centered interviews conducted with natural scientists in Germany.

The qualitative interviews for this paper reveal that scientists in dominated and dominating field positions are, more or less, open to the idea of giving a selection process by lots a try. Nonetheless, they are against pure randomization because from their point of view it is incompatible with scientific principles.

They rather favor a combination of grant lottery and peer review processes, assuming that only under these conditions could randomly allocated funding be an integral and legitimate part of science.

URL : Science rules! A qualitative study of scientists’ approaches to grant lottery

DOI : https://doi.org/10.1093/reseval/rvaa027

The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants

Authors : Rachel Heyard, Hanna Hottenrott

This study investigates the effect of competitive project funding on researchers’ publication outputs. Using detailed information on applicants at the Swiss National Science Foundation and their proposal evaluations, we employ a case-control design that accounts for individual heterogeneity of researchers and selection into treatment (e.g. funding).

We estimate the impact of the grant award on a set of output indicators measuring the creation of new research results (the number of peer-reviewed articles), its relevance (number of citations and relative citation ratios), as well as its accessibility and dissemination as measured by the publication of preprints and by altmetrics.

The results show that the funding program facilitates the publication and dissemination of additional research amounting to about one additional article in each of the three years following the funding.

The higher citation metrics and altmetrics by funded researchers suggest that impact goes beyond quantity and that funding fosters dissemination and quality.

URL : The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants

DOI : https://doi.org/10.1057/s41599-021-00891-x

Contest models highlight inherent inefficiencies of scientific funding competitions

Authors : Kevin Gross, Carl T. Bergstrom

Scientific research funding is allocated largely through a system of soliciting and ranking competitive grant proposals. In these competitions, the proposals themselves are not the deliverables that the funder seeks, but instead are used by the funder to screen for the most promising research ideas.

Consequently, some of the funding program’s impact on science is squandered because applying researchers must spend time writing proposals instead of doing science. To what extent does the community’s aggregate investment in proposal preparation negate the scientific impact of the funding program?

Are there alternative mechanisms for awarding funds that advance science more efficiently? We use the economic theory of contests to analyze how efficiently grant proposal competitions advance science, and compare them with recently proposed, partially randomized alternatives such as lotteries.

We find that the effort researchers waste in writing proposals may be comparable to the total scientific value of the research that the funding supports, especially when only a few proposals can be funded.

Moreover, when professional pressures motivate investigators to seek funding for reasons that extend beyond the value of the proposed science (e.g., promotion, prestige), the entire program can actually hamper scientific progress when the number of awards is small.

We suggest that lost efficiency may be restored either by partial lotteries for funding or by funding researchers based on past scientific success instead of proposals for future work.

URL : Contest models highlight inherent inefficiencies of scientific funding competitions

DOI : https://doi.org/10.1371/journal.pbio.3000065

ARL Libraries and Research: Correlates of Grant Funding

Author : Ryan P. Womack

While providing the resources and tools that make advanced research possible is a primary mission of academic libraries at large research universities, many other elements also contribute to the success of the research enterprise, such as institutional funding, staffing, labs, and equipment.

This study focuses on U.S. members of the ARL, the Association for Research Libraries. Research success is measured by the total grant funding received by the University, creating an ordered set of categories.

Combining data from the NSF National Center for Science and Engineering Statistics, ARL Statistics, and IPEDS, the primary explanatory factors for research success are examined.

Using linear regression, logistic regression, and the cumulative logit model, the best-fitting models generated by ARL data, NSF data, and the combined data set for both nominal and per capita funding are compared. These models produce the most relevant explanatory variables for research funding, which do not include library-related variables in most cases.

URL : http://arxiv.org/abs/1601.05104