An introduction to achieving policy impact for early career researchers

Authors : Megan C Evans, Christopher Cvitanovic

Scientists are increasingly required to demonstrate the real world tangible impacts arising from their research. Despite significant advances in scholarship dedicated to understanding and improving the relationships between science, policy and practice, much of the existing literature remains high level, theoretical, and not immediately accessible to early career researchers (ECRs) who work outside of the policy sciences.

In this paper, we draw on the literature and our own experiences working in the environmental sciences to provide an accessible resource for ECRs seeking to achieve policy impact in their chosen field. First, we describe key concepts in public policy to provide sufficient background for the non-expert.

Next, we articulate a number of practical steps and tools that can help ECRs to identify and enhance the policy relevance of their research, better understand the policy world in practice and identify a range of pathways to achieving impact.

Finally, we draw on our personal experiences to highlight some of the key individual characteristics and values that are needed to operate more effectively at the interface of science, policy and practice.

Our hope is that the information and tools provided here can help to empower ECRs to create their own pathways to impact that best suit their individual goals, circumstances, interests and strengths.

URL : An introduction to achieving policy impact for early career researchers

Alternative location : https://www.nature.com/articles/s41599-018-0144-2

Case for the double-blind peer review

Author : Lucie Tvrznikova

Peer review is a process designed to produce a fair assessment of research quality prior to publication of scholarly work in a journal. Demographics, nepotism, and seniority have been all shown to affect reviewer behavior suggesting the most common, single-blind review method (or the less common open review method) might be biased.

A survey of current research suggests that double-blind review offers a solution to many biases stemming from author’s gender, seniority, or location without imposing any major downsides.

URL : https://arxiv.org/abs/1807.01408

Open Access in Vocational Education and Training Research

Authors : Karin Langenkamp, Bodo Rödel, Kerstin Taufenbach, Meike Weiland

The article presents a research project at the Federal Institute for Vocational Education and Training in Germany and reflects the perspective of researchers in the field of vocational education and training (VET).

It investigates the technical and structural, policy-related, and normative and inherent academic research conditions exerting an influence on the acceptance, dissemination, and use of Open Access (OA).

The research project focuses on the German-speaking countries. VET research represents an interlinking of various related academic research areas, rather than comprising a stand-alone discipline.

Therefore, the assumption must be that the results of the project will be at least partially transferable to other fields within the social sciences and the humanities and will thus contribute towards findings with regard to OA across the whole of the latter domain.

The background to the project is underpinned by science communication and by media theory. The empirical basis of the study has its foundations in a Sequential Mixed Method Design with a qualitative strand, followed by a quantitative strand.

The qualitative exploration via focus groups will lead to hypotheses for the online survey. The online survey will be aimed at academic researchers from various disciplines who share common ground in that they address topics that are related to VET research. The realisation of the research project is planned for 2018–2020.

URL : Open Access in Vocational Education and Training Research

DOI : https://doi.org/10.3390/publications6030029

Enhancing the Research Data Management of Computer-Based Educational Assessments in Switzerland

Authors : Catharina Wasner, Ingo Barkow, Fabian Odoni

Since 2006 the education authorities in Switzerland have been obliged by the Constitution to harmonize important benchmarks in the educational system throughout Switzerland. With the development of national educational objectives in four disciplines an important basis for the implementation of this constitutional mandate was created.

In 2013 the Swiss National Core Skills Assessment Program (in German: ÜGK – Überprüfung der Grundkompetenzen) was initiated to investigate the skills of students, starting with three of four domains: mathematics, language of teaching and first foreign language in grades 2, 6 and 9. ÜGK uses a computer-based test and a sample size of 25.000 students per year.

A huge challenge for computer-based educational assessment is the research data management process. Data from several different systems and tools existing in different formats has to be merged to obtain data products researchers can utilize.

The long term preservation has to be adapted as well. In this paper, we describe our current processes and data sources as well as our ideas for enhancing the data management.

URL : Enhancing the Research Data Management of Computer-Based Educational Assessments in Switzerland

DOI : http://doi.org/10.5334/dsj-2018-018

A Conceptual Enterprise Framework for Managing Scientific Data Stewardship

Authors : Ge Peng, Jeffrey L. Privette, Curt Tilmes, Sky Bristol, Tom Maycock, John J. Bates, Scott Hausman, Otis Brown, Edward J. Kearns

Scientific data stewardship is an important part of long-term preservation and the use/reuse of digital research data. It is critical for ensuring trustworthiness of data, products, and services, which is important for decision-making.

Recent U.S. federal government directives and scientific organization guidelines have levied specific requirements, increasing the need for a more formal approach to ensuring that stewardship activities support compliance verification and reporting.

However, many science data centers lack an integrated, systematic, and holistic framework to support such efforts. The current business- and process-oriented stewardship frameworks are too costly and lengthy for most data centers to implement.

They often do not explicitly address the federal stewardship requirements and/or the uniqueness of geospatial data. This work proposes a data-centric conceptual enterprise framework for managing stewardship activities, based on the philosophy behind the Plan-Do-Check-Act (PDCA) cycle, a proven industrial concept.

This framework, which includes the application of maturity assessment models, allows for quantitative evaluation of how organizations manage their stewardship activities and supports informed decision-making for continual improvement towards full compliance with federal, agency, and user requirements.

URL : A Conceptual Enterprise Framework for Managing Scientific Data Stewardship

DOI : http://doi.org/10.5334/dsj-2018-015

Social media metrics for new research evaluation

Authors : Paul Wouters, Zohreh Zahedi, Rodrigo Costas

This chapter approaches, both from a theoretical and practical perspective, the most important principles and conceptual frameworks that can be considered in the application of social media metrics for scientific evaluation.

We propose conceptually valid uses for social media metrics in research evaluation. The chapter discusses frameworks and uses of these metrics as well as principles and recommendations for the consideration and application of current (and potentially new) metrics in research evaluation.

URL  : https://arxiv.org/abs/1806.10541

Ten considerations for open peer review

Authors : Birgit Schmidt, Tony Ross-Hellauer, Xenia van Edig, Elizabeth C Moylan

Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes.

But what is meant by `open peer review’, and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for?

Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues.

We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community.

URL : Ten considerations for open peer review

DOI : http://dx.doi.org/10.12688/f1000research.15334.1