EN

A Framework for the Assessment of Research and Its Impacts

Posted on by Hans Dillaerts

Author : Cinzia Daraio

This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, data as a main dimension, together with theory and methodology.

Each dimension of the framework is further characterized by three main building blocks: education, research, and innovation (theory); efficiency, effectiveness, and impact (methodology); and availability, interoperability, and “unit-free” property (data).

The different dimensions and their nine constituent building blocks are attributes of an overarching concept, denoted as “quality.” Three additional quality attributes are identified as implementation factors (tailorability, transparency, and openness) and three “enabling” conditions (convergence, mixed methods, and knowledge infrastructures) complete the framework.

A framework is required to develop models of metrics. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics. The proposed framework can be a useful reference for the development of the ethics of research evaluation.

It can act as a common denominator for different analytical levels and relevant aspects and is able to embrace many different and heterogeneous streams of literature. Directions for future research are provided.

URL : A Framework for the Assessment of Research and Its Impacts

DOI : https://doi.org/10.1515/jdis-2017-0018

 

EN

Co-creation and open innovation: Systematic literature review

Posted on by Hans Dillaerts

Authors : María Soledad Ramírez-Montoya,  Francisco-José García-Peñalvo

Open science, as a common good, opens possibilities for the development of nations, through innovations and collaborative constructions, which help to democratize knowledge. Advances in this area are still emerging, and the open science, cocreation of knowledge and open innovation triangle, is presented as an opportunity to generate an original contribution from research to open educational theory and practices.

The study analyzed the articles that addressed this triangle, in order to identify the contexts and challenges that arise in open innovation and the cocreation of knowledge to promote open science.

The method was a systematic literature review (SLR) of 168 articles published in open access format, from January 2014 to May 2017 in the Web of Science and Scopus databases.

In the validation process, the York University criteria were used: inclusion and exclusion, relevance of the pertinent studies, evaluation of the quality / validity of included studies and description of data / basic studies.

The findings showed that the mostwidely publicized contexts were in the United States and Brazil, in the business and academic sectors (closely followed by the social sector), and the challenges were open to innovation, opening and research.

The research concludes that the context and practices of collaboration are substantial elements for innovation and open science.

URL :  Co-creation and open innovation: Systematic literature review

Alternative location : http://eprints.rclis.org/32168/

EN

Wikipedia in higher education: Changes in perceived value through content contribution

Posted on by Hans Dillaerts

Authors : Joan Soler-Adillon, Dragana Pavlovic

Wikipedia is a widely used resource by university students, but it is not necessarily regarded as being reliable and trustworthy by them, nor is it seen as a context in which to make content contributions.

This paper presents a teaching and research project that consisted in having students edit or create Wikipedia articles and testing whether or not this experience changed their perceived value of the platform. We conducted our experience at Universitat Pompeu Fabra (Barcelona, Spain) and University of Niš (Niš, Serbia) with a total number of 240 students.

These students edited articles and answered two questionnaires, one before and one after the exercise. We compared the pre and post experience answers to the questionnaires with a series of paired samples ttests, through which our data showed that students did significantly change their perception of reliability and usefulness, and of likeliness of finding false information on Wikipedia.

Their appreciation of the task of writing Wikipedia articles, in terms of it being interesting and challenge also increased. They did not significantly change, however, their judgement on the social value of the platform, neither in the university nor in the general context.

In addition, the open questions and informal feedback allowed us to gather valuable insights towards the evaluation of the overall experience.

URL : Wikipedia in higher education: Changes in perceived value through content contribution

Alternative location : http://eprints.rclis.org/32171/

 

 

EN

How Many More Cites is a $3,000 Open Access Fee Buying You? Empirical Evidence from a Natural Experiment

Posted on by Hans Dillaerts

Authors :  Frank Mueller-Langer, Richard Watt

This paper analyzes the effect of open access (OA) status of published journal articles on peer recognition, as measured by the number of citations. Using cross-sectional and panel data from interdisciplinary mathematics and economics journals, we perform negative binomial, Poisson and linear regressions together with generalized method of moments/instrumental variable methods regressions.

We benefit from a natural experiment via hybrid OA pilot agreements. Under these agreements, OA status is exogenously assigned to all articles of authors affiliated with hybrid OA pilot institutions.

Our cross-sectional analysis of the full sample suggests that there is no citation benefit associated with hybrid OA. In contrast, for the subpopulation of journal articles for which neither OA pre-prints nor OA post-prints are available, we find positive hybrid OA effects for the full sample and each discipline separately.

We address the issue of selection bias by exploiting a panel of journal articles for which OA pre-prints are available. Citations to pre-prints allow us to identify the intrinsic quality of articles prior to publication in a journal.

The results from the panel analysis provide additional empirical evidence for a negligible hybrid OA citation effect.

URL : https://ssrn.com/abstract=3096572

EN

DataMed – an open source discovery index for finding biomedical datasets

Posted on by Hans Dillaerts

Authors : Xiaoling Chen, Anupama E Gururaj, Burak Ozyurt, Ruiling Liu, Ergin Soysal, Trevor Cohen, Firat Tiryaki, Yueling Li, Nansu Zong, Min Jiang, Deevakar Rogith, Mandana Salimi, Hyeon-eui Kim, Philippe Rocca-Serra, Alejandra Gonzalez-Beltran, Claudiu Farcas, Todd Johnson, Ron Margolis, George Alter, Susanna-Assunta Sansone, Ian M Fore, Lucila Ohno-Machado, Jeffrey S Grethe, Hua Xu

Objective

Finding relevant datasets is important for promoting data reuse in the biomedical domain, but it is challenging given the volume and complexity of biomedical data. Here we describe the development of an open source biomedical data discovery system called DataMed, with the goal of promoting the building of additional data indexes in the biomedical domain.

Materials and Methods

DataMed, which can efficiently index and search diverse types of biomedical datasets across repositories, is developed through the National Institutes of Health–funded biomedical and healthCAre Data Discovery Index Ecosystem (bioCADDIE) consortium.

It consists of 2 main components: (1) a data ingestion pipeline that collects and transforms original metadata information to a unified metadata model, called DatA Tag Suite (DATS), and (2) a search engine that finds relevant datasets based on user-entered queries.

In addition to describing its architecture and techniques, we evaluated individual components within DataMed, including the accuracy of the ingestion pipeline, the prevalence of the DATS model across repositories, and the overall performance of the dataset retrieval engine.

Results and Conclusion

Our manual review shows that the ingestion pipeline could achieve an accuracy of 90% and core elements of DATS had varied frequency across repositories. On a manually curated benchmark dataset, the DataMed search engine achieved an inferred average precision of 0.2033 and a precision at 10 (P@10, the number of relevant results in the top 10 search results) of 0.6022, by implementing advanced natural language processing and terminology services.

Currently, we have made the DataMed system publically available as an open source package for the biomedical community.

DOI : https://doi.org/10.1093/jamia/ocx121

 

EN

Open Access and the Theological Imagination

Posted on by Hans Dillaerts

Authors : Talea Anderson, David Squires

The past twenty years have witnessed a mounting crisis in academic publishing. Companies such as Reed-Elsevier, Wiley-Blackwell, and Taylor and Francis have earned unprecedented profits by controlling more and more scholarly output while increasing subscription rates to academic journals.

Thus publishers have consolidated their influence despite widespread hopes that digital platforms would disperse control over knowledge production. Open access initiatives dating back to the mid-1990s evidence a religious zeal for overcoming corporate interests in academic publishing, with key advocates branding their efforts as archivangelism.

Little attention has been given to the legacy or implications of religious rhetoric in open access debates despite its increasing pitch in recent years. This essay shows how the Protestant imaginary reconciles–rather than opposes–open access initiatives with market economics by tracing the rhetoric of openness to free-market liberalism.

Working against the tendency to accept the Reformation as an analogy for the relationship between knowledge production, publishers, and academics, we read Protestantism as a counterproductive element of the archivangelist inheritance.

URL : http://www.digitalhumanities.org/dhq/vol/11/4/000340/000340.html