The Rise of the Knowledge Broker

Posté le par

Knowledge brokers are people or organizations that move knowledge around and create connections between researchers and their various audiences.

This commentary reviews some of the literature on knowledge brokering and lays out some thoughts on how to analyze and theorize this practice.

Discussing the invisibility and interstitiality of knowledge brokers, the author argues that social scientists need to analyze more thoroughly their practices, the brokering devices they use, and the benefits and drawbacks of their double peripherality.

The author also argues that knowledge brokers do not only move knowledge, but they also produce a new kind of knowledge: brokered knowledge.”

URL : http://hal-ensmp.archives-ouvertes.fr/hal-00493794/fr/

Willingness to Share Research Data Is Related to…

Posté le par

Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results :

“Background : The widespread reluctance to share published research data is often hypothesized to be due to the authors’ fear that reanalysis may expose errors in their work or may produce conclusions that contradict their own. However, these hypotheses have not previously been studied systematically.

Methods and Findings : We related the reluctance to share research data for reanalysis to 1148 statistically significant results reported in 49 papers published in two major psychology journals. We found the reluctance to share data to be associated with weaker evidence (against the null hypothesis of no effect) and a higher prevalence of apparent errors in the reporting of statistical results. The unwillingness to share data was particularly clear when reporting errors had a bearing on statistical significance.

Conclusions : Our findings on the basis of psychological papers suggest that statistical results are particularly hard to verify when reanalysis is more likely to lead to contrasting conclusions. This highlights the importance of establishing mandatory data archiving policies.”

URL : http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0026828
doi:10.1371/journal.pone.0026828

Achieving rigor and relevance in online multimedia scholarly…

Posté le par

Achieving rigor and relevance in online multimedia scholarly publishing :

“This paper discusses the importance of relevance and rigor in scholarly publishing in a new media–rich world. We defend that scholarship should be useful and engaging to audiences through the use of new media, and at the same time scholarly publishers must develop and maintain methods of ensuring content accuracy and providing quality controls in the production of scholarly multimedia products. We review examples and a case study of existing scholarly publishing venues that attempt to maintain quality control standards while embracing innovative multimedia formats. We also present lessons learned from the case experience and challenges that face us in the scholarly publication of multimedia.”

URL : http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/3762/3119

Understanding collaboration in Wikipedia Wikipedia stands as…

Posté le par

Understanding collaboration in Wikipedia :

“Wikipedia stands as an undeniable success in online participation and collaboration. However, previous attempts at studying collaboration within Wikipedia have focused on simple metrics like rigor (i.e., the number of revisions in an article’s revision history) and diversity (i.e., the number of authors that have contributed to a given article) or have made generalizations about collaboration within Wikipedia based upon the content validity of a few select articles. By looking more closely at metrics associated with each extant Wikipedia article (N=3,427,236) along with all revisions (N=225,226,370), this study attempts to understand what collaboration within Wikipedia actually looks like under the surface. Findings suggest that typical Wikipedia articles are not rigorous, in a collaborative sense, and do not reflect much diversity in the construction of content and macro–structural writing, leading to the conclusion that most articles in Wikipedia are not reflective of the collaborative efforts of the community but, rather, represent the work of relatively few contributors.”

URL : http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/3613/3117

A Study of Innovative Features in Scholarly Open…

Posté le par

A Study of Innovative Features in Scholarly Open Access Journals :

“Background: The emergence of the Internet has triggered tremendous changes in the publication of scientific peer-reviewed journals. Today, journals are usually available in parallel electronic versions, but the way the peer-review process works, the look of articles and journals, and the rigid and slow publication schedules have remained largely unchanged, at least for the vast majority of subscription-based journals. Those publishing firms and scholarly publishers who have chosen the more radical option of open access (OA), in which the content of journals is freely accessible to anybody with Internet connectivity, have had a much bigger degree of freedom to experiment with innovations.

Objective: The objective was to study how open access journals have experimented with innovations concerning ways of organizing the peer review, the format of journals and articles, new interactive and media formats, and novel publishing revenue models.

Methods: The features of 24 open access journals were studied. The journals were chosen in a nonrandom manner from the approximately 7000 existing OA journals based on available information about interesting journals and include both representative cases and highly innovative outlier cases.

Results: Most early OA journals in the 1990s were founded by individual scholars and used a business model based on voluntary work close in spirit to open-source development of software. In the next wave, many long-established journals, in particular society journals and journals from regions such as Latin America, made their articles OA when they started publishing parallel electronic versions. From about 2002 on, newly founded professional OA publishing firms using article-processing charges to fund their operations have emerged. Over the years, there have been several experiments with new forms of peer review, media enhancements, and the inclusion of structured data sets with articles. In recent years, the growth of OA publishing has also been facilitated by the availability of open-source software for journal publishing.

Conclusions: The case studies illustrate how a new technology and a business model enabled by new technology can be harnessed to find new innovative ways for the organization and content of scholarly publishing. Several recent launches of OA journals by major subscription publishers demonstrate that OA is rapidly gaining acceptance as a sustainable alternative to subscription-based scholarly publishing.”

URL : http://www.jmir.org/2011/4/e115/

Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact

Posté le par

Background

Citations in peer-reviewed articles and the impact factor are generally accepted measures of scientific impact. Web 2.0 tools such as Twitter, blogs or social bookmarking tools provide the possibility to construct innovative article-level or journal-level metrics to gauge impact and influence. However, the relationship of the these new metrics to traditional metrics such as citations is not known.

Objective:

(1) To explore the feasibility of measuring social impact of and public attention to scholarly articles by analyzing buzz in social media, (2) to explore the dynamics, content, and timing of tweets relative to the publication of a scholarly article, and (3) to explore whether these metrics are sensitive and specific enough to predict highly cited articles.

Methods

Between July 2008 and November 2011, all tweets containing links to articles in the Journal of Medical Internet Research (JMIR) were mined. For a subset of 1573 tweets about 55 articles published between issues 3/2009 and 2/2010, different metrics of social media impact were calculated and compared against subsequent citation data from Scopus and Google Scholar 17 to 29 months later. A heuristic to predict the top-cited articles in each issue through tweet metrics was validated.

Results

A total of 4208 tweets cited 286 distinct JMIR articles. The distribution of tweets over the first 30 days after article publication followed a power law (Zipf, Bradford, or Pareto distribution), with most tweets sent on the day when an article was published (1458/3318, 43.94% of all tweets in a 60-day period) or on the following day (528/3318, 15.9%), followed by a rapid decay. The Pearson correlations between tweetations and citations were moderate and statistically significant, with correlation coefficients ranging from .42 to .72 for the log-transformed Google Scholar citations, but were less clear for Scopus citations and rank correlations. A linear multivariate model with time and tweets as significant predictors (P < .001) could explain 27% of the variation of citations. Highly tweeted articles were 11 times more likely to be highly cited than less-tweeted articles (9/12 or 75% of highly tweeted article were highly cited, while only 3/43 or 7% of less-tweeted articles were highly cited; rate ratio 0.75/0.07 = 10.75, 95% confidence interval, 3.4–33.6). Top-cited articles can be predicted from top-tweeted articles with 93% specificity and 75% sensitivity.

Conclusions

Tweets can predict highly cited articles within the first 3 days of article publication. Social media activity either increases citations or reflects the underlying qualities of the article that also predict citations, but the true use of these metrics is to measure the distinct concept of social impact. Social impact measures based on tweets are proposed to complement traditional citation metrics. The proposed twimpact factor may be a useful and timely metric to measure uptake of research findings and to filter research findings resonating with the public in real time.

URL : http://www.jmir.org/2011/4/e123/

Longitudinal Trends in the Performance of Scientific Peer…

Posté le par

Longitudinal Trends in the Performance of Scientific Peer Reviewers :

“Study objective : We characterize changes in review quality by individual peer reviewers over time.

Methods : Editors at a specialty journal in the top 11% of Institute of Scientific Information journals rated the quality of every review, using a validated 5-point quality score. Linear mixed-effect models were used to analyze rating changes over time, calculating within-reviewer trends plus predicted slope of change in score for each reviewer. Reviewers at this journal have been shown comparable to those at other journals.

Results : Reviews (14,808) were performed by 1,499 reviewers and rated by 84 editors during the 14-year study. Ninety-two percent of reviewers demonstrated very slow but steady deterioration in their scores (mean –0.04 points [–0.8%] per year). Rate of deterioration was unrelated to duration of reviewing but moderately correlated with mean reviewer quality score (R=0.52). The mean score of each reviewer’s first 4 reviews predicted subsequent performance with a sensitivity of 75% and specificity of 47%. Scores of the group stayed constant over time despite deterioration because newly recruited reviewers initially had higher mean quality scores than their predecessors.

Conclusion : This study, one of few tracking expert performance longitudinally, demonstrates that most journal peer reviewers received lower quality scores for article assessment over the years. This could be due to deteriorating performance (caused by either cognitive changes or competing priorities) or, to a partial degree, escalating expectations; other explanations were ruled out. This makes monitoring reviewer quality even more crucial to maintain the mission of scientific journals.”

URL : http://www.annemergmed.com/article/S0196-0644(10)01266-7/fulltext