The Social Structure of Consensus in Scientific Review

Authors : Misha Teplitskiy, Daniel Acuna, Aida Elamrani-Raoult, Konrad Kording, James Evans

Personal connections between creators and evaluators of scientific works are ubiquitous, and the possibility of bias ever-present. Although connections have been shown to bias prospective judgments of (uncertain) future performance, it is unknown whether such biases occur in the much more concrete task of assessing the scientific validity of already completed work, and if so, why.

This study presents evidence that personal connections between authors and reviewers of neuroscience manuscripts are associated with biased judgments and explores the mechanisms driving the effect.

Using reviews from 7,981 neuroscience manuscripts submitted to the journal PLOS ONE, which instructs reviewers to evaluate manuscripts only on scientific validity, we find that reviewers favored authors close in the co-authorship network by ~0.11 points on a 1.0 – 4.0 scale for each step of proximity.

PLOS ONE’s validity-focused review and the substantial amount of favoritism shown by distant vs. very distant reviewers, both of whom should have little to gain from nepotism, point to the central role of substantive disagreements between scientists in different “schools of thought.”

The results suggest that removing bias from peer review cannot be accomplished simply by recusing the closely-connected reviewers, and highlight the value of recruiting reviewers embedded in diverse professional networks.

URL : https://arxiv.org/abs/1802.01270

The rise of the middle author: Investigating collaboration and division of labor in biomedical research using partial alphabetical authorship

Authors : Philippe Mongeon, Elise Smith, Bruno Joyal, Vincent Larivière

Contemporary biomedical research is performed by increasingly large teams. Consequently, an increasingly large number of individuals are being listed as authors in the bylines, which complicates the proper attribution of credit and responsibility to individual authors.

Typically, more importance is given to the first and last authors, while it is assumed that the others (the middle authors) have made smaller contributions. However, this may not properly reflect the actual division of labor because some authors other than the first and last may have made major contributions.

In practice, research teams may differentiate the main contributors from the rest by using partial alphabetical authorship (i.e., by listing middle authors alphabetically, while maintaining a contribution-based order for more substantial contributions). In this paper, we use partial alphabetical authorship to divide the authors of all biomedical articles in the Web of Science published over the 1980–2015 period in three groups: primary authors, middle authors, and supervisory authors.

We operationalize the concept of middle author as those who are listed in alphabetical order in the middle of an authors’ list. Primary and supervisory authors are those listed before and after the alphabetical sequence, respectively.

We show that alphabetical ordering of middle authors is frequent in biomedical research, and that the prevalence of this practice is positively correlated with the number of authors in the bylines. We also find that, for articles with 7 or more authors, the average proportion of primary, middle and supervisory authors is independent of the team size, more than half of the authors being middle authors.

This suggests that growth in authors lists are not due to an increase in secondary contributions (or middle authors) but, rather, in equivalent increases of all types of roles and contributions (including many primary authors and many supervisory authors).

Nevertheless, we show that the relative contribution of alphabetically ordered middle authors to the overall production of knowledge in the biomedical field has greatly increased over the last 35 years.

URL : The rise of the middle author: Investigating collaboration and division of labor in biomedical research using partial alphabetical authorship

DOI : https://doi.org/10.1371/journal.pone.0184601

 

 

Making visible the invisible through the analysis of acknowledgements in the humanities

Author : Adrian A. Diaz-Faes, Maria Bordons

Purpose

Science is subject to a normative structure that includes how the contributions and interactions between scientists are rewarded. Authorship and citations have been the key elements within the reward system of science, whereas acknowledgements, despite being a well-established element in scholarly communication, have not received the same attention.

This paper aims to put forward the bearing of acknowledgements in the humanities to bring to the foreground contributions and interactions that, otherwise, would remain invisible through traditional indicators of research performance.

Design/methodology/approach

The study provides a comprehensive framework to understanding acknowledgements as part of the reward system with a special focus on its value in the humanities as a reflection of intellectual indebtedness.

The distinctive features of research in the humanities are outlined and the role of acknowledgements as a source of contributorship information is reviewed to support these assumptions.

Findings

Peer interactive communication is the prevailing support thanked in the acknowledgements of humanities, so the notion of acknowledgements as super-citations can make special sense in this area.

Since single-authored papers still predominate as publishing pattern in this domain, the study of acknowledgements might help to understand social interactions and intellectual influences that lie behind a piece of research and are not visible through authorship.

Originality/value

Previous works have proposed and explored the prevailing acknowledgement types by domain.

This paper focuses on the humanities to show the role of acknowledgements within the reward system and highlight publication patterns and inherent research features which make acknowledgements particularly interesting in the area as reflection of the socio-cognitive structure of research.

URL : https://arxiv.org/abs/1706.06334

Recognizing the Diversity of Contributions: A Case Study for Framing Attribution and Acknowledgement for Scientific Data

Authors : Chung-Yi Hou, Matthew Mayernik

As scientific data volumes, format types, and sources increase rapidly with the invention and improvement of scientific capabilities, the resulting datasets are becoming more complex to manage as well.

One of the significant management challenges is pulling apart the individual contributions of specific people and organizations within large, complex projects.

This is important for two aspects:1) assigning responsibility and accountability for scientific work, and 2) giving professional credit to individuals (e.g. hiring, promotion, and tenure) who work within such large projects.

This paper aims to review the extant practice of data attribution and how it may be improved. Through a case study of creating a detailed attribution record for a climate model dataset, the paper evaluates the strengths and weaknesses of the current data attribution method and proposes an alternative attribution framework accordingly.

The paper concludes by demonstrating that, analogous to acknowledging the different roles and responsibilities shown in movie credits, the methodology developed in the study could be used in general to identify and map out the relationships among the organizations and individuals who had contributed to a dataset.

As a result, the framework could be applied to create data attribution for other dataset types beyond climate model datasets.

URL : Recognizing the Diversity of Contributions: A Case Study for Framing Attribution and Acknowledgement for Scientific Data

DOI : http://dx.doi.org/10.2218/ijdc.v11i1.357

The Authorship Dilemma: Alphabetical or Contribution?

Authors : Margareta Ackerman, Simina Brânzei

Scientific communities have adopted different conventions for ordering authors on publications.

Are these choices inconsequential, or do they have significant influence on individual authors, the quality of the projects completed, and research communities at large? What are the trade-offs of using one convention over another?

In order to investigate these questions, we formulate a basic two-player game theoretic model, which already illustrates interesting phenomena that can occur in more realistic settings.

We find that alphabetical ordering can improve research quality, while contribution-based ordering leads to a denser collaboration network and a greater number of publications.

Contrary to the assumption that free riding is a weakness of the alphabetical ordering scheme, this phenomenon can occur under any contribution scheme, and the worst case occurs under contribution-based ordering.

Finally, we show how authors working on multiple projects can cooperate to attain optimal research quality and eliminate free riding given either contribution scheme.

URL : https://arxiv.org/abs/1208.3391

Linking Behavior in the PER Coauthorship Network

Authors : Katharine A. Anderson, Matthew Crespi, Eleanor C. Sayre

There is considerable long-term interest in understanding the dynamics of collaboration networks, and how these networks form and evolve over time. Most of the work done on the dynamics of social networks focuses on well-established communities.

Work examining emerging social networks is rarer, simply because data is difficult to obtain in real time. In this paper, we use thirty years of data from an emerging scientific community to look at that crucial early stage in the development of a social network.

We show that when the field is very young, islands of individual researchers labored in relative isolation, and the co authorship network is disconnected. Thirty years later, rather than a cluster of individuals, we find a true collaborative community, bound together by a robust collaboration network.

However, this change did not take place gradually — the network remained a loose assortment of isolated individuals until the mid-2000s, when those smaller parts suddenly knit themselves together into a single whole.

In the rest of this paper, we consider the role of three factors in these observed structural changes: growth, changes in social norms, and the introduction of institutions such as field-specific conferences and journals.

We have data from the very earliest years of the field, and thus are able to observe the introduction of two different institutions: the first field-specific conference, and the first field-specific journals.

We also identify two relevant behavioral shifts: a discrete increase in co authorship coincident with the first conference, and a shift among established authors away from collaborating with outsiders, towards collaborating with each other. The interaction of these factors gives us insight into the formation of collaboration networks more broadly.

URL : https://arxiv.org/abs/1609.09339

Patterns of authors contribution in scientific manuscripts

Authors : Edilson A. Corrêa Jr., Filipi N. Silv, Luciano da F. Costa, Diego R. Amancio

Science is becoming increasingly more interdisciplinary, giving rise to more diversity in the areas of expertise within research labs and groups. This also have brought changes to the role researchers in scientific works. As a consequence, multi-authored scientific papers have now became a norm for high quality research.

Unfortunately, such a phenomenon induces bias to existing metrics employed to evaluate the productivity and success of researchers. While some metrics were adapted to account for the rank of authors in a paper, many journals are now requiring a description of the specific roles of each author in a publication.

Surprisingly, the investigation of the relationship between the rank of authors and their contributions has been limited to a few studies. By analyzing such kind of data, here we show, quantitatively, that the regularity in the authorship contributions decreases with the number of authors in a paper.

Furthermore, we found that the rank of authors and their roles in papers follows three general patterns according to the nature of their contributions, such as writing, data analysis, and the conduction of experiments.

This was accomplished by collecting and analyzing the data retrieved from PLoS ONE and by devising an entropy-based measurement to quantify the effective number of authors in a paper according to their contributions.

The analysis of such patterns confirms that some aspects of the author ranking are in accordance with the expected convention, such as the fact that the first and last authors are more likely to contribute more in a scientific work.

Conversely, such analysis also revealed that authors in the intermediary positions of the rank contribute more in certain specific roles, such as the task of collecting data.

This indicates that the an unbiased evaluation of researchers must take into account the distinct types of scientific contributions.

URL : https://arxiv.org/abs/1609.05545