Guest post from Ian Caswell, UCL Press Journals Manager.

When discussing peer review, generally, I find it helpful to remind myself of some of the values as to perhaps why researchers publish in scholarly journals. In essence, it usually comes down to these 4 headings.

1. Knowledge and discovery
2. Evaluation and validation
3. Credit
4. Access to research

Within this environment, peer review is arguably the fundamental gold standard aspect of scholarly and academic publishing and is, at least in its most fundamental use of it, the facilitator for publishers to sell journals and its content.

So then, what is the role for peer review precisely, and what does it serve to accomplish?

An easy question to answer, right? In the book Editorial peer review: It’s strengths and weaknesses, the author writes the role of what peer review serves is, as ‘the goal of the process is to ensure that the valid article is accepted, the messy article cleaned up, and the invalid article rejected,’ thereby ensuring that the article made available to the reader is quality controlled.

In another book titled Peer review: A critical inquiry, the author here writes that the process of peer review also benefits the author, as they are later certified by the process if published proceeding peer review: “Careers are often made or destroyed by the process.”

In scholarly publishing, peer review acts to validate and assess work and is the current system used to assess the quality of a manuscript before it is published. Other experts in the relevant field assesses the research or article for things like fact, validity, and significance, that aid the assessors (i.e. Editors) to determine whether the manuscript should be published in the journal or not. I think it is pertinent to remember here that journals do play a vital role in the scientific and scholarly process, by refining research through peer review and disseminating it to appropriate communities by publication, and it is this role of review by peers that has been a part of scholarly communication since the appearance of the first journal in the 17th Century (see the brilliant book by Professor Aileen Fyfe et al, A History of Scientific Journals: Publishing at the Royal Society, 1665-2015.)

Challenges in peer-review

There has been a lot of discussion around the challenges peer review present, stemming from bias and prejudices towards authors, fraudulent behaviour, non-expertise reviews, and so on. In the article Peer review in a changing world: An international study measuring the attitudes of researchers by Mulligan et al in 2012, notes that:

“Although alternative forms of peer review have evolved to tackle issues of bias, it is less clear what effect, if any, they will have upon fraud. High‐profile cases of fraud and plagiarism have brought the debate about the efficacy of peer review to a wider audience, attracting greater public attention. Such incidences include [certain individuals], tipped to be a Nobel Prize winner, who published a series of fraudulent papers that were withdrawn from Nature, Science, Physical Review, and Applied Physics Letters.”

Journals typically tackle these types of concerns by anonymising authors and reviewers from each other to ‘enable a fairer and just review system’. In this article Mulligan et al surveyed around 40,000 published researchers that were randomly selected from the Web of Science (then known as the Thomson Reuters ISI list) and concluded that the majority of respondents were happy with the current system, but noted the system is imperfect and more can be done to ensure a higher level of efficacy and efficiency.

Now, being led by open science principles, it is largely seen that being more open and transparent with research publication and assessment can we increase scholarly rigor, accountability and trust.

What is open peer-review?

There is a growing evidence base of the challenges and flaws in the current anonymised peer review system (albeit, mainly within the biomedical and clinical sciences), and major publishers and journals are already testing open peer review processes (or have already implemented a practice of it already).

In April 2017, a systematic review of what open peer review is was published online in F1000Research (itself an innovative model of open peer review). It concluded: “Open peer review has neither a standardized definition, nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions.”

What this review very accurately depicts, is that there are a number of definitions of open peer review that can be collated together into themes and it purports there are 7 open traits to what open peer review concerns itself with, and that open peer review can take either a single aspect, or a multitude or mix of any of these traits, to operate as an open peer review model. Briefly, these are:

1. Open identities, where authors and reviewers are aware of each other’s identity.
2. Open reports, where the review reports are published alongside the relevant article
3. Open participation, where the wider community are able to contribute to the review process
4. Open interaction, where direct reciprocal discussion between author(s) and reviewers, and between the reviewers themselves, is allowed and encouraged
5. Open pre-review manuscripts, essentially, a pre-print server, where manuscripts are made immediately available (e.g., BiorXiv) in advance of any formal peer review procedures
6. Open final-version commenting, where the review or commenting on the final “version of record” is published
7. Open platforms (or “decoupled review”), where review is facilitated by a different organizational entity than the venue of publication

What is UCL Press doing?

At UCL Press, we launched our very own open peer review and open science journal called UCL Open Environment: a fully non-commercial, Open Science journal, publishing high impact, multi-disciplinary research, on real world environmental issues, with the overall aim of benefitting humanity. The journal is for any researcher or professional at knowledge-based universities, institutions, and organisations (including Non-Government Organisations, Think Tanks, Inter-Government Organisations, and the United Nations) and submissions are invited from those at all career stages, including early career researchers, mid-career professionals, and senior scholars. There are also no barriers to the Open Peer Review Process (whereby the identity of the reviewer and the report are made publicly visibly at all times); engagement from all will advance the greatest leaps and discoveries.

Reviewers are firstly asked to sign in to the system using their ORCID account and when they submit their review report, the report is posted up online in the preprint server alongside the article, under the CC-BY licence and assigned a unique DOI. You can find out more information about this at https://ucl-about.scienceopen.com/for-reviewers/peer-review-process.

Reviewers can therefore attain credit of their report and readers are able to follow the process openly online. We hope this will also aid the development for others (especially earlier career researchers and students) with examples on how a review is written and how an article is revised accordingly, aiming to improve the way we should engage critically and beneficially with research.

Readers of this blog can see for themselves how the journal works (you can see here the list of the latest submissions and open peer reviews, as well as here for publications accepted after peer review). It is my hope that readers will be encouraged to provide more open peer reviews or open comments, adding to the corpus of open debate around research, and consider contributing to UCL Open Environment, as we believe that by removing barriers and innovatively working openly and together will we accelerate finding solutions to the world’s most significant challenges.

UCL Publications Board and the Open Access Team are delighted to announce that on Friday 19 November UCL’s institutional repository, UCL Discovery, reached the milestone of 30 million downloads! UCL Discovery is UCL’s open access repository, showcasing and providing access to UCL research outputs from all UCL disciplines. UCL authors currently deposit around 1,750 outputs in the repository every month (average figure January-October 2021).

by Ray Hennessy on Unsplash https://unsplash.com/photos/gdTxVSAE5sk

Our 30 millionth download was of a journal article:
Huber, LR; Poser, BA; Bandettini, PA; Arora, K; Wagstyl, K; Cho, S; Goense, J; Nothnagel, N; Morgan, AT; van den Hurk, J; Müller, AK; Reynolds, RC; Glen, DR; Goebel, R; Gulban, OF; (2021) LayNii: A software suite for layer-fMRI. NeuroImage, 237, Article 118091. 10.1016/j.neuroimage.2021.118091.

This article introduces a new software suite, LayNii, to support layer-specific functional magnetic resonance imaging: the measurement of brain activity by detecting changes associated with blood flow. The software itself, which is compatible with Linux, Windows and MacOS, is also open source via Zenodo, DockerHub, and GitHub. The authors also made a preprint version of the article available via BioRxiv in advance of formal publication in NeuroImage. This demonstrates the combined value of open source software and open access to research publications.

The author of the article based at UCL, Dr Konrad Wagstyl, deposited the article in UCL Discovery in May 2021. Dr Wagstyl is a Sir Henry Wellcome Research Fellow at the Wellcome Centre for Human Neuroimaging, UCL, and co-leads the Multicentre Epilepsy Lesion Detection project, an open science collaboration to develop machine learning algorithms to automatically subtle focal cortical dysplasias – areas of abnormal brain cell development which can cause epilepsy and seizures – in patients round the world.

The UCL Office for Open Science and Scholarship recommends that researchers make any software or code they use available to aid others in reproducing their research. The Research Data Management team maintain a guide on best practice for software sustainability, preservation and sharing, and can give further support to UCL researchers as required.

Photograph by Dr Simone Datzberger

Recently a researcher asked for our advice on the best way to disseminate her preliminary findings from a cross-disciplinary research project on COVID-19. She wanted to ensure policy makers in East Africa had immediate access to the findings so that they could make informed decisions. The researcher was aware that traditional models of publishing were not appropriate, not simply because of the length of time it generally takes for an article to be peer-reviewed and published, but because the findings would, most likely, be inaccessible to her intended audience in a subscription-based journal.

The Research Support and Open Access team advised the researcher to take a two-pronged approach which would require her to: (1) upload the working paper with the preliminary findings in a subject-specific open-access preprint service; and (2) to publicise the research findings in an online platform that is both credible and open access. We suggested she use SocArXiv and publish a summary of her findings in The Conversation Africa, which has a special section on COVID-19. The Conversation has several country-specific editions for Australia, Canada English, Canada French, France, Global Perspectives, Indonesia, New Zealand, Spain, United Kingdom and the United States, and is a useful vehicle to get academic research read by decision makers and the members of the public. We also suggested that the researcher publicise the research on the IOE London Blog.

What are ‘working papers’ & ‘preprint services’?

UCL’s Institute of Education has a long-standing tradition of publishing working papers to signal work-in-progress, share initial findings, and elicit feedback from other researchers working in the same area. The preprint service used thus far at the IOE is RePEc (Research Papers in Economics), which includes papers in education and the related social sciences). RePEc is indexed by the database publisher EBSCO (in EconLit) and by Google Scholar and Microsoft Academic Search. Commercial platforms such as ResearchGate also trawl through RePEc and index content. Until it was purchased by Elsevier in May 2016, the Social Science Research Network or SSRN was the other popular preprint repository used by IOE researchers although its content is indexed mainly for its conference proceedings. The sale of SSRN to Elsevier resulted in a fallout between authors and the publisher, and this resulted in SocArXiv entering the scene. SocArXiv is an open access, open source preprint server for the social sciences which accepts text files, data and code. It is the brainchild of the non-profit Centre for Open Science (COS) whose mission is to increase openness, integrity and reproducibility of research – values that are shared by UCL and are promoted on this blog and by the newly formed Office of Open Science and Scholarship (for more information see also the Pillars of Open Science). In the spirit of openness, most papers on SocArXiv use the creative commons license CC-By Attribution-NonCommercial-NonDerivatives 4.0 International, which safeguards the rights of the author. As papers on SocArXiv are automatically assigned Digital Object Identifiers (DOI), they discoverable on the web, particularly as Scholar indexes SocArXiv content.

What are the benefits of using preprint servers?

Whilst research repositories such as UCL Discovery are curtailed by publisher policies on what research can be made open access, this is not always the case for papers submitted on preprint subject repositories. Without wanting to repeat what my colleague Patrycja Barczymska has already written in her post on preprints I can confirm that in addition to signalling the research findings and eliciting feedback, other benefits to depositing in preprint servers include the enhanced discoverability as most will automatically generate DOIs at the time a paper is uploaded, the possibility of obtaining early citations and the alternative metrics that indicate interest (e.g. the number of downloads, mentions, etc.)  that services such as SocArXiv provide. Researchers can also list open-access working papers in funding applications.

Does uploading a working paper in a pre-print server hinder the publication of the final paper?

Researchers are concerned, and rightly so, that publishers may not publish their final research output if preliminary findings are deposited in preprint servers as working papers. However, more often than not, working papers are exactly that – work in progress. They are not the final article that gets submitted for publication.  It is also likely that the preliminary findings and conclusions in the working paper will be somewhat different from the final version of the paper. It is worth knowing that some of the key social sciences publishers, such as Sage, Springer, and Taylor and Francis / Routledge and Wiley, explicitly state that they will accept content that has been deposited on a preprint server, as long as it is a non-commercial preprint service. In other words, researchers must not upload the working papers on platforms such as academia.edu and ResearchGate.

These ‘preprint-friendly’ publishers simply ask that the author informs them of the existence of a preprint and provides the DOI of the working paper at the time of submitting their article. Some ask that authors update the preprint to include the bibliographic details, including the new DOI, when their article is published, and that authors add a statement requesting readers to cite the published article rather than the preprint publication. Although a definitive list of individual journal policies does not exist, submission guidelines generally clarify issues related to preprints. Researchers may want to use the Sherpa Romeo service (and Sherpa Juliet for key funder policies) to obtain additional information.

More than a success story

The above case demonstrates how preliminary research findings can be shared expeditiously and in an open environment to aid the decision-making process.  It also demonstrates that open-access subject-specific preprint services can be beneficial to promoting both the research and the researcher, and that there is now wider acceptance among publishers that the traditional models of publishing are not always viable. This is especially true where cutting-edge research is required as in the case of research on COVID-19.