Pathways to Open Access
  • A A A

    Workshop summary: REF Exceptions & Subject Repositories

    By Alan Bracey, on 4 February 2016

    Our workshop on the 25th proved a practical and thought-provoking day, generating some excellent feedback. Of particular interest was HEFCE’s stance on exceptions, the opportunity to share current approaches to recording exceptions, and an exploration of reporting REF compliance with Europe PMC and arXiv. A special thanks to all our speakers. A summary of all sessions and links to slides follows.


    Hannah DeGroff provided an overview and update on Jisc’s OA Good Practice Programme, touching on the recently released REF Compliance Checker plugin for EPrints, and calling for institutions to join the Publications Router service (email for info). The Router will populate institutions’ repositories and CRISs with their researchers’ papers. Ben Johnson from HEFCE later emphasised that publishers are unlikely to sign up to the service if institutions haven’t already. Hannah’s slides are available here.


    Catherine Sharp from UCL reminded us of the exceptions, and summarised the key issues with each, including: difficulties in securing author manuscripts of multi-author papers, problems institutions have in tracking Gold OA papers paid directly by authors or other institutions, how to demonstrate that authors have considered “appropriateness” when publishing in a non-compliant title, who should record exceptions (authors or OA Teams) – and how, e.g. using various publications management systems. Alan Bracey outlined an analysis of UCL’s 2014 REF submission with regard to embargo and deposit exceptions, and presented a community list of non-compliant publishers and journals (see this blog post for more). Slides are available here.


    Ben Johnson outlined HEFCE’s position on exceptions. He reiterated that the scholarly publications system is broken, relies on out-of-date technology, and needs to be changed. He clarified some key points:

    • REF2021 has been committed to by the government in the Green PaperThe OA policy needs to be simple, flexible, but firm. The goal in the first instance is to increase open access.
    • HEFCE wishes to empower institutions to make judgements with regard to policy interpretation and implementation. “Best endeavours” in point 44 of the policy should be interpreted as best efforts by institutions to establish a framework to maximise compliance, rather than best efforts by academics themselves. Claiming exceptions will not be made difficult.
    • HEFCE understands that researchers’ deposit rates are unlikely to be sufficient without intervention by institutions, who in turn need reliable publication alerts to prompt action. 
    • The requirement to deposit manuscripts within three months of publication (begins April 2016) could be interpreted as applying to final publication in an issue. (However, HEFCE wants to avoid introducing delays into the deposit process.) This, and the requirement to deposit within 3 months of acceptance (due to start April 2017), will be reviewed in Autumn 2016.
    • If academics aren’t aware of journal policies when publishing, they should be. Work is needed to make some publishers’ OA policies clear – these publishers are not supporting open access on the ground (despite claims otherwise). Unclear or missing publisher policies is unacceptable, and passive non-compliance by publishers should be challenged. This will take time.
    • The “appropriateness” of a journal is for the academic to decide, following discussion with their institution. HEFCE will want to see evidence of a system in place to enable researchers to understand this requirement.


    Tom Parsons updated delegates on HEFCE’s new Sherpa REF service. Sherpa REF will provide researchers and institutions with a quick yes/no answer to the question of a journal’s compliance. Designed with a simple interface, a stable beta release is due in February. Tom explored some of the complications involved in boiling 4000 words of policy to a checklist for assessing journal policies. Sherpa REF entries have been entered manually, since publishers do not make machine-readable policies available.


    • Although the potentially small number of exceptions makes them manageable, institutions still need a process in place, rather than dealing with each exception as it comes up. Most institutions are not keeping formal evidence of exceptions, although one is keeping copies of acceptance emails.
    • In general, institutions are not communicating exceptions to researchers (it’s already hard enough communicating the bare bones of the policy). Institutions are not recording exceptions widely yet – workflows aren’t ready. A few are actively recording mainly deposit/embargo exceptions, with Excel, Symplectic’s OA Monitor, or Pure. Recording exceptions is problematic in publications management systems/CRISs. An added complication with the embargo exception: when does it start? Some publishers specify date of acceptance (e.g. RSC), some publication.
    • Agreement that exceptions should be managed by library staff rather than authors or departments – don’t want to offer a ‘get-out’. General preference for dealing with exceptions once manuscripts are deposited. Contact researchers if needed, but keep the message as simple as possible. However, there are certain exceptions that only researchers will know about in relation to a paper (e.g. unlawful/security risk). Also preference for central upload of manuscripts in a department by one person e.g. library staff or research administrators.
    • No consensus on how/where Sherpa REF fits in yet. It might include too much information to be helpful to researchers; the focus should be on deposit.
    • Some departments keep a list of recommended publications for researchers: it’s unlikely that a library’s REF blacklist/whitelist would influence this. There’s concern over librarians telling researchers where to publish. There’s a risk of directing researchers to/away from journals who subsequently change their OA policy. However, a REF blacklist may be useful for pressuring publishers to change policies.
    • There’s a general preference for reporting to HEFCE via upload from Pure/Symplectic/equivalent system, in RIOXX-compliant form.
    • Compliance of journals (embargoes) depends on the panel to which the paper is submitted. Institutions can use their 2014 submissions as a guide to which journals are submitted to which panels, however an article may subsequently be entered to panel A or B, instead of C or D.
    • There’s uncertainty on how to handle the Gold OA exception. HEFCE seem not to require evidence. There’s general preference for tracking Gold so institutions know which papers won’t need to comply. It’s difficult to track papers paid for by researchers themselves/other institutions. There’s also potential for mixed messages with the Gold exception, since deposit/action isn’t required from the researcher. Is the REF policy still a Green OA policy? Some institutions with their own OA funds are still encouraging Green deposit when paying Gold fees.


    Cecy Marden gave an overview of how the Trust uses Europe PMC to monitor compliance of funded journal papers with its open access policy, provided some tips and tricks for reporting, and discussed compliance rates. EPMC struggles to check OA licences accurately, because of errors in publishers’ metadata. The current solution is using the Cottage Labs tool to determine the type of licence used. Cecy’s slides (including example searches) are available here.


    Catherine Sharp presented results of testing Europe PMC and PMC to identify author manuscripts. Europe PMC allows searching for author manuscripts, but currently provides unreliable results. A fix for this is forthcoming, and other developments to aid REF reporting are being discussed. PMC successfully finds manuscripts, although numbers are low. Catherine’s slides (including example searches) are available here.


    Danny Kingsley led a discussion of the limitations of and possibilities for using arXiv to record REF compliance. ArXiv is an international subject repository hosting nearly 10,000 new open access papers a month. It generated over 14 million downloads in Nov 2015. But it would need changes to enable reporting for REF. Cambridge and other Russell Group institutions have discussed the potential cost and timeframe: about $75,000 to instate acceptance date and version, with 6 months’ development time. The cost could be shared by Russell Group universities and/or Jisc. However some problems have been identified: the risk of setting a precedent for funding changes to other subject repositories, that a significant proportion of deposits may turn out to be pre-prints/submitted versions instead of accepted manuscripts, and concerns over interfering with a successful community tool for the sake of reporting funder compliance. It was generally agreed that more needs to be known about how researchers use arXiv before committing to funding changes. Discussions are ongoing, and participation is invited from institutions who are not already involved. Slides are available here.