Guest post by Andrew Gray, UCL Bibliometrics Support Officer

The UCL Research Support team has recently launched a comprehensive new LibGuide on Research Metrics. This resource covers a range of topics, from how to use and understand bibliometrics (citation metrics and altmetrics) to guidance on specific tools and advice on handling publications data. Learn more about this guide to enhance your research impact and better understand the world of research metrics!

Bibliometrics

The core of the new guide is focusing on guidance for using and understanding research metrics, such as bibliometrics, citation metrics, and altmetrics. It explains how to access citation counts through Scopus and Web of Science, and more complex normalised metrics through InCites. It also gives guidance on how to best interpret and understand those metrics, and advice on metrics to avoid using. The guide also covers the UCL Bibliometrics Policy, which governs the use of bibliometric data for internal assessments at UCL, and sets some limits on what should be used.

Guidance for Tools

Within the LibGuide, you will also find guidance pages for how to use specialised services like InCites, Altmetric, and Overton to measure research impact. Additionally, the guide offers advice on using other tools that UCL does not subscribe to but may be beneficial for research support. This includes three freely available large bibliographic databases—Lens, Dimensions, and OpenAlex—which provide broader coverage than Web of Science and Scopus. It also outlines how to use a range of tools for citation-network based searching like Research Rabbit, Connected Papers, and Litmaps, as well as modern AI-supported search and summarising tools such as Scite, Keenious, and Consensus.

These are of course not the only tools available – especially with AI-supported tools, there are frequently tools being released – but these are ones we have been asked to investigate by students and researchers. If you would like feedback on another tool you are considering using, please get in touch.

Publications data

The LibGuide also addresses broader questions about using publications data. It outlines how to download publication and metrics datasets from Web of Science, Scopus, InCites, and Altmetric, and gives some guidance on how to link datasets from different sources together. Learn more about using publications data.

Additionally, the guide also explains how best to interpret data drawn from UCL-specific sources such as RPS, data ensuring you can make the most of the data available to you.

This new LibGuide is an important resource for anyone looking to expand their understanding of research metrics and manage their publications data. Visit the guide today to explore these tools and resources in detail.

Further support

We offer regular online or in-person training sessions as part of the Library Skills program. Please see the Library Skills calendar for dates and bookings. There are also three self-paced online sessions available through the Library Skills Moodle.

For any enquiries about bibliometrics, please contact us on bibliometrics@ucl.ac.uk 

Get involved!

The UCL Office for Open Science and Scholarship invites you to contribute to the open science and scholarship movement. Stay connected for updates, events, and opportunities. Follow us on X, formerly Twitter, LinkedIn, and join our mailing list to be part of the conversation!

 

 

Announcement from Paul Ayris, Pro-Vice Provost, UCL Library, Culture, Collections and Open Science

How can Open Science/Open Research support career progression and development? How does the adoption of Open Science/Open Research approaches benefit individuals in the course of their career?

The UCL Open Science Office, in conjunction with colleagues across UCL, has produced a series of Case Studies showing how UCL academics can use Open Science/Open Research approaches in their plans for career development, in applications for promotion and in appraisal documents.

In this way, Open Science/Open Research practice can become part of the Research Culture that UCL is developing.

The series of Case Studies covers each of the 8 pillars of Open Science/Open Research. They can be found on a new webpage: Open Science Case Studies 4 UCL.

It is only fair that academics should be rewarded for developing their skills and adopting best practice in research and in its equitable dissemination. The Case Studies show how this can be done, and each Case Study identifies a Key Message which UCL academics can use to shape their activities.

Examples of good practice are:

• Publishing outputs as Open Access outputs
• Sharing research data which is used as the building block of academic books and papers
• Creating open source software which is then available for others to re-use and develop
• Adopting practices allied to Reproducibility and Research Integrity
• The responsible use of Bibliometrics
• Public Engagement: Citizen Science and Co-Production as mechanisms to deliver results

Contact the UCL Open Science Office for further information at openscience@ucl.ac.uk.

Post by Andrew Gray (Bibliometrics Support Officer) and Dominic Allington Smith (Open Access Publications Manager)

Summary

UCL is a longstanding and steadfast supporter of open access publishing, organising funding and payment for gold open access, maintaining the UCL Discovery repository for green open access, and monitoring compliance with REF and research funder open access requirements.  Research data can  be made open access in the Research Data Repository, and UCL Press also publish open access books and journals.

The UCL Bibliometrics Team have recently conducted research to analyse UCL’s overall open access output, covering both total number of papers in different OA categories, and citation impact.  This blog post presents the key findings:

1. UCL’s overall open access output has risen sharply since 2011, flattened around 80% in the last few years, and is showing signs of slowly growing again – perhaps connected with the growth of transformative agreements.
2. The relative citation impact of UCL papers has had a corresponding increase, though with some year-to-year variation.
3. UCL’s open access papers are cited around twice as much, on average, as non-open-access papers.
4. UCL is consistently the second-largest producer of open access papers in the world, behind Harvard University.
5. UCL has the highest level of open access papers among a reference group of approximately 80 large universities, at around 83% over the last five years.

Overview and definitions

Publications data is taken from the InCites database.  As such, the data is primarily drawn from InCites papers attributed to UCL, filtered down to only articles, reviews, conference proceedings, and letters. It is based on published affiliations to avoid retroactive overcounting in past years: existing papers authored by new starters at UCL are excluded.

The definition of “open access” provided by InCites is all open access material – gold, green, and “bronze”, a catch-all category for material that is free-to-read but does not meet the formal definition of green or gold. This will thus tend to be a few percentage points higher than the numbers used for, for example, UCL’s REF open access compliance statistics.

Data is shown up to 2021; this avoids any complications with green open access papers which are still under an embargo period – a common restriction imposed by publishers when pursuing this route – in the most recent year.

1. UCL’s change in percentage of open access publications over time

(InCites all-OA count)

The first metric is the share of total papers recorded as open access.  This has grown steadily over time over the last decade, from under 50% in 2011 to almost 90% in 2021, with only a slight plateau around 2017-19 interrupting progress.

2. Citation impact of UCL papers over time

(InCites all-OA count, Category Normalised Citation Impact)

The second metric is the citation impact for UCL papers.  These are significantly higher than average: the most recent figure is above 2 (which means that UCL papers receive over twice as many citations as the world average; the UK university average is ~1.45) and continue a general trend of growing over time, with some occasional variation. Higher variation in recent years is to some degree expected, as it takes time for citations to accrue and stabilise.

3. Relative citation impact of UCL’s closed and Open Access papers over time

(InCites all-OA count, Category Normalised Citation Impact)

The third metric is the relative citation rates compared between open access and non-open access (“closed”) papers. Open access papers have a higher overall citation rate than closed papers: the average open access paper from 2017-21 has received around twice as many citations as the average closed paper.

4. World leading universities by number of Open Access publications

(InCites all-OA metric)

Compared to other universities, UCL produces the second-highest absolute number of open access papers in the world, climbing above 15,000 in 2021, and has consistently been the second largest publisher of open access papers since circa 2015.

The only university to publish more OA papers is Harvard. Harvard typically publishes about twice as many papers as UCL annually, but for OA papers this gap is reduced to about 1.5 times more papers than UCL.

5. World leading universities by percentage of Open Access publications

(5-year rolling average; minimum 8000 publications in 2021; InCites %all-OA metric)

UCL’s percentage of open access papers is consistently among the world’s highest.  The most recent data from InCites shows UCL as having the world’s highest level of OA papers (82.9%) among institutions with more than 8,000 papers published in 2021, having steadily risen through the global ranks in previous years.

Conclusion

The key findings of this research are very good news for UCL, indicating a strong commitment by authors and by the university to making work available openly.  Furthermore, whilst high levels of open access necessarily lead to benefits relating to REF and funder compliance, the analysis also indicates that making research outputs open access leads, on average, to a greater number citations, providing further justification for this support, as being crucial to communicating and sharing research outcomes as part of the UCL 2034 strategy.

Get involved!

The UCL Office for Open Science and Scholarship invites you to contribute to the open science and scholarship movement. Stay connected for updates, events, and opportunities. Follow us on X, formerly Twitter, LinkedIn, and join our mailing list to be part of the conversation!

 

Guest post by Andrew Gray (Bibliometrics Support Officer)

What is it? 

Altmetrics are the concept of “alternative metrics” – measuring the impact of research beyond scholarly literature. It encompasses a wide range of activity in diverse sources  social media (eg twitter, blogs), news publications, and grey literature (eg policy documents). This can help to get a wider sense of the impact of papers that might otherwise be missed were we to focus just on traditional academic citations. 

The primary commercial database for these is Altmetric (https://altmetric.com) – UCL has just taken out a one-year subscription to this service. We hope it will be useful for anyone interested in public engagement or research impact, as well as individual researchers looking at the response to their own work. 

It is open to everyone at UCL by visiting https://www.altmetric.com/explorer/login and entering your UCL email address. It will then authenticate through the UCL single-sign-on system. 

How does it work? 

Altmetric tracks a range of individual sources looking for DOIs, links to papers, or free-text descriptions of articles. It then matches these to the underlying paper and produces an index of the mentions. Here we can see the range of responses to a climate-change study. 

You will also sometimes see this coloured “doughnut” on publisher or repository sites – clicking through will get you to this same page. 

The most interesting part of the service, however, is the dashboard. This aggregates the results from all individual papers, and we can then filter down by subject area, date, publication venue, etc., to produce a more specific analysis. It is also possible to search for keywords to see the change in activity around a specific topic – one like “artificial intelligence” tends to show a steady level of interest, while one like “gravitational waves” shows very dramatic spikes connected with major discoveries. 

What can we do with it? 

The dashboard has been integrated with UCL’s RPS service, so it has a dataset of UCL papers since 2013, each linked to the faculty/department of the authors. This means we can do the same types of analysis for just UCL papers – or just those from a specific department or a specific author. 

The search can also be tweaked to identify specific topics. Here we can see policy documents published in 2022 which cite a Bartlett paper. 

Policy documents are one of the key strengths of Altmetric – they can be used as evidence of wider impact, especially for the social sciences. While they are formal documents, and very distinct from more ephemeral news or social media mentions, they are not indexed in most citation databases and so this impact can often be hard to trace. 

Altmetric data can also be exported – any set of results can be exported so that we can do detailed offline analysis of sets of papers, or at the individual mentions that make up the score. This data includes identifiers such as DOIs and ISBNs, meaning it can be linked up to other datasets easily 

What next? 

We are very keen to get this tool in the hands of as many people at UCL as possible and find how it can be used most effectively. Please have a go and let us know what you think! 

UCL-specific training and guidance is currently under development, and will be published in September 2022. Until then, please feel free to get in touch with the team (bibliometrics@ucl.ac.uk) with queries or requests for assistance. We are happy to arrange training as well. 

The tool is currently provided with a static dataset drawn from RPS, covering papers published 1 January 2013 up to 12 August 2022. We are working with the providers to improve the integration so that it will include “live” data, refreshed from RPS every night; until then, we plan to make periodic updates so that publications are added on a rolling basis. 

 

 

 

Did you intend to make new year resolutions but did not get round to it? Why not resolve to take some time this year to further develop your library research skills and ensure you are following best practices for research? UCL Library Services provides training and support to enable you to carry out your research effectively, including online guidance and self-paced tutorials, live online training sessions, tailored and individual training and specialist enquiry services.

Here are our top 5 suggested resolutions for researchers looking to enhance their library research skills and research practices:

Be FAIR

The principles of FAIR are designed to help lower barriers to research outputs and help other researchers find and understand them in order to reuse and repurpose them. This will in turn build further research opportunities and maximise the potential benefit of resources.

Findable – making research outputs discoverable by the wider academic community and the public.
Accessible – using unique identifiers, clear metadata, use of language and access protocols.
Interoperable – applying standards to encode and exchange data and metadata.
Reusable – enabling the repurposing of research outputs to maximise their research potential.

• Guides to Research Data Management and self-paced online training can help you understand and build the FAIR Principles into your work.
• Learn more about UCL’s current Research Data Policy

Practise open publishing

The goal of Open Access is to make all research material openly available online without restriction, to all readers, free from the barriers imposed by subscriptions. Open access is now required by many research funders and for the REF but it also has its own intrinsic benefits such as more exposure for your work, more citations, broader reach and wider readership worldwide.

• Find out about the different routes to Open Access and how you can get funding and support to do the same for your research
• Search for Open Access Research online

Get searching

Refine your literature searching skills for reliable, relevant and comprehensive results. Whether you are searching for references to inform your research, as background reading, to scope your research topic, for a literature review or a systematic review, a robust search strategy is essential to ensure you find all the relevant research without having to wade through excessive irrelevant results. Our support for literature searching includes a range of options to support you at every stage of your research:

• Introductory guides and self-paced online tutorials to help you refresh your knowledge of the essential skills.
• Advanced online guidance and support, including our guide to searching for systematic reviews.
• Live online sessions covering essentials of literature searching skills and searching particular online databases.

Organise your references

Get the most out of reference management software such as EndNote, Mendeley or Zotero, which enable you to gather and organise references and full text documents relevant to your research and to insert references in a Word document automatically, generating a reference list in the citation style of your choice. We provide support in using EndNote, Mendeley and Zotero to help you use the software more effectively and to troubleshoot your queries:

• Live online and face to face training sessions to get you started and for questions and answers.
• Online guides including suggested activities to get to know the software and frequently asked questions to help resolve your queries.

Understand bibliometrics

Bibliometrics is concerned with the analysis of research based on citation counts and patterns. The individual measures used are also commonly referred to as bibliometrics, or citation metrics.

• Our bibliometrics guidance to UCL authors and colleagues demonstrates the best way to approach bibliometric measures, and how to interpret and use the findings in relation to UCL’s bibliometrics policy and the wider context.
• Explore further with our live online bibliometrics training sessions and self-paced online tutorials.

 

New for the academic year 2021-22 the Office for Open Science and Scholarship is organising a monthly series of talks, showcases and training sessions across as many of the eight pillars as we can fit in for UCL colleagues and students at all levels.

All of the teams will be teaching their usual classes, keep watching your usual sources of training plus here and on Twitter for those, but these introductory sessions are intended to give a general overview of each subject area for a general audience with plenty of opportunities for discussion and questions. These introductory sessions will also be supplemented with ad hoc events throughout the year.

• November
  Departmental UKRI Briefings – contact catherine.sharp@ucl.ac.uk to arrange a briefing for your team
• December
  Introduction to the Office for Open Science & Scholarship – December 15th 2-3pm – Postponed, please express interest below
• January 22
  Introduction to responsible metrics – January 27th 2-3pm – Online
• February
  Introduction to Research Data Management – February 2nd 10-11am – Online
• March
  Getting started with the RDR – Friday 4th Mar 10-11am – Online
• April
  Open Science Conference (Dates TBC)
• May
  Citizen Science project showcase (Details & Dates TBC)
• June
  Citizen Science, Public Engagement & Research Impact (Dates TBC)
• July
  ORCiD, DOI and beyond – Introduction to Persistent identifiers (Dates TBC)

If you are interested in any of the sessions above then please complete the MS form and the organisers will get back to you with calendar details and joining instructions for planned sessions. Any sessions without firm dates, we will contact you as soon as details are confirmed.

On Wednesday 16th December the UCL Office for Open Science and Scholarship hosted a webinar focussing on the forthcoming Wellcome Trust Open Access policy, with particular reference to DORA, as well as how we are making progress towards fully being able to meet its terms.

Our first speaker was David Carr from the Wellcome trust who talked about the development of the Open Access focussed teams inside Wellcome Trust before outlining the new policy in full and describing in detail the elements which are distinct from the previous policy as shown in the image below.

David then moved on to describing the background to Wellcome’s commitment to responsible research evaluation, and the decision to include DORA in the new policy. He also described the feedback and redrafting process that it went through thanks to the feedback from the community.

Following on from David, we had a talk from Dr Ralitsa Madsen, who shared her experiences as a junior researcher around the issue of research evaluation and especially its relationship with transparency and Open Science.  She has worked with Chris Chambers of the UKRN to develop a policy template for funders to try and encourage more adoption, but also make it easier for them to adopt, by providing a ready-made solution!

We then turned to the Library Services contingent of the webinar speakers, starting out with Dr Paul Ayris, Pro-Vice-Provost for Library Services and the Office for Open Science and Scholarship. Paul walked us through the development of the UCL responsible metrics policy and the ways that it is being implemented in HR, recruitment and promotions processes.

Catherine Sharp, Head of Open Access Services followed up with a whistle-stop tour of the changes that have been made to the Open Access processes in order to support academic staff to meet the terms of the new policy, including numerous transformative arrangements with different publishers.

At the end of the webinar we had one remaining question which we put to David after the session:

What do ‘appropriate sanctions’ look like?

David’s response: There’s actually no change on this – the sanctions are actually already in place, and will remain as are when the new policy comes into effect in January.

Essentially we monitor compliance at end-of-grant reporting stage and when researchers apply for new funding.  If a researcher has non-compliant papers, then we will not activate new grants or funding renewals until any non-compliant Wellcome papers have been made open access.  Where papers reported in an end of grant report are not compliant, we will also not accept any new grant applications from that researcher until this has been resolved.  In extreme cases, we also have the option to suspend funding to a whole organisation.  See: https://wellcome.org/grant-funding/guidance/open-access-guidance/complying-with-our-open-access-policy

The recording is available below and also on MediaCentral.

Here at the Office for Open Science and Scholarship we are pleased to announce another webinar!

This time we have David Carr from the Wellcome Trust leading this fascinating webinar based around their new Open Access policy which comes into force in January.

• David Carr: New Wellcome Trust OA policy outline and overview of changes
• Ralitsa Madsen: Proposal on DORA alignment across multiple funders
• Dr Paul Ayris & Catherine Sharp: How we are implementing the new policy at UCL

Join us on Wednesday 16th December at 12 noon for a lunchtime webinar, and plenty of time to ask all of your questions!

Sign up on Eventbrite to get your link to join the session

---

Full event description:

In their new Open Access policy, which will come into force in January 2021, the Wellcome Trust has introduced a requirement that organisations receiving funding must commit to the core principles set out by DORA.

In this webinar we will be hearing from David Carr, the Programme Manager for Open Research at the Wellcome Trust, who will be outlining this policy and sharing his thoughts on these changes. Following that we will be hearing from Dr Ralitsa Madsen, Sir Henry Wellcome Postdoctoral Fellow at the UCL Cancer Institute about her proposal to develop a protocol on DORA alignment that all research funders should follow. The final speakers of the session will be our own Dr Paul Ayris, Pro-Vice Provost for UCL Library Services and the Office for Open Science & Scholarship, and Catherine Sharp, Head of Open Access Services with an outline of how UCL is implementing both the DORA principles, and the Wellcome policy in general.

We will have plenty of time for discussion among our speakers and for them to answer your questions so please join us for an interesting session before the policy comes into force