Guest post by Andrew Gray, Bibliometrics Support Officer

Altmetrics are the concept of “alternative metrics” – measuring the impact of research beyond scholarly literature. This covers a wide range of different things, ranging from social media discussions (e.g. Twitter or Facebook), mainstream news reporting, and grey literature such as policy documents. Understanding how research is being reported and discussed in these can help give us a broader understanding of the impact and reach of papers that we don’t see from looking at traditional scholarly citations.

UCL has a subscription to Altmetric, the primary commercial database for this information. It covers a broad range of materials. We also subscribe to a second source, which focuses purely on policy documents – Overton and can be a helpful complement.

There are several ways in which looking at altmetrics can give us information that wouldn’t otherwise be available. For example, we can see how different audiences outwith academia are responding to research, and we can look at what they’re saying to get an idea of the kind of response.

Some of the altmetric indicators (particularly Mendeley bookmarks) seem to have a close correlation with subsequent citations and can give us an early view of what citation figures may be like six months to a year in future.

Lastly, tracing policy citations through Altmetric or Overton can effectively demonstrate the wider research impact, for example, for use in a funding report or application.

Looking at activity

So what data can we see? Altmetric provides an aggregated “score” for each paper, indicating an overall activity level. While this isn’t a very exact measure, it lets us identify papers with high and low activity levels.

Looking over the past few years at UCL, the most obvious thing is that discussion of research is dominated by COVID-19. It accounts for thirteen out of the fifteen most heavily discussed UCL papers overall – by comparison, were we to look at pure citation counts, COVID papers account for none of UCL’s top fifteen overall, and only perhaps four out of the top fifteen from the past few years. This very striking difference highlights how altmetrics and citations can show different things.

The colour swatches on each show how the activity is broken down. For example, in this paper, we can see that most of the activity is from X/Twitter (light blue), with smaller contributions from Facebook (dark blue), news media (red) and blogs (yellow). Clicking through will let us drill down to see all the activity details.

Diving into data – day by day

One thing that surprised us with Altmetric is the sheer volume of data that they make available. Reports of 100,000+ papers can be downloaded, including DOIs and PubMed IDs, making it easy to link data to other sources such as RPS and InCites. This lets us do some analyses that wouldn’t be possible in other sources – but do tell us something unexpected.

For example, it gives us the exact date papers were published. Looking at around 50,000 UCL papers published in 2020-22, we find that the response differs depending on the day of the week – papers on Wednesday and Thursday are above-average, and papers on Tuesdays are below average.

In part, it is because some of the most prestigious publications have fixed publication days – most Nature papers are released on Wednesdays, for example. These journals have a large share of high-impact papers and an excellent publicising system.

The weekends are interesting. Not many papers come out on the weekends, but the ones that do, have a noticeable citation/bookmarking penalty compared to weekday ones, suggesting they are less impactful on average. And they make much less of a stir in the news media – a weekend paper is less than half as likely to get news coverage as a weekday one.

But social media has a sharp difference – Sunday papers get significantly more Twitter activity than Saturday ones. An intriguing mystery!

Using Altmetric at UCL

Altmetric and Overton are both available to any user at UCL. You simply need to log in to Altmetric using a UCL email, which will set up your user account. For Overton, you can browse the data without an individual account or set up an account to save searches and other functionalities.

We have integrated Altmetric with RPS, the central UCL publications database. Every two to four weeks, every paper in RPS since 2013 is exported, tagged with the UCL author(s) and associated departments, and uploaded into Altmetric.

This means that we can use the Altmetric dashboard to dig down into UCL outputs in some detail – we can ask it questions like “news stories in the last month referring to a piece of research published by someone in Chemistry”. It is also possible to save and circulate reports from the dashboard – this report shows the top 20 papers from Chemistry in 2023 by Altmetric Activity.

Similar functionality is not yet available for Overton, but if you would like to search for papers from a specific department, we would recommend generating a list of DOIs from InCites (or even from Altmetric itself!) and importing those as an advanced search.

We will be running introductory training sessions for both Altmetric and Overton in the coming term – please contact bibliometrics@ucl.ac.uk if you would be interested in attending these or booking a 1:1 meeting to go through the services.

Since the UCL’s Office for Open Science and Scholarship founding, the team has been gathering resources to support researchers, academic staff, students, and everyone interested in learning and developing their skills and understandings about Open Science and the transition towards more democratic models to produce and share knowledge in equitable and inclusive ways.

The fast growth of Open practices and their translation into renewed local, regional, and global policies made necessary systematic resource realignments. During the last months, we have committed to re-organise those assets based on the myriad of users and their backgrounds far beyond the academic field, creating our new user-friendly website, ‘Training and Support Resources for Research’. Organised in the following up-to-date categories:

• Advanced Research Computing: practical, hands-on training for various IT skills related to research, including high-performance computing, research software engineering and programming.

• Citizen Science: our brand-new site with information about what UCL is doing in Citizen Science, innovative projects, and UCL’s Citizen Science Academy. Keep an eye on the Open@UCL Blog, where we will soon expand and go deep into the world of Citizen Science!

• UCL Copyright advice: information and assistance by the UCL’s Library Services Copyright Team offers a wide range of copyright issues to UCL students and staff.

• Creating Accessible Content: a compendium of simple steps to make your content more accessible and provide a more inclusive experience for all.

• Doctoral Students Resources: a place for rigorous academic and non-academic creative researchers. Explore the resources and expand your skills to support your research, professional development and employability.

• Information Governance: this site is for all members of UCL who manage highly confidential research information, including principal and chief investigators, staff, students, senior managers, and even those who just supervise people who directly handle confidential information and support staff who do not have direct access to data.

• Bibliometrics support and training: information on training sessions at UCL and external training resources relating to bibliometrics.

• Open Access: designed to help UCL researchers understand how to make publications open access, meet open access requirements, use UCL’s Research Publications Service (RPS) and take advantage of open access funding.

• Research Data Management and Planning: from the initial planning of a project through to archiving and sharing, the research data management team advises the UCL community on managing research outputs – across the research data lifecycle – in line with UCL’s expectations and external funding agencies’ requirements.

• Research Funding Management: learn more about post-award processes through the online training course of the Fundamentals of Research Funding Management.

• Research Integrity: summary of training opportunities currently available for staff and students. This list is not exhaustive and is intended to provide guidance as to options available. It will also be updated so do re-visit this page.

• Research Transparency: research transparency covers how we ensure our research is responsible, reproducible, open and evidence-based.

If you scroll down the website, you will also find UCL’s Organisational Development training, Short Courses and some of UCL’s Communities and Forums that you can join to share your research, get advice and learn something new. We also collated an overview of the Research Support Teams!

Stay tuned to our news, events and training opportunities by subscribing to our mailing list,  following us on Twitter @UCLopenscience, or getting in touch with the Office for Open Science, and one of our teammates will answer as soon as we can!

A new exciting year is starting, and what better way to give the initial kick than celebrating the achievements and milestones of the multiple teams linked to the UCL Office for Open Science & Scholarship (OOSS). We are proud to see how the OOSS kept growing and consolidating itself inside UCL’s institutional culture, supporting academic staff, researchers and students.

One of last year’s highlights was undoubtedly the UCL Open Science Conference, reuniting people from all over the world in sessions discussing Citizen Science, Open data and code, Open and the Global South and more. You can still watch the recordings of day one and day two on our blog! And get ready to participate in the 2023 version in late April. You will hear about it soon on our pages and social media. We are working to make the event hybrid to facilitate participation across territories, do not miss your spot!

After various months of revisions and collaborative work, we published new Open Science Resources for 2022-2023. The first is the video ‘Open Science and Scholarship as part of UCL Research Culture’ and ‘Open Science – a practical guide for PhD students’.

Our office connects to several other teams inside UCL that make an exquisite blend of services, skills and expertise, and we want to celebrate their achievements and news.

The UCL Open Access team has grown their range of transformative publisher agreements and upgraded Research Publications Services (RPS). Users will notice a refreshed look and feel, differences to the Homepage layout, and a new menu structure and navigation. Check the step-by-step guide if you missed it!

UCL Press has proven the importance of open-access scholarly publishers, reaching six million downloads last May (and close to seven million now!), reaching 246 countries and territories, and publishing 272 titles since its launch in 2015.

The Bibliometrics team now is able to support Altmetric, which will be useful for anyone interested in public engagement or research impact, as well as individual researchers looking at the response to their work online. Altmetrics are “alternative metrics” – measuring the impact of research beyond scholarly literature. Helping to get a wider sense of the impact of papers that might otherwise be missed were we to focus on traditional academic citations.

Our Research and Data Management team upgraded their webpages, reviewed dozens of data management plans, and created brand new online Data Management Plan Templates with DMP online. Their services and the Bibliometrics team were both classified as excellent regarding the user’s experience of our online support service. We are proud of such a hard-working and supportive team!

During the second half of 2022, the OOSS gained two additional members: a Citizen Science Coordinator and a Support Officer. Both new team members are currently working with the Office Coordinator on ambitious projects that will see the light later this year, aiming to diversify the support and resources of our virtual office for wider audiences.

Undoubtedly, the diversity of professionals, backgrounds and interests made our small office inside Library, Culture, Collections & Open Science (LCCOS) a prosperous place to develop services, ideas and projects for wonderful audiences inside and outside our university.

Last year’s achievements were only possible due to the support of the university to embrace an open culture, thanks to the collaborative work between the teams, and always supporting each other and the office users. We will keep working together to democratise knowledge and keep UCL one of the Open Science & Scholarship leaders worldwide.

Guest post by Andrew Gray (Bibliometrics Support Officer)

What is it? 

Altmetrics are the concept of “alternative metrics” – measuring the impact of research beyond scholarly literature. It encompasses a wide range of activity in diverse sources  social media (eg twitter, blogs), news publications, and grey literature (eg policy documents). This can help to get a wider sense of the impact of papers that might otherwise be missed were we to focus just on traditional academic citations. 

The primary commercial database for these is Altmetric (https://altmetric.com) – UCL has just taken out a one-year subscription to this service. We hope it will be useful for anyone interested in public engagement or research impact, as well as individual researchers looking at the response to their own work. 

It is open to everyone at UCL by visiting https://www.altmetric.com/explorer/login and entering your UCL email address. It will then authenticate through the UCL single-sign-on system. 

How does it work? 

Altmetric tracks a range of individual sources looking for DOIs, links to papers, or free-text descriptions of articles. It then matches these to the underlying paper and produces an index of the mentions. Here we can see the range of responses to a climate-change study. 

You will also sometimes see this coloured “doughnut” on publisher or repository sites – clicking through will get you to this same page. 

The most interesting part of the service, however, is the dashboard. This aggregates the results from all individual papers, and we can then filter down by subject area, date, publication venue, etc., to produce a more specific analysis. It is also possible to search for keywords to see the change in activity around a specific topic – one like “artificial intelligence” tends to show a steady level of interest, while one like “gravitational waves” shows very dramatic spikes connected with major discoveries. 

What can we do with it? 

The dashboard has been integrated with UCL’s RPS service, so it has a dataset of UCL papers since 2013, each linked to the faculty/department of the authors. This means we can do the same types of analysis for just UCL papers – or just those from a specific department or a specific author. 

The search can also be tweaked to identify specific topics. Here we can see policy documents published in 2022 which cite a Bartlett paper. 

Policy documents are one of the key strengths of Altmetric – they can be used as evidence of wider impact, especially for the social sciences. While they are formal documents, and very distinct from more ephemeral news or social media mentions, they are not indexed in most citation databases and so this impact can often be hard to trace. 

Altmetric data can also be exported – any set of results can be exported so that we can do detailed offline analysis of sets of papers, or at the individual mentions that make up the score. This data includes identifiers such as DOIs and ISBNs, meaning it can be linked up to other datasets easily 

What next? 

We are very keen to get this tool in the hands of as many people at UCL as possible and find how it can be used most effectively. Please have a go and let us know what you think! 

UCL-specific training and guidance is currently under development, and will be published in September 2022. Until then, please feel free to get in touch with the team (bibliometrics@ucl.ac.uk) with queries or requests for assistance. We are happy to arrange training as well. 

The tool is currently provided with a static dataset drawn from RPS, covering papers published 1 January 2013 up to 12 August 2022. We are working with the providers to improve the integration so that it will include “live” data, refreshed from RPS every night; until then, we plan to make periodic updates so that publications are added on a rolling basis.