X Close

UCL Department of Science, Technology, Engineering and Public Policy

Home

Applied in Focus. Global in Reach

Menu

Archive for the 'Education' Category

Machine Learning for Unlocking the Policy Impact of Transdisciplinary Research

By Basil Mahfouz, on 8 July 2024

As the 31st International Conference on Transdisciplinary Engineering 2024 kicks off at UCL East, over 100 engineers from around the world—including the United Kingdom, Brazil, Mexico, United States, China, Japan, Sweden, Singapore, and others—converge in London to explore how transdisciplinary engineering can drive social change and improve the world. This conference provides a platform for discussing the crucial role of engineering and science in addressing societal challenges through innovative, interdisciplinary approaches.

We already know that transdisciplinary research teams comprising engineers and others, tend to produce research that is more likely to have policy and commercial impact. Yet, with over 334 categorized research fields, there are more than 35 billion possible combinations for interdisciplinary work for teams of up to 5 researchers. Depending on the combination of researcher capabilities, some interdisciplinary teams may be better suited for disruptive science, developing patents, or informing policy. But which combinations of fields lead to which type of impact?

Supported by Elsevier and working with the Growth Lab at Harvard Kennedy School, we are applying complexity methods and machine learning on bibliometric data to understand which combination of researcher capabilities leads to high-impact research. For this blog, we’ll discuss our work within the context of the impact of interdisciplinary climate research on public policy.

To calculate interdisciplinarity, we determine the capabilities of authors based on their publication history in different fields. Each author is represented by a vector indicating the number of times they have published in each field. These author vectors are then used to calculate the disciplinary diversity (DDA) of each paper, reflecting the combined expertise and capabilities of the co-authoring team.

As a first step, we ran a series of statistical analyses and regressions to evaluate the relationship between a paper’s interdisciplinarity score and the number of policy citations it received. Preliminary results show that interdisciplinarity explains almost 15% of the variance in policy citations, making it the strongest predictor of policy impact we have identified so far. In fact, transdisciplinarity is found to be three times better at predicting policy citations than conventional metrics of research excellence, exceeding the combined effects of academic citations, journal impact factor, and author h-index.

We then aggregated our analysis at the topic level by calculating the average interdisciplinarity score of papers within each topic. The first observation is that climate change-related topics with high interdisciplinarity are less common than those with low diversity. The distribution of these topics is shown in Figure 1.

Fig 1: Distribution of paper interdisciplinarity in climate research

The second observation is that topics with higher ratios of policy citations have nearly double the average interdisciplinarity score compared to topics with relatively low policy citations. Figure 2 illustrates the difference in interdisciplinary distribution between the low and high policy relevance groups.

Figure 2: Interdisciplinary Distribution by Policy Relevance

Finally, using the paper vectors, we are developing a machine learning model to understand which combinations of author capabilities and team dynamics lead to high policy impact. The preliminary model has already found that for climate change-related research overall, teams involving researchers with expertise in economics, meteorology & atmospheric sciences, general & internal medicine, ecology, and horticulture tend to be associated with higher policy impact.

Moving forward, we are refining the model to predict the optimal team dynamics for high impact within specific policy topics. We are expanding the training data to include additional metrics and features, such as academic seniority, international collaboration, and research text. Furthermore, we will incorporate the distances between research fields to study the effects of deep interdisciplinarity, bringing together researchers from relatively less connected fields, on social impact.

As TE2024 brings together engineers and researchers from around the globe, this work exemplifies the potential of interdisciplinary collaboration in addressing global challenges and driving social change through innovative, data-driven approaches. Equipped with machine learning tools like this, researchers can develop targeted strategies to form data-informed transdisciplinary teams, optimized for maximum societal benefit and impact.

UCL’s MPA Students Host Second Summit on Sustainability

By leonie.dunn, on 16 May 2024

On 26 April, a cross-faculty student committee representing the MPA degrees from the UCL Institute for Innovation and Public Purpose (IIPP), UCL Department of Science, Technology, Engineering, and Public Policy (STEaPP), and UCL Department of Political Science (DoPS) hosted the second annual Summit on Sustainability.

MPA students The Summit brought together students, academics, and professionals for an interactive and collaborative learning experience as well as engaging discussions about ongoing action in sustainability. This year’s theme, the Power and Politics of Sustainability Transitions, aimed to navigate the increasingly complex world of activism, policy, and conflict surrounding sustainable transitions and solutions.

Collaboration with the UCL Climate Action Unit opened the Summit

Dr. Kris de Meyer from the UCL Climate Action Unit started the event off with a talk about climate change and the uncertainty of the future. He demonstrated how we are doing what we can because it’s what we know, but posed the question: “If we knew the solution, what might we do differently to get there?” The thought-provoking talk set the tone for the event and explored the different realities that can and do exist in sustainability.

Jon Alexander and Jane Davidson in fireside chat about collaboration

Co-founder of The New Citizenship Project and co-author of Citizens, Jon Alexander has worked to center the public at the heart of collaborative issues like climate change and economic insecurity – to treat people like Citizens not Consumers. He sat down with Jane Davidson, former Welsh Government minister and current chair of the Wales Net Zero 2035 Group, to discuss community participation in policymaking. Ms. Davidson delved into her history has a lawmaker in Wales, living through community collaboration for a better future. Her passion for sustainability led to the Wellbeing of Future Generations Act (2015), putting sustainability at the forefront of all government and public organization action.

Panel with Jon Alexander and Jane DavidsonPolicy Pathways Simulation adapted from in-field work for MPA participation

After lunch, the UCL Climate Action Unit once again took the stage to deliver a two-hour activity in which the participating students and staff joined a Climate Strategy Advisory Board to advise on a hypothetical green transition plan set out by the Prime Minister. Policy Pathways was created by the Unit as a virtual exercise for policymakers and politicians to collaborate and deliver different methods of sustainability transitions through financial regulation, taxation, and public subsidies. The simulation was adapted for the Summit with the goal of providing groups with differing perspectives the chance to discuss and debate amongst themselves, leading to somewhat differing strategies.

UCL Climate Action interactive workshopAfter the activity, Dr. de Meyer showed that, in the field, stakeholders struggled to utilise the existing policy tools to deliver effective and affordable green transitions. He brought the conversation back to his opening talk, wherein he stated that we can only do what we know, so we must expand that knowledge in order to expand the reality of sustainability.

Multi-profession panel explores navigating power and politics in the field of sustainability

Moderated by one of the student organizers, the three-person panel discussed different actors’ roles in change, and how power plays into the sustainability movement when it comes to justice and representation. Selina Newell, Director of Climate Action Implementation at C40, Fatou Jeng, founder of Clean Earth Gambia and Youth Climate Advisor to the UN Secretary General, and Asad Rehman, Executive Director of War on Want, unpacked the different levels of action, from individual movements to global affairs. Much of the conversation focused on equity being utmost important for sustainability movements and recognising economic inequality as a major point of conflict when it comes to global change. The cross-sector backgrounds of the panellists offered unique insight into the different powers that activists and policymakers have to enact change through sustainability transitions.

Multi-professional panelClare Farrell speaks on the failure of ‘sustainability’ and where to go from here

To round out the Summit, keynote speaker Clare Farrell, co-founder of Extinction Rebellion, delivered a talk titled Why Sustainability Isn’t Working Out. She spoke on her background in fashion and ethics, and how her work on the ground in activism led to the group’s message of civil rebellion for democratic change. Her thought-proving talk demonstrated the slow-moving and barrier-filled process of change through established political and economic institutions, and how the conventional perception of ‘sustainability’ has failed to come to fruition. Ms. Farrell guided the attendees to think about what must change now in order to achieve future goals in green transitions. She wrapped up the event by evoking hope in action now for change later.

Clare Farrell Extinction Rebellion as speakerAfter closing remarks, guests and attendees were invited to food and drinks for a two-hour networking social with open discussion and further questions.

Continuing cross-MPA collaboration and learning

Last year’s summit paved the way for the collaborative effort made this year. The three departments delivered learning in different contexts. Unifying those views for a cohesive and multi-perspective educational opportunity allowed the attendees and the committee to learn from both each other as well as the guests. Collaboration between the MPAs is imperative as we the students prepare to embark on our professional journeys as decision makers. Our ability to learn from each other must be fostered now so that it is not limited to these formal institutions. After coming together for this year’s Summit, we hope the relationship between the departments’ MPA programs continues to grow and furthers collaboration in the coming years.

Authors Note

Written by Erin Sebastian.

Erin along with the other organisers of the Sustainability Summit would like to give a special thanks to Kazuhiro Naito and Liam Orme for photography.

 

The importance of collaboration to advance digital health

By luis.lacerda, on 27 March 2024

Earlier this month the Government announced a £3billion+ package to update fragmented and outdated IT systems across the NHS and transform the use of data to ease administrative burdens. That same week, the Policy Impact Unit (PIU) hosted a roundtable on digital health in the UK, bringing together colleagues from across UCL (see co-authors) as well as visiting researchers from the FioCruz Oswaldo Cruz Foundation in Brazil.

FioCruz is a federal public research foundation working with academic autonomy under the Ministry of Health of Brazil which was responsible for coordinating the COVID-19 vaccination campaign. The Brazilian delegation were keen to hear about UK experiences on health digitisation, challenges and opportunities, as well as developing a deeper understanding of the context and evaluation of several commitments agreed under the Brazil-UK High-Level Strategic Dialogues from 2020, some of which focussed on health cooperation and were funded by the Official Development Assistance (ODA) [1].

The main challenges discussed in the meeting, in relation to the digitalisation of the NHS, were systems’ interoperability, training and workforce capacity. Although there has been a push towards the adoption of federated data platforms (FDP), which will sit across NHS trusts and integrated care systems allowing them to connect data they already hold in a secure and safe environment, these are still disjointed and connecting them relies on platform providers talking to each other, which often does not happen.

Common challenges: interoperability, training and workforce capacity

The adoption of new digital health approaches is also reliant on having trained healthcare professionals to understand the power of data and new technologies. Particularly in primary care and GPs it is essential to have digitally literate colleagues that can engage communities, be clear and transparent about how health data is used and input it correctly to build FDPs that can be further used for research and to invest on the health of the nation[2]. Programmes like the NHS “Developing healthcare workers’ confidence in artificial intelligence” and inclusive digital healthcare are important, because there is a risk that ambitions to digitise the NHS, which are well intended, could exacerbate existing health inequalities and exclude some groups.

Incidentally, there is still a lack of progress to de-identify General Practice data and address low levels of confidence in new technologies among diverse communities – such as highlighted in the Health and Social Care Committee’s recent evaluation. Trust can be undermined as is the societal buy-in needed to deliver on ambitions for a more digital NHS.

Opportunities and way forward: innovation in regulatory mechanisms

On the flipside, there is an opportunity to bring people in early on to discussions on how AI tools are being used in medical devices, and how to properly manage the balance of risk and benefits such technologies may bring. The recent launch of the UK Regulatory Science and Innovation Networks was discussed, as well as the launch of a MHRA roadmap to create a framework for medical devices in the UK. Patients, researchers and industry representatives being included in this process, and being clear about how data can be used for the purposes of research, poses a great opportunity to bring real impact to clinical practice in terms of diagnosis, treatment, and monitoring of diseases.

Including other global partners in this conversation is essential given the importance of sharing learnings in different contexts, but also given the increasingly important role of international recognition in the medical domain as a factor to evidence impact. Specifically for global issues such as AI and post-market surveillance, where it is very difficult for regulators to know how new tools will perform before they are deployed, there is now a chance to have new standards emerge to shape digital health strategies across countries. We hope that visits like this inspire colleagues to work collaboratively and look forward to hearing from FioCruz how their visit is supporting Brazilian policy decisions on the development of digital health strategies.

Authors Note

Written by Dr. Luís Lacerda, Policy Impact Unit and co-authored by Professor Amitava Banerjee, UCL Institute of Health Informatics, Professor Derek Hill, UCL Dept of Medical Physics & Biomedical Engineering and Professor Patty Kostkova, UCL Institute for Risk & Disaster Reduction.

References

[1] For a list of projects funded under the scheme, please visit https://devtracker.fcdo.gov.uk/

[2] A particular good example was the COVID-19 registry where data such as vaccination rates, long-covid reports were put together in the same place and from different countries.

What can we do to decrease the cost of advanced cancer therapies and make them available for all?

By luis.lacerda, on 9 February 2024

There are 3 million people living with cancer in the UK, predicted to rise to 4 million by 2030[1]. Different societal groups are affected differently, in particular ethnic minorities who experience poorer outcomes[2]. Health inequalities are complex and their root causes diverse, including the fact that some cancers are more prevalent in specific communities[3]. Advanced research on targeted and personalised treatments can therefore bring hope to improve outcomes in the future and to “close the gap” in the access to cancer care. But how can these be made more affordable and included in holistic government strategies to manage cancer care?

Illustration of two people, two pill bottles and two DNA strandsAt UCL, the Future Targeted Healthcare Manufacturing Hub (FTHM Hub), which brings together academics, manufacturers, and policymakers, has been addressing manufacturing, business, and regulatory challenges to ensure that new targeted biological medicines can be developed quickly and manufactured at a cost affordable to society. This includes innovative research on the manufacture of promising cancer therapies ranging from Chimeric antigen receptor T-cell (CAR-T) therapies through to targeted drug therapies such as antibody-drug conjugates and cancer vaccines. The Hub engages with and supports several clinical groups at UCL that develop advanced therapy medicinal products (ATMPs), some of which have been commercialised or are being translated into the clinic.

The FTHM Hub’s work also includes more fundamental research into optimising manufacturing by innovating processes and finding new ways of reducing production costs of these therapies. Examples of this activity include manufacturing autologous CAR-T therapy at the patient’s bedside or in an automated “GMP-in-a box” system[4], which can bring about benefits in terms of cost reductions, accelerating bench-to-bedside innovation, and mitigate risks that are generated by market shortages[5].

The Hub has worked closely with healthcare specialists and regulatory authorities to analyse how CAR-Ts and other high-cost therapies affect NHS England’s ability to resource other health services. It has conducted detailed supply chain economics analysis to identify key cost of goods drivers for CAR-T therapies, supply chain optimisation, and to assess the risk-reward trade-offs between centralised and distributed manufacture.

The recent agreement reached between the Department of Health and Social Care (DHSC), NHS England and the Association of the British Pharmaceutical Industry (ABPI) on a voluntary scheme for branded medicines pricing, access and growth is a welcomed programme to explore how industry and government can better work to support the delivery of new advanced treatments for cancer, but this is not enough.

Furthermore, and for this important work to continue, investment and support on advanced manufacturing is required to understand possible implementation challenges of novel options such GMP-in-a-box in clinical settings. The new UK’s life sciences manufacturing funding to build resilience for future health emergencies is a good opportunity to do this to expand on the FTHM Hub’s work and ensure every patient living with cancer will have accessibility of treatment irrespective of geographical location.

In addition, time and cost of travel to specialised centres can pose an economic burden to patients and carers due to disparities in cancer care. New centres will also need dedicated staff to help deliver advanced therapies and the FTHM Hub is also training a new generation of professionals to enable rollout of those to patients.

In the week that marks World Cancer Day, the FTHM Hub continues to develop important work to treat patients with cancer and it is our hope at the Policy Impact Unit that we can work towards imagining new futures together, close the care gap, and bring better outcomes for all of those living with cancer.

 

References

[1] https://www.macmillan.org.uk/dfsmedia/1a6f23537f7f4519bb0cf14c45b2a629/11424-10061/Macmillan%20statistics%20fact%20sheet%20February%202023

[2] Martins, T., Abel, G., Ukoumunne, O.C. et al. Ethnic inequalities in routes to diagnosis of cancer: a population-based UK cohort study. Br J Cancer 127, 863–871 (2022). https://doi.org/10.1038/s41416-022-01847-x

[3] Delon, C., Brown, K.F., Payne, N.W.S. et al. Differences in cancer incidence by broad ethnic group in England, 2013–2017. Br J Cancer 126, 1765–1773 (2022). https://doi.org/10.1038/s41416-022-01718-5

[4] Pereira Chilima, T. & S. Farid. 2019. ‘A roadmap to successful commercialization of autologous CAR T-cell products with centralized and bedside manufacture.’ Cell Gene Therapies VI 73. Comisel, R. 2022. Decisional Tools for Supply Chain Economics of Cell and Gene Therapy Products. Diss. UCL (University College London).

[5] Bicudo, E. & I. Brass. 2023, ‘Advanced therapies, hospital exemptions & marketing authorizations: the UK’s emerging regulatory framework for point-of-care manufacture’ Cell and Gene Therapy Insights 9(1), 101-120.

Adversarial Attacks, Robustness and Generalization in Deep Reinforcement Learning

By Ezgi Korkmaz, on 20 December 2023

Reinforcement learning has achieved substantial progress on successfully completing tasks, from solving complex games to large language models (i.e. GPT-4) including many different fields from medical applications to self-driving vehicles and finance, by learning from raw high-dimensional data with the utilization of deep neural networks as function approximators.

The vulnerabilities of deep reinforcement learning policies against adversarial attacks have been demonstrated in prior studies [1,2,3,4]. However, a recent study takes these vulnerabilities one step further and introduces natural attacks (i.e. natural changes to the environment given that these changes are imperceptible) while providing a contradistinction between adversarial attacks and natural attacks. The instances of such changes include, but are not limited to creating a blur, introduction of compression artifacts, or perspective projection of the state observations at a level that humans cannot perceive the change.

Intriguingly, the results reported demonstrate that these natural attacks are at least equally, and often more imperceptible compared to adversarial attacks, while causing larger drop in policy performance. While these results carry significant concerns regarding artificial intelligence safety [5,6,7], they further raise questions on the model’s security. Note that the prior studies on adversarial attacks on deep reinforcement learning rely on the strong adversary assumption, in which the adversary has access to the policy’s perception system, training details of the policy (e.g. algorithm, neural network architecture, training dataset), and the ability to alter observations in real time with simultaneous modifications to the observation system of the policy with computationally demanding adversarial formulations. Thus, the fact that natural attacks described in [8] are black-box adversarial attacks, i.e. the adversary does not have access to the training details of the policy and the policy’s perception system to compute the adversarial perturbations, raises further questions on machine learning safety and responsible artificial intelligence.

Furthermore, the second part of the paper investigates the robustness of adversarially trained deep reinforcement learning policies (i.e. robust reinforcement learning) under natural attacks, and demonstrates that vanilla trained deep reinforcement learning policies are more robust than adversarially trained policies. While these results reveal further security concerns regarding the robust reinforcement learning algorithms, they further demonstrate that adversarially trained deep reinforcement learning policies cannot generalize at the same level as straightforward vanilla trained deep reinforcement learning algorithms.

This study overall, while providing a contradistinction between adversarial attacks and natural black-box attacks, further reveals the connection between generalization in reinforcement learning and the adversarial perspective.

Author’s Note: This blog post is based on the paper ‘Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness’ published in AAAI 2023.
References:
[1] Adversarial Attacks on Neural Network Policies, ICLR 2017.
[2] Investigating Vulnerabilities of Deep Neural Policies. Conference on Uncertainty in Artificial Intelligence (UAI).
[3] Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPs. AAAI Conference on Artificial Intelligence, 2022. [Paper Link]
[4] Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions. International Conference on Machine Learning, ICML 2023. [Paper Link]
[5] New York Times. Global Leaders Warn A.I. Could Cause ‘Catastrophic’ Harm, November 2023.
[6] The Washington Post. 17 fatalities, 736 crashes: The shocking toll of Tesla’s Autopilot, June 2023.
[7] The Guardian. UK, US, EU and China sign declaration of AI’s ‘catastrophic’ danger, November 2023.
[8] Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness, AAAI Conference on Artificial Intelligence, 2023. [Paper Link]
[9] Understanding and Diagnosing Deep Reinforcement Learning. International Conference on Machine Learning, ICML 2024.