X Close

UCL Department of Science, Technology, Engineering and Public Policy

Home

Applied in Focus. Global in Reach

Menu

Archive for the 'Digital Technology and Policy Laboratory' Category

Adversarial Attacks, Robustness and Generalization in Deep Reinforcement Learning

By Ezgi Korkmaz, on 20 December 2023

Reinforcement learning has achieved substantial progress on successfully completing tasks, from solving complex games to large language models (i.e. GPT-4) including many different fields from medical applications to self-driving vehicles and finance, by learning from raw high-dimensional data with the utilization of deep neural networks as function approximators.

The vulnerabilities of deep reinforcement learning policies against adversarial attacks have been demonstrated in prior studies [1,2,3,4]. However, a recent study takes these vulnerabilities one step further and introduces natural attacks (i.e. natural changes to the environment given that these changes are imperceptible) while providing a contradistinction between adversarial attacks and natural attacks. The instances of such changes include, but are not limited to creating a blur, introduction of compression artifacts, or perspective projection of the state observations at a level that humans cannot perceive the change.

Intriguingly, the results reported demonstrate that these natural attacks are at least equally, and often more imperceptible compared to adversarial attacks, while causing larger drop in policy performance. While these results carry significant concerns regarding artificial intelligence safety [5,6,7], they further raise questions on the model’s security. Note that the prior studies on adversarial attacks on deep reinforcement learning rely on the strong adversary assumption, in which the adversary has access to the policy’s perception system, training details of the policy (e.g. algorithm, neural network architecture, training dataset), and the ability to alter observations in real time with simultaneous modifications to the observation system of the policy with computationally demanding adversarial formulations. Thus, the fact that natural attacks described in [8] are black-box adversarial attacks, i.e. the adversary does not have access to the training details of the policy and the policy’s perception system to compute the adversarial perturbations, raises further questions on machine learning safety and responsible artificial intelligence.

Furthermore, the second part of the paper investigates the robustness of adversarially trained deep reinforcement learning policies (i.e. robust reinforcement learning) under natural attacks, and demonstrates that vanilla trained deep reinforcement learning policies are more robust than adversarially trained policies. While these results reveal further security concerns regarding the robust reinforcement learning algorithms, they further demonstrate that adversarially trained deep reinforcement learning policies cannot generalize at the same level as straightforward vanilla trained deep reinforcement learning algorithms.

This study overall, while providing a contradistinction between adversarial attacks and natural black-box attacks, further reveals the connection between generalization in reinforcement learning and the adversarial perspective.

Author’s Note: This blog post is based on the paper ‘Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness’ published in AAAI 2023.
References:
[1] Adversarial Attacks on Neural Network Policies, ICLR 2017.
[2] Investigating Vulnerabilities of Deep Neural Policies. Conference on Uncertainty in Artificial Intelligence (UAI).
[3] Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPs. AAAI Conference on Artificial Intelligence, 2022. [Paper Link]
[4] Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions. International Conference on Machine Learning, ICML 2023. [Paper Link]
[5] New York Times. Global Leaders Warn A.I. Could Cause ‘Catastrophic’ Harm, November 2023.
[6] The Washington Post. 17 fatalities, 736 crashes: The shocking toll of Tesla’s Autopilot, June 2023.
[7] The Guardian. UK, US, EU and China sign declaration of AI’s ‘catastrophic’ danger, November 2023.
[8] Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness, AAAI Conference on Artificial Intelligence, 2023. [Paper Link]

First RespondXR: Digital vulnerability of immersive training for first responders

By Niamh F Healy, on 16 February 2022

For the past few months, I have had the pleasure of working as a research assistant on the First Respond XR project. The pilot study, led by Dr Leonie Tanczer, Lecturer in International Security and Emerging Technologies at UCL STEaPP, has been funded by the SPRITE Hub and explores the digital vulnerabilities associated with using Extended Reality (XR) to train police officers in the UK.

XR is an umbrella term used in reference to different types of virtual reality technology: immersive, three-dimensional, computer-generated environments. Popular examples of XR include Oculus Rift, a fully-immersive VR gaming headset, or Pokémon Go, which superimposes Pokémon onto the user’s environment via their smartphone cameras, an example of augmented reality (AR).

As the application areas of this technology are manifold, our four-month-long pilot study (December 2021 – March 2022) has the ambition to map the social, ethical, technical, and legal risks associated with the use of XR technology in the police training context. Our team (Dr Leonie Tanczer, Professor David McIlhatton, Professor Jill Marshall, Dr Mark McGill, Dr Lena Podoletz, Marina Heilbrunn and Niamh Healy) is set together with human-computer interaction (HCI) researchers, legal experts, security academics, and criminology and policing specialists. The multidisciplinary nature of the team, encompassing social, legal and technical expertise, benefits the examination of this timely issue and aids a holistic analysis of XR systems in the policing context. To date, we have been conducting a literature review to identify existing use cases of XR for police training as well as applications in other first responders domains (i.e., health, military, fire service). Our legal team has also begun mapping the complex legal context surrounding police use of XR for training purposes.

In this blog, I share some of our social team’s initial findings, set out our next research steps, and explain how interested parties could get involved in our study.

(more…)

Tracking the spread of science with machine learning

By Basil Mahfouz, on 18 November 2021

On 3-5 November 2021, I joined research professionals from across the Network for Advancing and Evaluating the Societal Impact of Science (AESIS) to discuss state of the art methods for evaluating the impact of research. Participants showcased institutional best practices, stakeholder engagement strategies, as well as how to leverage emerging data sources.  In this blog, I reflect on the conversations initiated at the conference, drawing upon insights gained throughout my research at STEaPP.

Photo by Markus Spiske on Unsplash

(more…)

The Infinite Game of Disinformation

By Alex Shepherd, on 15 October 2020

Alex Shepherd (@palexshepherd) is a nationally recognised subject matter expert on disinformation. He has delivered talks on the subject at the University of Oxford and the University of Cambridge, and has actively engaged with representatives from the UK government’s Sub-Committee on Disinformation. He is currently a senior AI researcher at Oxford Brookes University and a Digital Technologies and Policy MPA candidate at UCL STEaPP. 

Disinformation is one of the most important issues we face today, not only due to the massive social impact and disruption it creates globally, but also due to its exceptionally robust nature. This blog post, inspired by the tweetstorm “Some thoughts on disinformation”, attempts to explain disinformation’s robustness through the lens of game theory and analysis of technology trends.

Man using tablet to view fake news website

The concept of infinite games and finite games was popularised by Simon Sinek in his book, The Infinite Game, and at a keynote speech he delivered at a New York Times event. The book was influenced, in part, by James P. Carse’s book Finite and Infinite Games, which in turn was influenced by basic game theory.

(more…)

COVID-19: IoT and Cybersecurity

By fredrikskippervold, on 27 August 2020

Fredrik Johan Skippervold is a UCL MPA Graduate within Digital Technologies and Policy 18/19. He holds a Bachelor of Law with Spanish and is currently a researcher in the PETRAS National Centre of Excellence for IoT Systems Cybersecurity.

Introduction

Over the past four months (April – July) my colleague Dr Catherine Wheller and I have been following the impacts of COVID-19 on cybersecurity and the Internet of Things (IoT) within the UK and beyond. The pandemic has inspired a range of IoT innovations to help stop the spread of the virus. We have written weekly landscape briefings (LB) that provide up to date information on the latest developments in this area. In this blog I will talk about how we set about collecting information and how we put together these reports, as well as highlight some of the major developments which include discussions surrounding privacy and ethics. To note, a final summary briefing will be posted alongside this blogpost. The summary, which can be found here, includes a detailed timeline of events, provides an overview of how IoT devices are helping to stop the spread of the virus (UK and globally) and presents discussions around so-called ‘immunity passports’.

Cybersecurity

(more…)