By Robert Kirk, PhD Candidate

Reinforcement Learning (RL) could be used in a range of applications such as autonomous vehicles and robotics, but to fulfil this potential we need RL algorithms that can be used in the real world. Reality is varied, non-stationarity and open-ended, and to handle this algorithms need to be robust to variation in their environments, and be able to transfer and adapt to unseen (but similar) environments during their deployment. Generalisation in RL is all about creating methods that can tackle these difficulties, challenging a common assumption in previous RL research that the training and testing environments are identical.

However, reading RL generalisation research can be challenging, as there is confusion about the exact problem being tackled and the use of terminology. This is because generalisation refers to a class of problems within RL, rather than a specific problem. Claiming to improve “generalisation” without being more specific about the type of generalisation that is being improved is underspecified. In fact, it seems unlikely we could improve all types of generalisation with a single method, as an analogy of the No Free Lunch theorem may apply.

To address this confusion, we’ve written a survey and critical review of the field of generalisation in RL. We formally describe the class of generalisation problems and use this formalism to discuss benchmarks for generalisation as well as methods. Given the field is so young, there’s also a lot of future directions to explore, and we highlight ones we think are important.

To find out more, check out the extended blogpost here!

Happy New Year to one and all from all of us at the UKRI CDT in Foundational AI.

With each new year comes new hopes for the future and chance to reflect on the events of the past. The UCL Centre for Artificial Intelligence has hosted a number of events this past year focusing on equality, diversity and inclusion.  We began 2021 with LGBTQ+ in AI, followed by Women in AI, Black in AI and culminating with Disabled in Ai, a day before National Disability Awareness Day in November.

Due to covid and closures, it has taken a little longer than I had hoped to finalise the videos for our last event, which was titled WeThe15 in AI, and done in collaboration with The Global Disability Innovation Hub.

Having had the opportunity to held organise a couple of these important events I have valued the privilege of learning new ideas, hearing personal journeys and gaining insights into how AI and our academic world can support those in under-represented communities.

One of the first things I noticed when organising the WeThe15 in AI event was just how wide a subject matter disability is. Whilst most people would assume that a disability is always physical, or noticeable, there are so many more individuals who are part of the disabled community who have hidden disabilities, but whose voices are of equal value and importance.

Another thing I noticed is just how many voices wanted to be heard. Usually when organising an event, we schedule in break times between sessions to help pad out and structure the day, however with this event I found myself having to do some creative timetabling and forsake breaks in order to get people included.  (Lesson for next time is to make this an all-day event and not just an afternoon affair).

Our event was pretty evenly divided between two areas of research – Neurodiversity and Research Innovation as well as presentations from individuals within the disabled community and how they navigate the world using AI and tech innovation.

Our first presenters were Prof Marie Schaer and Dr Thomas Maillart from the University of Geneva, presenting their current research on using AI image processing technology to support early diagnosis of autism in young children. Their research has shown that there is a short period in a child’s life where their learning gap develops due to late-diagnosis of autism (a neuro-diverse way of processing the world around us from sight, sound, sense and comprehension) and by providing an early diagnosis, intervention steps can be put in place to support the learning journey, ensuring that children stay on the same learning flight path and navigate the world of learning better.

Our second presenter was a young PhD candidate called Will Dudley, who is studying at our sister CDT at Imperial College London and is a member of the disabled community. Will discussed his journey to research via his lived experiences as someone who has to navigate the world with challenges not faced by most. His insights into how he has had to adapt to learn and find tools that support his needs shines a light on the topic of inclusion and design for disabled students and researchers.

Our third presenter was Luis Canto E Castro, from City Maas, an organisation that used AI technology to help wheelchair users navigate city landscapes. Luis, who is originally from South Africa, told us of his personal experiences as a wheelchair user moving to London and finding tools to help him know where he could, and could not, access. He started with a story about being invited to interview for a role and ending up having a quick 5 minute conversation on the pavement outside the building because there was no accessibility for him to be able to enter and be formally interviewed. Accessibility and inclusion in both design and implementation were again prominent themes in Luis’ talk.

Our first panel of the afternoon focused on Autism and AI and was chaired by Alice Renard, a UCL student who set up the UCL Autistic Society and who is neurodiverse. The panel included Thomas Maillart from our first presentation, Susie Chen, an autism advocate and fellow member of UCL Autistic Society, Luke Muschialli, whose has a close family member with autism, Dr Joe Mintz from the Institute of Education at UCL, whose current work is looking at ways that AI can help autistic children learn in schools and Dr Larissa Romualdo Suzuki, who is Head of Data and AI at Google Cloud and an autism advocate.

The panel discussed a variety of matters surrounding autism, including acceptance, the need for AI tech to support those on the spectrum and personal journeys of navigating the world of research and employment with ASD.

Our key note speaker for the event was Professor Cathy Holloway from the Global Disability Innovation Hub, who discussed Ai assisted innovative technology and how her work helps individuals within the disabled community, the majority of whom live in poverty and need access to assistive technology in order to survive and thrive.

Our second panel discussed AI and disability innovation and featured a variety of experts in their field. Chaired by Daniel Hajas from the Disability Innovation Hub, our panellists included Mr Klaus Höckner, CEO, Austrian Association for Blind and Visually Impaired, Mr Bernard Chiira, Director at Innovate Now, Co-Founder Inclusive Education Network, Ms Pamela Molina-Toledo, Executive Director, World Federation of the Deaf (WFD), Mr Saqib Shaikh, Tech Lead at Microsoft AI & Research, Dr Yulia Sarviro, Senior Project Manager, Smart Cities for All, at G3ict  and Ms Susan Scott-Parker OBE, Chief Executive, Business Disability

Again, what became really clear from the conversations that were had was just how vital it is to include disabled people at the very start of conversations and conception of ideas in order for technology to be truly assistive to their needs and not just design for design’s sake.

Our final presentation was by Professor Aldo Faisal from Imperial College London, who discussed AI for disabilities: from assistive technology to human augmentation.  Prof Faisal is part of the UKRI CDT in AI for Healthcare and presented work on the research that he and his team undertake to ensure that the robotics Ai work they create suits the needs of the disabled community they serve.

The entire afternoon was an incredibly humbling experience.  We felt truly honoured to have so many important voices speak and tell us of their journey and their research. Our EDI events were created to ensure under-represented communities were able to have a platform on which they could share their ideas and truths. To be able to have the opportunity to be a witness to this and to better understand the need for inclusion, from conception to inception of research, felt like a privilege.  Too often it seems that the disabled community has been the recipient of innovative ideas as a secondary outcome of work created by and for the abled community. Understanding their needs and creating technology by and for the disabled community, whose secondary outcome may or may not be useful for the abled community would be a more inclusive world. Knowing that so many great minds are working towards this outcome is a comforting thought. Together, we can build a better future.

If you are interested in watching this event, please do check out our WeThe15 in Ai playlist on our You Tube channel.