Thomas M. S. Smith

My research focuses on how artificially intelligent agents can learn to represent the world around them as predictive knowledge, and how they can use this knowledge to better complete tasks. In particularly my research addresses methods for agents to learn which behaviours can be completed successfully in their surroundings, and to predict the consequences of these behaviours. I consider these behaviours under the framework of options and hierarchical reinforcement learning.

Currently I am considering methods for learning and applying affordances. Affordance is best captured by the question “what can I do here?”. It describes the behaviours that a particular agent is able to complete given some environment features. For example, a button affords being pressed, or a door affords being opened. These affordances can speed up learning and planning for an agent.

I am part of the Bath Reinforcement Learning Lab, supervised by Professor Özgür Şimşek.

news

Jul 21, 2023	Attending ICML 2023 with Dan Beechey next week to present our work: Explaining Reinforcement Learning with Shapley Values

selected publications

2023

ICML

Explaining Reinforcement Learning with Shapley Values

Daniel Beechey, Thomas M. S. Smith, and Özgür Şimşek

In Proceedings of the 40th International Conference on Machine Learning, 23–29 jul 2023

Abs PDF

For reinforcement learning systems to be widely adopted, their users must understand and trust them. We present a theoretical analysis of explaining reinforcement learning using Shapley values, following a principled approach from game theory for identifying the contribution of individual players to the outcome of a cooperative game. We call this general framework Shapley Values for Explaining Reinforcement Learning (SVERL). Our analysis exposes the limitations of earlier uses of Shapley values in reinforcement learning. We then develop an approach that uses Shapley values to explain agent performance. In a variety of domains, SVERL produces meaningful explanations that match and supplement human intuition.