Full Record

New Search | Similar Records

Author
Title A spiking neural network of state transition probabilities in model-based reinforcement learning
URL
Publication Date
University/Publisher University of Waterloo
Abstract The development of the field of reinforcement learning was based on psychological studies of the instrumental conditioning of humans and other animals. Recently, reinforcement learning algorithms have been applied to neuroscience to help characterize neural activity and animal behaviour in instrumental conditioning tasks. A specific example is the hybrid learner developed to match human behaviour on a two-stage decision task. This hybrid learner is composed of a model-free and a model-based system. The model presented in this thesis is an implementation of that model-based system where the state transition probabilities and Q-value calculations use biologically plausible spiking neurons. Two variants of the model demonstrate the behaviour when the state transition probabilities are encoded in the network at the beginning of the task, and when these probabilities are learned over the course of the task. Various parameters that affect the behaviour of the model are explored, and ranges of these parameters that produce characteristically model-based behaviour are found. This work provides an important first step toward understanding how a model-based system in the human brain could be implemented, and how this system contributes to human behaviour.
Subjects/Keywords reinforcement learning; model-based reinforcement learning; spiking neural model; state transition probability; decision task
Language en
Country of Publication ca
Record ID handle:10012/12574
Repository waterloo
Date Retrieved
Date Indexed 2019-06-26

Sample Search Hits | Sample Images | Cited Works

…a novel model of how the state transition probabilities used in modelbased reinforcement learning (RL) may be represented and learned by a network of spiking neurons. In this chapter, we will provide a brief overview of the relevant RL…

…psychological and neuroscientific studies of human and animal reinforcement learning will be discussed. By the end of this chapter, it should be apparent that the computational processes underlying biological model-based RL are not as well understood as those of…

…model-free RL, thus producing one motivation for the research presented in this thesis. 1.1 Historical background The basic idea of reinforcement learning has been around for centuries, for as long as humans have tried to train other creatures to do…

…have borrowed ideas from instrumental conditioning to develop the field of reinforcement learning (RL). 1.2 Reinforcement learning Central to reinforcement learning (RL) are the terms “state”, “action”, and “reward”. A state is a…

…formula. SARSA stands for (s, a, r, s0 , a0 ) the current state and action, the reward, and the future state and action. 1.3 Reinforcement learning in neuroscience Aspects of reinforcement learning theory were adopted by psychologists and…

…behaviour on various reinforcement learning tasks to that produced by artificial agents using model-free RL algorithms, e.g., [7, 49, 61]. In most cases, and unsurprisingly, humans have been found to exhibit behaviours other than those predicted by…

…not as well understood how the goal-directed system may be realized in the brain, but it is proposed that it uses a form of what is known as model-based RL. 1.4 Model-based reinforcement learning Model-based RL learns an explicit model of the world…

…characterize how these two systems could work together from a computational perspective. In the case of humans, it is certain that we use a more sophisticated learning strategy than simple model-free reinforcement learning; we do not require extensive amounts…

.