Full Record

New Search | Similar Records

Title Action selection in modular reinforcement learning
Publication Date
Date Accessioned
Degree MSin Computer Sciences
Discipline/Department Computer Sciences
Degree Level masters
University/Publisher University of Texas – Austin
Abstract Modular reinforcement learning is an approach to resolve the curse of dimensionality problem in traditional reinforcement learning. We design and implement a modular reinforcement learning algorithm, which is based on three major components: Markov decision process decomposition, module training, and global action selection. We define and formalize module class and module instance concepts in decomposition step. Under our framework of decomposition, we train each modules efficiently using SARSA($\lambda$) algorithm. Then we design, implement, test, and compare three action selection algorithms based on different heuristics: Module Combination, Module Selection, and Module Voting. For last two algorithms, we propose a method to calculate module weights efficiently, by using standard deviation of Q-values of each module. We show that Module Combination and Module Voting algorithms produce satisfactory performance in our test domain.
Subjects/Keywords Modular reinforcement learning; Action selection; Module weight
Contributors Ballard, Dana H. (Dana Harry), 1946- (advisor)
Language en
Country of Publication us
Record ID handle:2152/25916
Repository texas
Date Retrieved
Date Indexed 2020-10-15
Grantor The University of Texas at Austin
Note [] text; [department] Computer Sciences;

Sample Search Hits | Sample Images

…RL problem with large state space. We propose to take a modular reinforcement learning approach, which we will discuss in Chapter 2. 4 Thesis Organization The rest of thesis is organized as follows: Chapter 2 summarizes related reinforcement

learning research to resolve curse of dimensionality problem, and proposes 3 research goals for the thesis. Chapter 3 introduces a test domain, and demonstrates our modular reinforcement learning algorithm. In Chapter 4, we present two alternative…

…This strategy is embedded in modular reinforcement learning and hierarchical reinforcement learning, two different but closely related approaches. Modular reinforcement learning [7, 10, 12, 20] decomposes original RL problem into modules. The…

…Evidences from neuroscience studies also support a modular model in human reinforcement learning [13]. Neuroimaging and lesion study results suggest that, human brain modularizes a complex learning problem to make it more computationally tractable…

…eater game [8]. These successful results suggest modular reinforcement learning might be a promising approach to curse of dimensionality problem. A close relative to modular reinforcement learning is hierarchical reinforcement learning [2…

…are shown to be supported by biological studies or be successful in several practical domains, they raise many new issues. The first problem of previously mentioned modular reinforcement learning algorithms is global action selection. Given multiple…

…global task, hence arbitrator needs to perform credit assignment to the modules when giving them feedbacks. On the contrary, we might choose to use a more bottom-up approach for modular reinforcement learning. The arbitrator should only define modules…

…making to select global action. Hence, the third goal of this thesis is to develop a bottom-up modular reinforcement learning architecture to reduce computational costs of the arbitrator. 7 Chapter 3 Modular Reinforcement Learning Algorithm 1 Test…