Full Record

New Search | Similar Records

 Author Zhang, Ruohan Title Action selection in modular reinforcement learning URL http://hdl.handle.net/2152/25916 Publication Date 2014 Date Accessioned 2014-09-16 20:04:19 Degree MSin Computer Sciences Discipline/Department Computer Sciences Degree Level masters University/Publisher University of Texas – Austin Abstract Modular reinforcement learning is an approach to resolve the curse of dimensionality problem in traditional reinforcement learning. We design and implement a modular reinforcement learning algorithm, which is based on three major components: Markov decision process decomposition, module training, and global action selection. We define and formalize module class and module instance concepts in decomposition step. Under our framework of decomposition, we train each modules efficiently using SARSA($\lambda$) algorithm. Then we design, implement, test, and compare three action selection algorithms based on different heuristics: Module Combination, Module Selection, and Module Voting. For last two algorithms, we propose a method to calculate module weights efficiently, by using standard deviation of Q-values of each module. We show that Module Combination and Module Voting algorithms produce satisfactory performance in our test domain. Subjects/Keywords Modular reinforcement learning; Action selection; Module weight Contributors Ballard, Dana H. (Dana Harry), 1946- (advisor) Language en Country of Publication us Record ID handle:2152/25916 Repository texas Date Retrieved 2020-10-09 Date Indexed 2020-10-15 Grantor The University of Texas at Austin Note [] text; [department] Computer Sciences;

Sample Search Hits | Sample Images

…RL problem with large state space. We propose to take a modular reinforcement learning approach, which we will discuss in Chapter 2. 4 Thesis Organization The rest of thesis is organized as follows: Chapter 2 summarizes related reinforcement

learning research to resolve curse of dimensionality problem, and proposes 3 research goals for the thesis. Chapter 3 introduces a test domain, and demonstrates our modular reinforcement learning algorithm. In Chapter 4, we present two alternative…

…This strategy is embedded in modular reinforcement learning and hierarchical reinforcement learning, two different but closely related approaches. Modular reinforcement learning [7, 10, 12, 20] decomposes original RL problem into modules. The…

…Evidences from neuroscience studies also support a modular model in human reinforcement learning [13]. Neuroimaging and lesion study results suggest that, human brain modularizes a complex learning problem to make it more computationally tractable…

…eater game [8]. These successful results suggest modular reinforcement learning might be a promising approach to curse of dimensionality problem. A close relative to modular reinforcement learning is hierarchical reinforcement learning [2…

…are shown to be supported by biological studies or be successful in several practical domains, they raise many new issues. The first problem of previously mentioned modular reinforcement learning algorithms is global action selection. Given multiple…

…global task, hence arbitrator needs to perform credit assignment to the modules when giving them feedbacks. On the contrary, we might choose to use a more bottom-up approach for modular reinforcement learning. The arbitrator should only define modules…

…making to select global action. Hence, the third goal of this thesis is to develop a bottom-up modular reinforcement learning architecture to reduce computational costs of the arbitrator. 7 Chapter 3 Modular Reinforcement Learning Algorithm 1 Test…