Full Record

New Search | Similar Records

Author
Title Integrating reinforcement learning into a programming language
URL
Publication Date
Date Accessioned
Degree PhD
Discipline/Department Computer Science
Degree Level doctoral
University/Publisher Georgia Tech
Abstract Reinforcement learning is a promising solution to the intelligent agent problem, namely, given the state of the world, which action should an agent take to maximize goal attainment. However, reinforcement learning algorithms are slow to converge for larger state spaces and using reinforcement learning in agent programs requires detailed knowledge of reinforcement learning algorithms. One approach to solving the curse of dimensionality in reinforcement learning is decomposition. Modular reinforcement learning, as it is called in the literature, decomposes an agent into concurrently running reinforcement learning modules that each learn a ``selfish'' solution to a subset of the original problem. For example, a bunny agent might be decomposed into a module that avoids predators and a module that finds food. Current approaches to modular reinforcement learning support decomposition but, because the reward scales of the modules must be comparable, they are not composable  – a module written for one agent cannot be reused in another agent without modifying its reward function. This dissertation makes two contributions: (1) a command arbitration algorithm for modular reinforcement learning that enables composability by decoupling the reward scales of reinforcement learning modules, and (2) a Scala-embedded domain-specific language  – AFABL (A Friendly Adaptive Behavior Language)  – that integrates modular reinforcement learning in a way that allows programmers to use reinforcement learning without knowing much about reinforcement learning algorithms. We empirically demonstrate the reward comparability problem and show that our command arbitration algorithm solves it, and we present the results of a study in which programmers used AFABL and traditional programming to write a simple agent and adapt it to a new domain, demonstrating the promise of language-integrated reinforcement learning for practical agent software engineering.
Subjects/Keywords Machine learning; Reinforcement learning; Modular reinforcement learning; Programming languages; Domain specific languages; Software engineering; Artificial intelligence; Intelligent agents
Contributors Isbell, Charles L. (advisor); Bodner, Douglas (committee member); Riedl, Mark (committee member); Rugaber, Spencer (committee member); Thomaz, Andrea (committee member)
Language en
Country of Publication us
Record ID handle:1853/58683
Repository gatech
Date Indexed 2020-05-13
Issued Date 2017-06-26 00:00:00
Note [degree] Ph.D.;

Sample Search Hits | Sample Images | Cited Works

Reinforcement Learning . . . 20 2.2.3 Modular Reinforcement Learning . . . . . . . . . . . . . . . . 21 Robust Command Arbitration for Modular Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . 25 Modular reinforcement learning

…and using reinforcement learning in agent programs requires detailed knowledge of reinforcement learning algorithms. One approach to solving the curse of dimensionality in reinforcement learning is decomposition. Modular reinforcement learning, as it…

…avoids predators and a module that finds food. Current approaches to modular reinforcement learning support decomposition but, because the reward scales of the modules must be comparable, they are not composable – a module written for one agent cannot be…

…reused in another agent without modifying its reward function. This dissertation makes two contributions: (1) a command arbitration algorithm for modular reinforcement learning that enables composability by decoupling the reward scales of…

reinforcement learning modules, and (2) a Scala-embedded domainspecific language – AFABL (A Friendly Adaptive Behavior Language) – that integrates modular reinforcement learning in a way that allows programmers to use reinforcement learning

…action sequences into subsequences. Another kind of adaptive programming – known as modular reinforcement learning (MRL) – is based on concurrent problem decomposition in which an agent may take only one action at a time but must pursue several…

…needs of practical software engineering for reuse and composability inspires a new AI algorithm for modular reinforcement learning. Integrating this new formulation of modular reinforcement learning and associated algorithms into a programming language…

…enables a new kind of software engineering: modular adaptive agent programming. In particular, this work makes the following contributions: • We explain a problem with the current state of the art in modular reinforcement learning, namely, that performance…

.