Combining Reinforcement Learning with Symbolic Planning

Matthew Grounds; Daniel Kudenko

doi:10.1007/978-3-540-77949-0_6

Combining Reinforcement Learning with Symbolic Planning

Matthew Grounds, Daniel Kudenko

Computer Science

Research output: Contribution to conference › Paper › peer-review

Abstract

One of the major difficulties in applying Q-learning to real-world domains is the sharp increase in the number of learning steps required to converge towards an optimal policy as the size of the state space is increased. In this paper we propose a method, PLANQ-learning, that couples a Q-learner with a STRIPS planner. The planner shapes the reward function, and thus guides the Q-learner quickly to the optimal policy. We demonstrate empirically that this combination of high-level reasoning and low-level learning displays significant improvements in scaling-up behaviour as the state-space grows larger, compared to both standard Q-learning and hierarchical Q-learning methods.

Original language	Undefined/Unknown
Pages	75-86
DOIs	https://doi.org/10.1007/978-3-540-77949-0_6
Publication status	Published - 2007

Access to Document

10.1007/978-3-540-77949-0_6

http://dx.doi.org/10.1007/978-3-540-77949-0_6

Cite this

@conference{78a8cc0071794617805fa358ffd3c522,

title = "Combining Reinforcement Learning with Symbolic Planning",

abstract = "One of the major difficulties in applying Q-learning to real-world domains is the sharp increase in the number of learning steps required to converge towards an optimal policy as the size of the state space is increased. In this paper we propose a method, PLANQ-learning, that couples a Q-learner with a STRIPS planner. The planner shapes the reward function, and thus guides the Q-learner quickly to the optimal policy. We demonstrate empirically that this combination of high-level reasoning and low-level learning displays significant improvements in scaling-up behaviour as the state-space grows larger, compared to both standard Q-learning and hierarchical Q-learning methods. ",

author = "Matthew Grounds and Daniel Kudenko",

year = "2007",

doi = "10.1007/978-3-540-77949-0_6",

language = "Undefined/Unknown",

pages = "75--86",

}

TY - CONF

T1 - Combining Reinforcement Learning with Symbolic Planning

AU - Grounds, Matthew

AU - Kudenko, Daniel

PY - 2007

Y1 - 2007

N2 - One of the major difficulties in applying Q-learning to real-world domains is the sharp increase in the number of learning steps required to converge towards an optimal policy as the size of the state space is increased. In this paper we propose a method, PLANQ-learning, that couples a Q-learner with a STRIPS planner. The planner shapes the reward function, and thus guides the Q-learner quickly to the optimal policy. We demonstrate empirically that this combination of high-level reasoning and low-level learning displays significant improvements in scaling-up behaviour as the state-space grows larger, compared to both standard Q-learning and hierarchical Q-learning methods.

AB - One of the major difficulties in applying Q-learning to real-world domains is the sharp increase in the number of learning steps required to converge towards an optimal policy as the size of the state space is increased. In this paper we propose a method, PLANQ-learning, that couples a Q-learner with a STRIPS planner. The planner shapes the reward function, and thus guides the Q-learner quickly to the optimal policy. We demonstrate empirically that this combination of high-level reasoning and low-level learning displays significant improvements in scaling-up behaviour as the state-space grows larger, compared to both standard Q-learning and hierarchical Q-learning methods.

U2 - 10.1007/978-3-540-77949-0_6

DO - 10.1007/978-3-540-77949-0_6

M3 - Paper

SP - 75

EP - 86

ER -