Parallel reinforcement learning with linear function approximation

Matthew Grounds; Daniel Kudenko

Parallel reinforcement learning with linear function approximation

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we investigate the use of parallelization in reinforcement learning (RL), with the goal of learning optimal policies for single-agent RL problems more quickly by using parallel hardware. Our approach is based on agents using the SARSA(lambda) algorithm, with value functions represented using, linear function approximators. In our proposed method, each agent learns independently in a separate simulation of the single-agent problem. The agents periodically exchange information extracted from the weights of their approximators, accelerating convergence towards the optimal policy. We develop three increasingly efficient versions of this approach to parallel RL, and present empirical results for an implementation of the methods on a Beowulf cluster.

Original language	English
Title of host publication	ADAPTIVE AGENTS AND MULTI-AGENT SYSTEMS
Editors	K Tuyls, A Nowe, Z Guessoum, D Kudenko
Place of Publication	BERLIN
Publisher	Springer
Pages	60-74
Number of pages	15
Volume	4865 LNAI
ISBN (Print)	978-3-540-77947-6
Publication status	Published - 2008

Cite this

@inproceedings{55f3ed828c6642e7b14f262c2f36ce94,

title = "Parallel reinforcement learning with linear function approximation",

abstract = "In this paper, we investigate the use of parallelization in reinforcement learning (RL), with the goal of learning optimal policies for single-agent RL problems more quickly by using parallel hardware. Our approach is based on agents using the SARSA(lambda) algorithm, with value functions represented using, linear function approximators. In our proposed method, each agent learns independently in a separate simulation of the single-agent problem. The agents periodically exchange information extracted from the weights of their approximators, accelerating convergence towards the optimal policy. We develop three increasingly efficient versions of this approach to parallel RL, and present empirical results for an implementation of the methods on a Beowulf cluster.",

author = "Matthew Grounds and Daniel Kudenko",

year = "2008",

language = "English",

isbn = "978-3-540-77947-6",

volume = "4865 LNAI",

pages = "60--74",

editor = "K Tuyls and A Nowe and Z Guessoum and D Kudenko",

booktitle = "ADAPTIVE AGENTS AND MULTI-AGENT SYSTEMS",

publisher = "Springer",

address = "Germany",

}

TY - GEN

T1 - Parallel reinforcement learning with linear function approximation

AU - Grounds, Matthew

AU - Kudenko, Daniel

PY - 2008

Y1 - 2008

N2 - In this paper, we investigate the use of parallelization in reinforcement learning (RL), with the goal of learning optimal policies for single-agent RL problems more quickly by using parallel hardware. Our approach is based on agents using the SARSA(lambda) algorithm, with value functions represented using, linear function approximators. In our proposed method, each agent learns independently in a separate simulation of the single-agent problem. The agents periodically exchange information extracted from the weights of their approximators, accelerating convergence towards the optimal policy. We develop three increasingly efficient versions of this approach to parallel RL, and present empirical results for an implementation of the methods on a Beowulf cluster.

AB - In this paper, we investigate the use of parallelization in reinforcement learning (RL), with the goal of learning optimal policies for single-agent RL problems more quickly by using parallel hardware. Our approach is based on agents using the SARSA(lambda) algorithm, with value functions represented using, linear function approximators. In our proposed method, each agent learns independently in a separate simulation of the single-agent problem. The agents periodically exchange information extracted from the weights of their approximators, accelerating convergence towards the optimal policy. We develop three increasingly efficient versions of this approach to parallel RL, and present empirical results for an implementation of the methods on a Beowulf cluster.

UR - http://www.scopus.com/inward/record.url?scp=49949106524&partnerID=8YFLogxK

M3 - Conference contribution

SN - 978-3-540-77947-6

VL - 4865 LNAI

SP - 60

EP - 74

BT - ADAPTIVE AGENTS AND MULTI-AGENT SYSTEMS

A2 - Tuyls, K

A2 - Nowe, A

A2 - Guessoum, Z

A2 - Kudenko, D

PB - Springer

CY - BERLIN

ER -

Parallel reinforcement learning with linear function approximation

Abstract

Other files and links

Cite this