Overcoming incorrect knowledge in plan-based reward shaping

Kyriakos Efthymiadis; Sam Devlin; Daniel Kudenko

doi:10.1017/S026988891500017X

Overcoming incorrect knowledge in plan-based reward shaping

Kyriakos Efthymiadis, Sam Devlin, Daniel Kudenko

Computer Science

Research output: Contribution to journal › Article › peer-review

Abstract

Reward shaping has been shown to significantly improve an agent's performance in reinforcement learning. Plan-based reward shaping is a successful approach in which a STRIPS plan is used in order to guide the agent to the optimal behaviour. However, if the provided knowledge is wrong, it has been shown the agent will take longer to learn the optimal policy. Previously, in some cases, it was better to ignore all prior knowledge despite it only being partially incorrect. This paper introduces a novel use of knowledge revision to overcome incorrect domain knowledge when provided to an agent receiving plan-based reward shaping. Empirical results show that an agent using this method can outperform the previous agent receiving plan-based reward shaping without knowledge revision.

Original language	English
Pages (from-to)	31-43
Number of pages	13
Journal	The Knowledge Engineering Review
Volume	31
Issue number	1
DOIs	https://doi.org/10.1017/S026988891500017X
Publication status	Published - 11 Feb 2016

Access to Document

10.1017/S026988891500017X

Cite this

@article{0a088f9b58b343cc8ec6f9a267de6680,

title = "Overcoming incorrect knowledge in plan-based reward shaping",

abstract = "Reward shaping has been shown to significantly improve an agent's performance in reinforcement learning. Plan-based reward shaping is a successful approach in which a STRIPS plan is used in order to guide the agent to the optimal behaviour. However, if the provided knowledge is wrong, it has been shown the agent will take longer to learn the optimal policy. Previously, in some cases, it was better to ignore all prior knowledge despite it only being partially incorrect. This paper introduces a novel use of knowledge revision to overcome incorrect domain knowledge when provided to an agent receiving plan-based reward shaping. Empirical results show that an agent using this method can outperform the previous agent receiving plan-based reward shaping without knowledge revision.",

author = "Kyriakos Efthymiadis and Sam Devlin and Daniel Kudenko",

year = "2016",

month = feb,

day = "11",

doi = "10.1017/S026988891500017X",

language = "English",

volume = "31",

pages = "31--43",

journal = "The Knowledge Engineering Review",

issn = "0269-8889",

publisher = "Cambridge University Press",

number = "1",

}

TY - JOUR

T1 - Overcoming incorrect knowledge in plan-based reward shaping

AU - Efthymiadis, Kyriakos

AU - Devlin, Sam

AU - Kudenko, Daniel

PY - 2016/2/11

Y1 - 2016/2/11

N2 - Reward shaping has been shown to significantly improve an agent's performance in reinforcement learning. Plan-based reward shaping is a successful approach in which a STRIPS plan is used in order to guide the agent to the optimal behaviour. However, if the provided knowledge is wrong, it has been shown the agent will take longer to learn the optimal policy. Previously, in some cases, it was better to ignore all prior knowledge despite it only being partially incorrect. This paper introduces a novel use of knowledge revision to overcome incorrect domain knowledge when provided to an agent receiving plan-based reward shaping. Empirical results show that an agent using this method can outperform the previous agent receiving plan-based reward shaping without knowledge revision.

AB - Reward shaping has been shown to significantly improve an agent's performance in reinforcement learning. Plan-based reward shaping is a successful approach in which a STRIPS plan is used in order to guide the agent to the optimal behaviour. However, if the provided knowledge is wrong, it has been shown the agent will take longer to learn the optimal policy. Previously, in some cases, it was better to ignore all prior knowledge despite it only being partially incorrect. This paper introduces a novel use of knowledge revision to overcome incorrect domain knowledge when provided to an agent receiving plan-based reward shaping. Empirical results show that an agent using this method can outperform the previous agent receiving plan-based reward shaping without knowledge revision.

UR - http://www.scopus.com/inward/record.url?scp=84958165165&partnerID=8YFLogxK

U2 - 10.1017/S026988891500017X

DO - 10.1017/S026988891500017X

M3 - Article

AN - SCOPUS:84958165165

SN - 0269-8889

VL - 31

SP - 31

EP - 43

JO - The Knowledge Engineering Review

JF - The Knowledge Engineering Review

IS - 1

ER -

Overcoming incorrect knowledge in plan-based reward shaping

Abstract

Access to Document

Other files and links

Cite this