PAC-MDP learning with knowledge-based admissible models

TY - GEN

T1 - PAC-MDP learning with knowledge-based admissible models

AU - Grzes, Marek

AU - Kudenko, Daniel

PY - 2010

Y1 - 2010

N2 - PAC-MDP algorithms approach the exploration-exploitation problem of reinforcement learning agents in an effective way which guarantees that with high probability, the algorithm performs near optimally for all but a polynomial number of steps. The performance of these algorithms can be further improved by incorporating domain knowledge to guide their learning process. In this paper we propose a framework to use partial knowledge about effects of actions in a theoretically well-founded way. Empirical evaluation shows that our proposed method is more efficient than reward shaping which represents an alternative approach to incorporate background knowledge. Our solution is also very competitive when compared with the Bayesian Exploration Bonus (BEB) algorithm. BEB is not PAC-MDP, however it can exploit domain knowledge via informative priors. We show how to use the same kind of knowledge in the PAC-MDP framework in a way which preserves all theoretical guarantees of PAC-MDP learning.

AB - PAC-MDP algorithms approach the exploration-exploitation problem of reinforcement learning agents in an effective way which guarantees that with high probability, the algorithm performs near optimally for all but a polynomial number of steps. The performance of these algorithms can be further improved by incorporating domain knowledge to guide their learning process. In this paper we propose a framework to use partial knowledge about effects of actions in a theoretically well-founded way. Empirical evaluation shows that our proposed method is more efficient than reward shaping which represents an alternative approach to incorporate background knowledge. Our solution is also very competitive when compared with the Bayesian Exploration Bonus (BEB) algorithm. BEB is not PAC-MDP, however it can exploit domain knowledge via informative priors. We show how to use the same kind of knowledge in the PAC-MDP framework in a way which preserves all theoretical guarantees of PAC-MDP learning.

KW - domain knowledge

KW - heuristics

KW - reinforcement learning

M3 - Conference contribution

SN - 978-0-9826571-1-9

SP - 349

EP - 356

BT - Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2010)

PB - International Foundation for Autonomous Agents and Multiagent Systems

CY - Richland, SC

T2 - 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2010)

Y2 - 10 May 2010 through 14 May 2010

ER -