Learning to coordinate using commitment sequences in cooperative multi-agent systems

S Kapetanakis; D Kudenko; M J A Strens

Learning to coordinate using commitment sequences in cooperative multi-agent systems

S Kapetanakis, D Kudenko, M J A Strens

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We report on an investigation of the learning of coordination in cooperative multi-agent systems. Specifically, we study solutions that are applicable to independent agents i.e. agents that do not observe one another's actions. In previous research [5] we have presented a reinforcement learning approach that converges to the optimal joint action even in scenarios with high miscoordination costs. However, this approach failed in fully stochastic environments. In this paper, we present a novel approach based on reward estimation with a shared action-selection protocol. The new technique is applicable in fully stochastic environments where mutual observation of actions is not possible. We demonstrate empirically that our approach causes the agents to converge almost always to the optimal joint action even in difficult stochastic scenarios with high miscoordination penalties.

Original language	English
Title of host publication	ADAPTIVE AGENTS AND MULTI-AGENT SYSTEMS II
Editors	D Kudenko, D Kazakov, E Alonso
Place of Publication	BERLIN
Publisher	Springer
Pages	106-118
Number of pages	13
ISBN (Print)	3-540-25260-6
Publication status	Published - 2005
Event	4th Symposium on Adaptive Agents and Multi-Agent Systems - Leeds Duration: 29 Mar 2004 → 30 Mar 2004

Conference

Conference	4th Symposium on Adaptive Agents and Multi-Agent Systems
City	Leeds
Period	29/03/04 → 30/03/04

Cite this

@inproceedings{c778f8c45fba4ac58951fafbd7c56394,

title = "Learning to coordinate using commitment sequences in cooperative multi-agent systems",

abstract = "We report on an investigation of the learning of coordination in cooperative multi-agent systems. Specifically, we study solutions that are applicable to independent agents i.e. agents that do not observe one another's actions. In previous research [5] we have presented a reinforcement learning approach that converges to the optimal joint action even in scenarios with high miscoordination costs. However, this approach failed in fully stochastic environments. In this paper, we present a novel approach based on reward estimation with a shared action-selection protocol. The new technique is applicable in fully stochastic environments where mutual observation of actions is not possible. We demonstrate empirically that our approach causes the agents to converge almost always to the optimal joint action even in difficult stochastic scenarios with high miscoordination penalties.",

author = "S Kapetanakis and D Kudenko and Strens, {M J A}",

year = "2005",

language = "English",

isbn = "3-540-25260-6",

pages = "106--118",

editor = "D Kudenko and D Kazakov and E Alonso",

booktitle = "ADAPTIVE AGENTS AND MULTI-AGENT SYSTEMS II",

publisher = "Springer",

address = "Germany",

note = "4th Symposium on Adaptive Agents and Multi-Agent Systems ; Conference date: 29-03-2004 Through 30-03-2004",

}

TY - GEN

T1 - Learning to coordinate using commitment sequences in cooperative multi-agent systems

AU - Kapetanakis, S

AU - Kudenko, D

AU - Strens, M J A

PY - 2005

Y1 - 2005

N2 - We report on an investigation of the learning of coordination in cooperative multi-agent systems. Specifically, we study solutions that are applicable to independent agents i.e. agents that do not observe one another's actions. In previous research [5] we have presented a reinforcement learning approach that converges to the optimal joint action even in scenarios with high miscoordination costs. However, this approach failed in fully stochastic environments. In this paper, we present a novel approach based on reward estimation with a shared action-selection protocol. The new technique is applicable in fully stochastic environments where mutual observation of actions is not possible. We demonstrate empirically that our approach causes the agents to converge almost always to the optimal joint action even in difficult stochastic scenarios with high miscoordination penalties.

AB - We report on an investigation of the learning of coordination in cooperative multi-agent systems. Specifically, we study solutions that are applicable to independent agents i.e. agents that do not observe one another's actions. In previous research [5] we have presented a reinforcement learning approach that converges to the optimal joint action even in scenarios with high miscoordination costs. However, this approach failed in fully stochastic environments. In this paper, we present a novel approach based on reward estimation with a shared action-selection protocol. The new technique is applicable in fully stochastic environments where mutual observation of actions is not possible. We demonstrate empirically that our approach causes the agents to converge almost always to the optimal joint action even in difficult stochastic scenarios with high miscoordination penalties.

M3 - Conference contribution

SN - 3-540-25260-6

SP - 106

EP - 118

BT - ADAPTIVE AGENTS AND MULTI-AGENT SYSTEMS II

A2 - Kudenko, D

A2 - Kazakov, D

A2 - Alonso, E

PB - Springer

CY - BERLIN

T2 - 4th Symposium on Adaptive Agents and Multi-Agent Systems

Y2 - 29 March 2004 through 30 March 2004

ER -