Learning Cooperative Behaviours in Adversarial Multi-agent Systems

Ni Wang; Gautham Das; Alan Gregory Millard

doi:10.1007/978-3-031-15908-4_15

Learning Cooperative Behaviours in Adversarial Multi-agent Systems

Ni Wang^*, Gautham Das, Alan Gregory Millard

^*Corresponding author for this work

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This work extends an existing virtual multi-agent platform called RoboSumo to create TripleSumo—a platform for investigating multi-agent cooperative behaviors in continuous action spaces, with physical contact in an adversarial environment. In this paper we investigate a scenario in which two agents, namely ‘Bug’ and ‘Ant’, must team up and push another agent ‘Spider’ out of the arena. To tackle this goal, the newly added agent ‘Bug’ is trained during an ongoing match between ‘Ant’ and ‘Spider’. ‘Bug’ must develop awareness of the other agents’ actions, infer the strategy of both sides, and eventually learn an action policy to cooperate. The reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward structure combining dense and sparse rewards. The cooperative behavior is quantitatively evaluated by the mean probability of winning the match and mean number of steps needed to win.

Original language	English
Title of host publication	Towards Autonomous Robotic Systems
Subtitle of host publication	23rd Annual Conference, TAROS 2022, Culham, UK, September 7–9, 2022, Proceedings
Publisher	Springer
Pages	179-189
Number of pages	11
ISBN (Electronic)	9783031159084
ISBN (Print)	9783031159077
DOIs	https://doi.org/10.1007/978-3-031-15908-4_15
Publication status	Published - 1 Sept 2022
Event	23rd Annual Conference Towards Autonomous Robotic Systems, TAROS 2022 - Culham, United Kingdom Duration: 7 Sept 2022 → 9 Sept 2022

Publication series

Name	Lecture Notes in Computer Science (LNCS)
Publisher	Springer
Volume	13546
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	23rd Annual Conference Towards Autonomous Robotic Systems, TAROS 2022
Country/Territory	United Kingdom
City	Culham
Period	7/09/22 → 9/09/22

Access to Document

10.1007/978-3-031-15908-4_15

Cite this

@inproceedings{2e7999ad2baf48538d9abe12d272e1ef,

title = "Learning Cooperative Behaviours in Adversarial Multi-agent Systems",

abstract = "This work extends an existing virtual multi-agent platform called RoboSumo to create TripleSumo—a platform for investigating multi-agent cooperative behaviors in continuous action spaces, with physical contact in an adversarial environment. In this paper we investigate a scenario in which two agents, namely {\textquoteleft}Bug{\textquoteright} and {\textquoteleft}Ant{\textquoteright}, must team up and push another agent {\textquoteleft}Spider{\textquoteright} out of the arena. To tackle this goal, the newly added agent {\textquoteleft}Bug{\textquoteright} is trained during an ongoing match between {\textquoteleft}Ant{\textquoteright} and {\textquoteleft}Spider{\textquoteright}. {\textquoteleft}Bug{\textquoteright} must develop awareness of the other agents{\textquoteright} actions, infer the strategy of both sides, and eventually learn an action policy to cooperate. The reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward structure combining dense and sparse rewards. The cooperative behavior is quantitatively evaluated by the mean probability of winning the match and mean number of steps needed to win.",

author = "Ni Wang and Gautham Das and Millard, {Alan Gregory}",

year = "2022",

month = sep,

day = "1",

doi = "10.1007/978-3-031-15908-4_15",

language = "English",

isbn = "9783031159077",

series = "Lecture Notes in Computer Science (LNCS)",

publisher = "Springer",

pages = "179--189",

booktitle = "Towards Autonomous Robotic Systems",

address = "Germany",

note = "23rd Annual Conference Towards Autonomous Robotic Systems, TAROS 2022 ; Conference date: 07-09-2022 Through 09-09-2022",

}

Wang, N, Das, G & Millard, AG 2022, Learning Cooperative Behaviours in Adversarial Multi-agent Systems. in Towards Autonomous Robotic Systems: 23rd Annual Conference, TAROS 2022, Culham, UK, September 7–9, 2022, Proceedings. Lecture Notes in Computer Science (LNCS), vol. 13546, Springer, pp. 179-189, 23rd Annual Conference Towards Autonomous Robotic Systems, TAROS 2022, Culham, United Kingdom, 7/09/22. https://doi.org/10.1007/978-3-031-15908-4_15

Learning Cooperative Behaviours in Adversarial Multi-agent Systems. / Wang, Ni; Das, Gautham; Millard, Alan Gregory.
Towards Autonomous Robotic Systems: 23rd Annual Conference, TAROS 2022, Culham, UK, September 7–9, 2022, Proceedings. Springer, 2022. p. 179-189 (Lecture Notes in Computer Science (LNCS); Vol. 13546).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Learning Cooperative Behaviours in Adversarial Multi-agent Systems

AU - Wang, Ni

AU - Das, Gautham

AU - Millard, Alan Gregory

PY - 2022/9/1

Y1 - 2022/9/1

N2 - This work extends an existing virtual multi-agent platform called RoboSumo to create TripleSumo—a platform for investigating multi-agent cooperative behaviors in continuous action spaces, with physical contact in an adversarial environment. In this paper we investigate a scenario in which two agents, namely ‘Bug’ and ‘Ant’, must team up and push another agent ‘Spider’ out of the arena. To tackle this goal, the newly added agent ‘Bug’ is trained during an ongoing match between ‘Ant’ and ‘Spider’. ‘Bug’ must develop awareness of the other agents’ actions, infer the strategy of both sides, and eventually learn an action policy to cooperate. The reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward structure combining dense and sparse rewards. The cooperative behavior is quantitatively evaluated by the mean probability of winning the match and mean number of steps needed to win.

AB - This work extends an existing virtual multi-agent platform called RoboSumo to create TripleSumo—a platform for investigating multi-agent cooperative behaviors in continuous action spaces, with physical contact in an adversarial environment. In this paper we investigate a scenario in which two agents, namely ‘Bug’ and ‘Ant’, must team up and push another agent ‘Spider’ out of the arena. To tackle this goal, the newly added agent ‘Bug’ is trained during an ongoing match between ‘Ant’ and ‘Spider’. ‘Bug’ must develop awareness of the other agents’ actions, infer the strategy of both sides, and eventually learn an action policy to cooperate. The reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward structure combining dense and sparse rewards. The cooperative behavior is quantitatively evaluated by the mean probability of winning the match and mean number of steps needed to win.

U2 - 10.1007/978-3-031-15908-4_15

DO - 10.1007/978-3-031-15908-4_15

M3 - Conference contribution

SN - 9783031159077

T3 - Lecture Notes in Computer Science (LNCS)

SP - 179

EP - 189

BT - Towards Autonomous Robotic Systems

PB - Springer

T2 - 23rd Annual Conference Towards Autonomous Robotic Systems, TAROS 2022

Y2 - 7 September 2022 through 9 September 2022

ER -