Distributed reinforcement learning for adaptive and robust network intrusion response

TY - JOUR

T1 - Distributed reinforcement learning for adaptive and robust network intrusion response

AU - Malialis, Kleanthis

AU - Devlin, Sam

AU - Kudenko, Daniel

PY - 2015/7/3

Y1 - 2015/7/3

N2 - Distributed denial of service (DDoS) attacks constitute a rapidly evolving threat in the current Internet. Multiagent Router Throttling is a novel approach to defend against DDoS attacks where multiple reinforcement learning agents are installed on a set of routers and learn to rate-limit or throttle traffic towards a victim server. The focus of this paper is on online learning and scalability. We propose an approach that incorporates task decomposition, team rewards and a form of reward shaping called difference rewards. One of the novel characteristics of the proposed system is that it provides a decentralised coordinated response to the DDoS problem, thus being resilient to DDoS attacks themselves. The proposed system learns remarkably fast, thus being suitable for online learning. Furthermore, its scalability is successfully demonstrated in experiments involving 1000 learning agents. We compare our approach against a baseline and a popular state-of-the-art throttling technique from the network security literature and show that the proposed approach is more effective, adaptive to sophisticated attack rate dynamics and robust to agent failures.

AB - Distributed denial of service (DDoS) attacks constitute a rapidly evolving threat in the current Internet. Multiagent Router Throttling is a novel approach to defend against DDoS attacks where multiple reinforcement learning agents are installed on a set of routers and learn to rate-limit or throttle traffic towards a victim server. The focus of this paper is on online learning and scalability. We propose an approach that incorporates task decomposition, team rewards and a form of reward shaping called difference rewards. One of the novel characteristics of the proposed system is that it provides a decentralised coordinated response to the DDoS problem, thus being resilient to DDoS attacks themselves. The proposed system learns remarkably fast, thus being suitable for online learning. Furthermore, its scalability is successfully demonstrated in experiments involving 1000 learning agents. We compare our approach against a baseline and a popular state-of-the-art throttling technique from the network security literature and show that the proposed approach is more effective, adaptive to sophisticated attack rate dynamics and robust to agent failures.

KW - DDoS attacks

KW - decentralised coordination

KW - distributed control

KW - network security

UR - http://www.scopus.com/inward/record.url?scp=84943198439&partnerID=8YFLogxK

U2 - 10.1080/09540091.2015.1031082

DO - 10.1080/09540091.2015.1031082

M3 - Article

AN - SCOPUS:84943198439

SN - 0954-0091

VL - 27

SP - 234

EP - 252

JO - Connection Science

JF - Connection Science

IS - 3

ER -