Article

Multi-agent reinforcement learning for intrusion detection

Authors:

Daniel KudenkoAuthors Info & Claims

ALAMAS'05/ALAMAS'06/ALAMAS'07: Proceedings of the 5th, 6th and 7th European conference on Adaptive and learning agents and multi-agent systems: adaptation and multi-agent learning

Pages 211 - 223

Published: 01 January 2005 Publication History

Abstract

Intrusion Detection Systems (IDS) have been investigated for many years and the field has matured. Nevertheless, there are still important challenges, e.g., how an IDS can detect new and complex distributed attacks. To tackle these problems, we propose a distributed Reinforcement Learning (RL) approach in a hierarchical architecture of network sensor agents. Each network sensor agent learns to interpret local state observations, and communicates them to a central agent higher up in the agent hierarchy. These central agents, in turn, learn to send signals up the hierarchy, based on the signals that they receive. Finally, the agent at the top of the hierarchy learns when to signal an intrusion alarm. We evaluate our approach in an abstract network domain.

References

[1]

Barto, A.G., Mahadevan, S.: Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems 13(4), 341-379 (2003).

Digital Library

[2]

Barford, P., Jha, S., Yegneswaran, V.: Fusion and Filtering in Distributed Intrusion Detection Systems. In: Proceedings of the 42nd Annual Allerton Conference on Communication, Control and Computing (September 2004).

[3]

Bass, T.: Intrusion Detection Systems and Multisensor Data Fusion. Communications of the ACM 43(4), 99-105 (2000).

Digital Library

[4]

Chang, T.H., Kaelbling, L.: All learning is local: Multi-agent learning in global reward games. In: Advances in NIPS, vol. 14 (2004).

[5]

Elfwing, S., Uchibe, E., Doya, K., Christensen, H.I.: Multi-agent reinforcement learning: using macro actions to learn a mating task. In: IROS 2004. Intelligent Robots and Systems (2004).

[6]

Jennings, N., Sycara, K., Wooldridge, M.: A roadmap of agents research and development. Autonomous Agents and Multi-Agent Systems 1, 7-38 (1998) In: {12}.

Digital Library

[7]

Kapetanakis, S., Kudenko, D., Strens, M.: Learning to coordinate using commitment sequences in cooperative multi-agent systems. In: AISB 2003. Proceedings of the Third Symposium on Adaptive Agents and Multi-agent Systems, Society for the study of Artificial Intelligence and Simulation of Behaviour (2003).

[8]

Kostiadis, K., Hu, H.: KaBaGe-RL: Kanerva-based generalisation and reinforcement learning for possession football. In: IROS 2001. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2001).

[9]

Mirkovic, J., Reiher, P.: A taxonomy of DDoS attack and DDoS defense mechanisms. ACM SIGCOMM Computer Communication Review 34(2) (April 2004).

Digital Library

[10]

Moore, D., Shannon, C., Voelker, G.M., Savage, S.: Internet Quarantine: Requirements for Containing Self-Propagating Code. In: INFOCOM 2003. 22th Joint Conference of the IEEE Computer and Communications Societies, March 30-April 3, 2003, vol. 3, pp. 1901-1910 (2003).

[11]

Neumann, P.G., Porras, P.A.: Experience with EMERALD to DATE. In: 1st USENIX Workshop on Intrusion Detection and Network Monitoring, Santa Clara, California (April 11-12, 1999).

Digital Library

[12]

Panait, L., Luke, S.: Cooperative Multi-Agent Learning: The State of the Art. Autonomous Agents and Multi-Agent Systems 11(3), 387-434 (2005).

Digital Library

[13]

Porta, J., Celaya, E.: Reinforcement Learning for Agents with Many Sensors and Actuators Acting in Categorizable Environments. Journal of Artificial Intelligence Research 23, 79-122 (2005).

Digital Library

[14]

Powers, R., Shoham, Y.: New criteria and a new algorithm for learning in multiagent systems. In: Advances in Neural Information Processing Systems (forthcoming), Rubinstein, A.: Modeling Bounded Rationality. MIT Press, Washington (1998).

[15]

Sen, S., Weiss, G.: Learning in Multiagent Systems. In: Weiss, G. (ed.) Multiagent Systems, A Modern Approach to Distributed Artificial Intelligence, pp. 259-298. MIT Press, Cambridge (1999).

Digital Library

[16]

Siaterlis, C., Maglaris, B.: Towards multisensor data fusion for DoS detection. In: Proceedings of the 2004 ACM Symposium on Applied Computing, pp. 439-446 (2004).

Digital Library

[17]

Stone, P., Sutton, R.S., Singh, S.: Reinforcement Learning for 3 vs. 2 Keepaway. In: Stone, P., Balch, T., Kreatzschmarr, G. (eds.) RoboCup-2000: Robot Soccer World Cup IV, Springer, Berlin (2001).

Digital Library

[18]

Sutton, R., Barto, A.: Reinforcement Learning, An Introduction. MIT Press, Cambridge (1998).

Digital Library

[19]

Wasniowski, R.A.: Multisensor Agent Based Intrusion Detection. Transactions on Engineering, Computing and Technology 5, 110-113 (2005).

[20]

Yegneswaran, V., Barford, P., Jha, S.: Global Intrusion Detection in the DOMINO Overlay System. In: Proceedings of the Network and Distributed System Security Symposium (2004).

Cited By

Laskar MHuang JSmetana VStewart CPouw KAn AChan SLiu L(2021)Extending Isolation Forest for Anomaly Detection in Big Data via K-MeansACM Transactions on Cyber-Physical Systems10.1145/34609765:4(1-26)Online publication date: 22-Sep-2021
https://dl.acm.org/doi/10.1145/3460976
Otoum SKantarci BMouftah H(2021)A Comparative Study of AI-Based Intrusion Detection Techniques in Critical InfrastructuresACM Transactions on Internet Technology10.1145/340609321:4(1-22)Online publication date: 22-Jul-2021
https://dl.acm.org/doi/10.1145/3406093
Xie YWang YHe HXiang YYu SLiu X(2016)A General Collaborative Framework for Modeling and Perceiving Distributed Network BehaviorIEEE/ACM Transactions on Networking10.1109/TNET.2015.251260924:5(3162-3176)Online publication date: 1-Oct-2016
https://dl.acm.org/doi/10.1109/TNET.2015.2512609
Show More Cited By

Recommendations

Multi-Agent Reinforcement Learning for Intrusion Detection: A Case Study and Evaluation
MATES '08: Proceedings of the 6th German conference on Multiagent System Technologies

In this paper we propose a novel approach to train <em>Multi-Agent Reinforcement Learning</em>(MARL) agents to cooperate to detect intrusions in the form of normal and abnormal states in the network. We present an architecture of distributed sensor and ...
Multi-Agent Reinforcement Learning for Intrusion Detection: A case study and evaluation
Proceedings of the 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence

In this paper we propose a novel approach to train Multi-Agent Reinforcement Learning (MARL) agents to cooperate to detect intrusions in the form of normal and abnormal states in the network. We present an architecture of distributed sensor and decision ...
Hierarchical multi-agent reinforcement learning
AGENTS '01: Proceedings of the fifth international conference on Autonomous agents

In this paper we investigate the use of hierarchical reinforcement learning to speed up the acquisition of cooperative multi-agent tasks. We extend the MAXQ framework to the multi-agent case. Each agent uses the same MAXQ hierarchy to decompose a task ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ALAMAS'05/ALAMAS'06/ALAMAS'07: Proceedings of the 5th , 6th and 7th European conference on Adaptive and learning agents and multi-agent systems: adaptation and multi-agent learning

January 2005

255 pages

ISBN:3540779477

Editors:
Karl Tuyls
Maastricht University, The Netherlands
,
Ann Nowe
Vrije Universiteit Brussel, Belgium
,
Zahia Guessoum
University of Pierre and Marie Curie, France
,
Daniel Kudenko
The University of York, United Kingdom

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 January 2005

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Laskar MHuang JSmetana VStewart CPouw KAn AChan SLiu L(2021)Extending Isolation Forest for Anomaly Detection in Big Data via K-MeansACM Transactions on Cyber-Physical Systems10.1145/34609765:4(1-26)Online publication date: 22-Sep-2021
https://dl.acm.org/doi/10.1145/3460976
Otoum SKantarci BMouftah H(2021)A Comparative Study of AI-Based Intrusion Detection Techniques in Critical InfrastructuresACM Transactions on Internet Technology10.1145/340609321:4(1-22)Online publication date: 22-Jul-2021
https://dl.acm.org/doi/10.1145/3406093
Xie YWang YHe HXiang YYu SLiu X(2016)A General Collaborative Framework for Modeling and Perceiving Distributed Network BehaviorIEEE/ACM Transactions on Networking10.1109/TNET.2015.251260924:5(3162-3176)Online publication date: 1-Oct-2016
https://dl.acm.org/doi/10.1109/TNET.2015.2512609
Arshadi LJahangir A(2014)Benford's law behavior of Internet trafficJournal of Network and Computer Applications10.5555/2773807.277404140:C(194-205)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.5555/2773807.2774041
Baig Z(2012)ReviewJournal of Network and Computer Applications10.1016/j.jnca.2012.01.00635:3(1151-1161)Online publication date: 1-May-2012
https://dl.acm.org/doi/10.1016/j.jnca.2012.01.006

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents