skip to main content
research-article

Automated Anomaly Detector Adaptation using Adaptive Threshold Tuning

Published:01 April 2013Publication History
Skip Abstract Section

Abstract

Real-time network- and host-based Anomaly Detection Systems (ADSs) transform a continuous stream of input data into meaningful and quantifiable anomaly scores. These scores are subsequently compared to a fixed detection threshold and classified as either benign or malicious. We argue that a real-time ADS’ input changes considerably over time and a fixed threshold value cannot guarantee good anomaly detection accuracy for such a time-varying input. In this article, we propose a simple and generic technique to adaptively tune the detection threshold of any ADS that works on threshold method. To this end, we first perform statistical and information-theoretic analysis of network- and host-based ADSs’ anomaly scores to reveal a consistent time correlation structure during benign activity periods. We model the observed correlation structure using Markov chains, which are in turn used in a stochastic target tracking framework to adapt an ADS’ detection threshold in accordance with real-time measurements. We also use statistical techniques to make the proposed algorithm resilient to sporadic changes and evasion attacks. In order to evaluate the proposed approach, we incorporate the proposed adaptive thresholding module into multiple ADSs and evaluate those ADSs over comprehensive and independently collected network and host attack datasets. We show that, while reducing the need of human threshold configuration, the proposed technique provides considerable and consistent accuracy improvements for all evaluated ADSs.

References

  1. Aggarwal, C. C., Han, J., Wang, J., and Yu, P. S. 2006. A framework for on-demand classification of evolving data streams. IEEE Trans. Knowl. Data Eng. 18, 5, 577--589. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Agosta, J. M., Wasser, C. D., Chandrashekar, J., and Livadas, C. 2007. An adaptive anomaly detector for worm detection. In Proceedings of the 2nd USENIX Workshop on Tackling Computer Systems Problems with Machine Learning Techniques. USENIX Association, Berkeley, CA, 3:1--3:6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Ali, M. Q., Khan, H., Sajjad, A., and Khayam, S. A. 2009. On achieving good operating points on an ROC plane using stochastic anomaly score prediction. In Proceedings of the 16th ACM Conference on Computer and Communications Security (CCS’09). ACM, New York, 314--323. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ali, S., Haq, I., Rizvi, S., Rasheed, N., Sarfraz, U., Khayam, S. A., and Mirza, F. 2010. On mitigating sampling-induced accuracy loss in traffic anomaly detection systems. ACM SIGCOMM Comput. Commun. Rev. 40, 4--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Arbor PeakFlow. Arbor networks’ peakflow product. http://www.arbornetworks.com/peakflowsp.Google ScholarGoogle Scholar
  6. Ashfaq, A. B., Joseph, M., Mumtaz, A., Ali, M. Q., Sajjad, A., and Khayam, S. A. 2008. A comparative evaluation of anomaly detectors under portscan attacks. In Proceedings of the 11th International Symposium on Recent Advances in Intrusion Detection (RAID’08). Springer-Verlag, Berlin, 351--371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bifet, A., Holmes, G., Pfahringer, B., Kirkby, R., and Gavaldà, R. 2009. New ensemble methods for evolving data streams. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’09). ACM, 139--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Cardenas, A. A., Baras, J. S., and Seamon, K. 2006. A framework for the evaluation of intrusion detection systems. In Proceedings of the IEEE Symposium on Security and Privacy (SP’06). IEEE. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chen, S., Wang, H., Zhou, S., and Yu, P. S. 2008. Stop chasing trends: Discovering high order models in evolving data. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering (ICDE’08). IEEE Computer Society, Los Alamitos, CA, 923--932. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Cisco Anomaly Guard. Cisco anomaly guard module homepage. www.cisco.com/en/US/products/ps6235/.Google ScholarGoogle Scholar
  11. Cretu-Ciocarlie, G. F., Stavrou, A., Locasto, M. E., and Stolfo, S. J. 2009. Adaptive anomaly detection via self-calibration and dynamic updating. In Proceedings of the 12th International Symposium on Recent Advances in Intrusion Detection (RAID’09). Springer-Verlag, Berlin, 41--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Crovella, M. E., and Bestavros, A. 1997. Self-similarity in world wide web traffic: Evidence and possible causes. IEEE/ACM Trans. Netw. 5, 835--846. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Forrest, S., Hofmeyr, S. A., Somayaji, A., and Longstaff, T. A. 1996. A sense of self for unix processes. In Proceedings of the 1996 IEEE Symposium on Security and Privacy (SP’96). IEEE Computer Society, Los Alamitos, CA, 120--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. FTP Brute Forcer. Ssh2ftpcrack ftp/ssh brute forcer. http://packetstormsecurity.org/files/98155/SSH2FTPCrack-FTP-SSH-Brute-Forcer.html.Google ScholarGoogle Scholar
  15. Gao, D., Reiter, M. K., and Song, D. 2005. Behavioral distance for intrusion detection. In Proceedings of the 8th International Symposium on Recent Advances in Intrusion Detection (RAID). 63--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Gao, J., Fan, W., and Han, J. 2007. On appropriate assumptions to mine data streams: Analysis and practice. In Proceedings of the 7th IEEE International Conference on Data Mining (ICDM’07). IEEE Computer Society, Los Alamitos, CA, 143--152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Gartner Report. 2003. Gartner information security hype cycle declares Intrusion detection systems a market failure money slated for intrusion detection should be invested in firewalls. http://www.gartner.com/about/press_releases/pr11june2003c.jsp.Google ScholarGoogle Scholar
  18. Gu, G., Fogla, P., Dagon, D., Lee, W., and Skoric, B. 2006. Towards an information-theoretic framework for analyzing intrusion detection systems. In Proceedings of the 11th European Symposium on Research in Computer Security (ESORICS’06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Gu, Y., McCullum, A., and Towsley, D. 2005. Detecting anomalies in network traffic using maximum entropy estimation. In Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement (IMC’05). USENIX Association, Berkeley, CA, 32--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Heap Profiler. HPROF: A heap/CPU profiling tool in j2se5.0. http://docs.oracle.com/javase/7/docs/technotes/samples/hprof.html.Google ScholarGoogle Scholar
  21. Hollinger, G., Djugash, J., and Singh, S. 2008. Tracking a moving target in cluttered environments with ranging radios: Extended results. Tech. rep. CMU-RI-TR-08-07, Robotics Institute, Carnegie Mellon University.Google ScholarGoogle Scholar
  22. Ide, T. and Kashima, H. 2004. Eigenspace-based anomaly detection in computer systems. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’04). ACM, New York, 440--449. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Jung, J., Paxson, V., Berger, A. W., and Balakrishnan, H. 2004. Fast portscan detection using sequential hypothesis testing. In Proceedings of the IEEE Symposium on Security and Privacy (SP’04). IEEE Computer Society, Los Alamitos, CA.Google ScholarGoogle Scholar
  24. Kang, D. K., Fuller, D., and Honavar, V. 2005. Learning classifiers for misuse and anomaly detection using a bag of system calls representation. In Proceedings of 6th IEEE Systems Man and Cybernetics Information Assurance Workshop (IAW’05).Google ScholarGoogle Scholar
  25. Kolter, J. Z., and Maloof, M. A. 2005. Using additive expert ensembles to cope with concept drift. In Proceedings of the 22nd International Conference on Machine Learning (ICML’05). ACM, New York, 449--456. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Lakhina, A., Crovella, M., and Diot, C. 2004. Diagnosing network-wide traffic anomalies. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM’04). ACM, New York, 219--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Lakhina, A., Crovella, M., and Diot, C. 2005. Mining anomalies using traffic feature distributions. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM’05). ACM, New York, 217--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. LBNL Dataset. LBNL/ICSI enterprise tracing project. http://www.icir.org/enterprise-tracing/Overview.html.Google ScholarGoogle Scholar
  29. Lippmann, R. P., Haines, J. W., Fried, D. J., Korba, J., and Das, K. 2000. The 1999 DARPA offline intrusion detection evaluation. Comput. Netw. 34, 579--595. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Mahoney, M. V. and Chan, P. K. 2001. PHAD: Packet header anomaly detection for indentifying hostile network traffic. Tech. rep. CS-2001-4, Florida Tech.Google ScholarGoogle Scholar
  31. Masud, M. M., Chen, Q., Khan, L., Aggarwal, C., Gao, J., Han, J., and Thuraisingham, B. 2010. Addressing concept-evolution in concept-drifting data streams. In Proceedings of the IEEE International Conference on Data Mining (ICDM’10) IEEE Computer Society, Los Alamitos, CA, 929--934. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Masud, M. M., Gao, J., Khan, L., Han, J., and Thuraisingham, B. M. 2011. Classification and novel class detection in concept-drifting data streams under time constraints. IEEE Trans. Knowl. Data Eng. 23, 6, 859--874. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Merhav, M., Gutman, M., and Ziv, J. 1989. On the estimation of the order of a markov chain and universal data compression. IEEE Trans. Inf. Theory 35, 5, 1014--1019. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Merhav, M., Gutman, M., and Ziv, J. 2010. Detecting intrusions through system call sequence and argument analysis. IEEE Trans. Depend. Secure Comput. 7, 4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. MIT Dataset. MIT lincoln laboratory, information systems technology. http://www.ll.mit.edu/mission/communications/ist/corpora/ideval/data/index.html.Google ScholarGoogle Scholar
  36. Moore, D., Shannon, C., and Claffy, K. 2002. Code-red: A case study on the spread and victims of an internet worm. In Code-Red: A Case Study on the Spread and Victims of an Internet Worm, ACM, New York.Google ScholarGoogle Scholar
  37. Netsparker tool. Netsparker, web application security scanner. http://www.mavitunasecurity.com/netsparker/.Google ScholarGoogle Scholar
  38. Nexgin Dataset. Nexgin rc dataset. http://www.nexginrc.org/Datasets/Default.aspx.Google ScholarGoogle Scholar
  39. NUST Dataset. NUST traffic datasets. http://wisnet.seecs.nust.edu.pk/projects/nes/datasets.html.Google ScholarGoogle Scholar
  40. Pang, R., Allman, M., Bennett, M., Lee, J., Paxson, V., and Tierney, B. 2005. A first look at modern enterprise traffic. In Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement (IMC’05). USENIX Association, Berkeley, CA, 2--2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ryu, Y. U. and Rhee, H. S. 2008. Evaluation of intrusion detection systems under a resource constraint. ACM Trans. Inf. Syst. Secur. 11, 20:1--20:24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Shannon, C. and Moore, D. 2004. The spread of the witty worm. In Proceedings of IEEE Security and Privacy (SP’04) 2, 46--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Sqlninja tool. Sqlninja, a SQL server injection and takeover tool. http://sqlninja.sourceforge.net/.Google ScholarGoogle Scholar
  44. Sung, A. H., Xu, J., Chavez, P., and Mukkamala, S. 2004. Static analyzer of vicious executables (SAVE). In Proceedings of the 20th Annual Computer Security Applications Conference (ACSAC’04). IEEE Computer Society, Los Alamitos, CA, 326--334. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Symantec Security. Symantec security response. http://securityresponse.symantec.com/avcenter.Google ScholarGoogle Scholar
  46. TADM toolkit. Tadm, toolkit for advanced discriminative modeling. http://tadm.sourceforge.net.Google ScholarGoogle Scholar
  47. Tcpdump tool. Tcpdump/libpcap public repository. http://www.tcpdump.org/.Google ScholarGoogle Scholar
  48. Trees, H. L. V. 2001. Detection, Estimation and Modulation Theory: Part I 1st Ed. Wiley-Interscience.Google ScholarGoogle Scholar
  49. Twycross, J. and Williamson, M. M. 2003. Implementing and testing a virus throttle. In Proceedings of the 12th Conference on USENIX Security Symposium. USENIX Association, Berkeley, CA, 20--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. UNM Dataset. Computer immune systems, datasets. http://www.cs.unm.edu/~immsec/data/synth-sm.html.Google ScholarGoogle Scholar
  51. Wang, K. and Stolfo, S. J. 2004. Anomalous payload-based network intrusion detection. In Proceedings of the 7th International Symposium on Recent Advances in Intrusion Detection (RAID). 203--222.Google ScholarGoogle Scholar
  52. WisNet ADS. Wisnet ADS comparison homepage. http://wisnet.niit.edu.pk/projects/adeval.Google ScholarGoogle Scholar
  53. Yu, Z., Tsai, J. J. P., and Weigert, T. 2007. An automatically tuning intrusion detection system. IEEE Trans. Syst., Man, and Cybernet. 37, 373--384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Yu, Z., Tsai, J. J. P., and Weigert, T. 2008. An adaptive automatically tuning intrusion detection system. ACM Trans. Autonom. Adaptive Syst. 3, 10:1--10:25. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Automated Anomaly Detector Adaptation using Adaptive Threshold Tuning

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Information and System Security
            ACM Transactions on Information and System Security  Volume 15, Issue 4
            April 2013
            117 pages
            ISSN:1094-9224
            EISSN:1557-7406
            DOI:10.1145/2445566
            Issue’s Table of Contents

            Copyright © 2013 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 1 April 2013
            • Accepted: 1 January 2013
            • Revised: 1 October 2012
            • Received: 1 February 2011
            Published in tissec Volume 15, Issue 4

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!