skip to main content
research-article

Evaluation of Intrusion Detection Systems Under a Resource Constraint

Published:01 July 2008Publication History
Skip Abstract Section

Abstract

An intrusion detection system plays an important role in a firm's overall security protection. Its main purpose is to identify potentially intrusive events and alert the security personnel to the danger. A typical intrusion detection system, however, is known to be imperfect in detection of intrusive events, resulting in high false-alarm rates. Nevertheless, current intrusion detection models unreasonably assume that upon alerts raised by a system, an information security officer responds to all alarms without any delay and avoids damages of hostile activities. This assumption of responding to all alarms with no time lag is often impracticable. As a result, the benefit of an intrusion detection system can be overestimated by current intrusion detection models. In this article, we extend previous models by including an information security officer's alarm inspection under a constraint as a part of the process in determining the optimal intrusion detection policy. Given a potentially hostile environment for a firm, in which the intrusion rates and costs associated with intrusion and security officers' inspection can be estimated, we outline a framework to establish the optimal operating points for intrusion detection systems under security officers' inspection constraint. The optimal solution to the model will provide not only a basis of better evaluation of intrusion detection systems but also useful insights into operations of intrusion detection systems. The firm can estimate expected benefits for running intrusion detection systems and establish a basis for increase in security personnel to relax security officers' inspection constraint.

References

  1. Anderson, R. 2001. Why information security is hard---An economic perspective. In Proceedings of the Seventeenth Annual Computer Security Applications Conference (ACSAC'01). New Orleans, LA.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Axelsson, S. 2000. The base-rate fallacy and the difficulty of intrusion detection. ACM Trans. Inform. Syst. Secur. 3, 3, 186--205.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Cardenas, A. A., Ramezani, V., and Baras, J. S. 2003. HMM sequential hypothesis tests for intrusion detection in MANETs. Technical Report 2003-47, Department of Electrical and Computer Engineering, Maryland University.]]Google ScholarGoogle Scholar
  4. Cavusoglu, H. and Raghunathan, S. 2004. Configuration of detection software: A comparison of decision and game theory approaches. Decis. Anal. 1, 3, 131--148.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Durst, R., Champion, T., Witten, B., Miller, E., and Spagnuolo, L. 1999. Testing and evaluating computer intrusion detection systems. Comm. ACM 42, 7, 53--61.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Endorf, C., Schultz, E., and Mellander, J. 2004. Intrusion Detection & Prevention. Emeryville, CA: McGraw-Hill/Osborne.]]Google ScholarGoogle Scholar
  7. Gaffney, J. E. and Ulvila, J. W. 2001. Evaluation of intrusion detectors: A decision theory approach. In Proceedings of IEEE Symposium on Security and Privacy (SP'01). Oakland, CA, 50--61.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Kemmerer, R. A. and Vigna, G. 2002. Intruder detection: A brief history and overview. Secur. Privacy, Suppl. Comput. 0, 27--30.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kent, S. 2000. On the trail of intrusions into information systems. IEEE Spectrum 37, 12, 52--56.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Lee, W., Fan, W., Miller, M., Stolfo, S. J., and Zadok, E. 2002. Toward cost-sensitive modeling for intrusion detection and response. J. Comput. Secur. 10, 5--22.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Lippmann, R. P., Cunningham, R. K., Fried, D. J., Graf, I., Kendall, K. R., Webster, S. E., and Zissman, M. A. 1999. Results of the DARPA 1998 off-line intrusion detection evaluation. In Proceedings of the 2nd International Workshop on the Recent Advances in Intrusion Detection (RAID'99). West Lafayette, IN.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Lippmann, R. P., Fried, D. J., Graf, I., Haines, J. W., Kendall, K. R., McClung, D., Weber, D., Webster, S. E., Wyschogrod, D., Cunningham, R. K., and Zissman, M. A. 2000. Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation. In Proceedings of DARPA Information Survivability Conference and Exposition (DISCEX'00). Vol. 2. Hilton Head, SC, 1012--1026.]]Google ScholarGoogle Scholar
  13. Metz, C. E. 1978. Basic principles of ROC analysis. Seminars in Nuclear Medicine 8, 4, 283--298.]]Google ScholarGoogle Scholar
  14. Neumann, P. G. and Porras, P. A. 1999. Experience with EMERALD to date. In Proceedings of the 1st USENIX Workshop on Intrusion Detection and Network Monitoring (ID'99). Santa Clara, CA. 73--80.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Parker, D. B. 1983. Fighting Computer Crime. New York: Charles Scribner's Sons.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Porras, P. A. and Neumann, P. G. 1997. EMERALD: Event monitoring enabling responses to anomalous live disturbances. In Proceedings of the 20th NIST-NCSC National Information Systems Security Conference (NISSC'97). Baltimore, MD, 353--365.]]Google ScholarGoogle Scholar
  17. Proctor, P. E. 2000. The Practical Intrusion Detection Handbook. Prentice Hall, Englewood Cliffs, NJ.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Ryu, Y. U. and Yue, W. T. 2003. A risk-based evaluation of intrusion detection systems in the presence of the base-rate fallacy. Working paper, Department of Information Systems and Operations Management, School of Management, The University of Texas at Dallas.]]Google ScholarGoogle Scholar
  19. Stolfo, S. J., Lee, W., Chan, P. K., Fan, W., and Eskin, E. 2001. Data mining-based intrusion detectors: An overview of the Columbia IDS project. SIGMOD Record 30, 4 (December), 5--14.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. The Snort Project. 2007. Snort#8482; User Manual 2.6.1. Sourcefire, Inc.]]Google ScholarGoogle Scholar
  21. Ulvila, J. W. and Gaffney, J. E. 2004. A decision analysis method for evaluating computer intrusion detection systems. Decis. Anal. 1, 1, 35--50.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Warrender, C., Forrest, S., and Pearlmutter, B. 1999. Detecting intrusions using system calls: Alternative data models. In Proceedings of the IEEE Symposium on Security and Privacy (SP'99). Oakland, CA, 133--145.]]Google ScholarGoogle Scholar
  23. Zweig, M. H. and Campbell, G. 1993. Receiver-operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clin. Chem. 39, 561--577.]]Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Evaluation of Intrusion Detection Systems Under a Resource Constraint

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!