skip to main content
research-article

Fast and Black-box Exploit Detection and Signature Generation for Commodity Software

Published:01 December 2008Publication History
Skip Abstract Section

Abstract

In biology, a vaccine is a weakened strain of a virus or bacterium that is intentionally injected into the body for the purpose of stimulating antibody production. Inspired by this idea, we propose a packet vaccine mechanism that randomizes address-like strings in packet payloads to carry out fast exploit detection and signature generation. An exploit with a randomized jump address behaves like a vaccine: it will likely cause an exception in a vulnerable program’s process when attempting to hijack the control flow, and thereby expose itself. Taking that exploit as a template, our signature generator creates a set of new vaccines to probe the program in an attempt to uncover the necessary conditions for the exploit to happen. A signature is built upon these conditions to shield the underlying vulnerability from further attacks. In this way, packet vaccine detects exploits and generates signatures in a black-box fashion, that is, not relying on the knowledge of a vulnerable program’s source and binary code. Therefore, it even works on the commodity software obfuscated for the purpose of copyright protection. In addition, since our approach avoids the expense of tracking the program’s execution flow, it performs almost as fast as a normal run of the program and is capable of generating a signature of high quality within seconds or even subseconds. We present the design of the packet vaccine mechanism and an example of its application. We also describe our proof-of-concept implementation and the evaluation of our technique using real exploits.

References

  1. Anagnostakis, K. G., Siridoglou, S., Akritidis, P., Xinidis, K., Markatos, E., and Keromytis, A. 2005. Detecting targeted attacks using shadow honeypots. In Proceedings of the USENIX Security Symposium (SECURITY’05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Associated Press. 2006. Microsoft warns against outside fixes. http://biz.yahoo.com/ap/060331/microsoft_s_security_snags.html?.v=4.Google ScholarGoogle Scholar
  3. Ballista. 2006. The Ballista@ Project: COTS Software Robustness Testing. http://www.ece.cmu.edu/~koopman/ballista/.Google ScholarGoogle Scholar
  4. Barton, J. H., Czeck, E. W., Segall, Z. Z., and Siewiorek, D. P. 1990. Fault injection.Google ScholarGoogle Scholar
  5. Brumley, D., Newsome, J., Song, D., Wang, H., and Jha, S. 2006. Towards automatic generation of vulnerability-based signatures. In Proceedings of the IEEE Symposium on Security and Privacy (SSP’06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Carrette, G. J. 2006. CRASHME: Random input testing. http://people.delphiforums.com/gjc/crashme.html.Google ScholarGoogle Scholar
  7. Costa, M., Crowcroft, J., Castro, M., Rowstron, A. I. T., Zhou, L., Zhang, L., and Barham, P. T. 2005. Vigilante: End-to-end containment of internet worms. In Proceedings of 19th ACM Symposium on Operating Systems Principles (SOSP’05). 133--147. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Crandall, J. R. and Chong, F. T. 2004. Minos: Control data attack prevention orthogonal to memory model. In Proceedings of the 40th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO-34). 221--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Crandall, J. R., Su, Z., and Wu, S. F. 2005. On deriving unknown vulnerabilities from zero-day polymorphic and metamorphic worm exploits. In Proceedings of the 12th ACM Conference on Computer and Communications Security (CCS’05). 235--248. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Crandall, J. R., Wu, S. F., and Chong, F. T. 2005. Experiences Using Minos as a Tool for Capturing and Analyzing Novel Worms for Unknown Vulnerabilities. In Proceedings of the GI International Conference on Detection of Intrusions & Malware, and Vulnerability Assessment (DIMVA’’05). 32--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Cui, W., Peinado, M., Wang, H. J., and Locasto, M. E. 2007. Shieldgen: Automatic data patch generation for unknown vulnerabilities with informed probing. In Proceedings of the IEEE Symposium on Security and Privacy (SSP’07). 252--266. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Dreger, H., Kreibich, C., Paxson, V., and Sommer, R. 2005. Enhancing the accuracy of network-based intrusion detection with host-based context. In Proceedings of the GI International Conference on Detection of Intrusions & Malware, and Vulnerability Assessment (DIMVA’’05). 206--221. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Dunlap, G. W., King, S. T., Cinar, S., Basrai, M. A., and Chen, P. M. 2002. Revirt: Enabling intrusion analysis through virtual-machine logging and replay. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI’02) Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. HoneyNet. 2006. http://www.honeynet.org/.Google ScholarGoogle Scholar
  15. Kanawati, G. A., Kanawati, N. A., and Abraham, J. A. 1995. FERRARI: A flexible software-based fault and error injection system. IEEE Trans. Comput. 44, 2, 248--260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Kim, H.-A. and Karp, B. 2004. Autograph: Toward automated, distributed worm signature detection. In Proceedings of 13th USENIX Security Symposium (SECURITY’04). 271--286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kreibich, C. and Crowcroft, J. 2004. Honeycomb: Creating intrusion detection signatures using honeypots. SIGCOMM Comput. Comm. Rev. 34, 1, 51--56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kruegel, C., Kirda, E., Mutz, D., Robertson, W., and Vigna, G. 2005. Polymorphic worm detection using structural information of executables. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection (RAID’05). 207--226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Liang, Z. and Sekar, R. 2005a. Automatic generation of buffer overflow attack signatures: An approach based on program behavior models. In Proceedings of the Annual Computer Security Applications Conference (CSAC’05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Liang, Z. and Sekar, R. 2005b. Fast and automated generation of attack signatures: A basis for building self-protecting servers. In Proceedings of the 12th ACM Conference on Computer and Communications Security (CCS’05). 213--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Liang, Z., Sekar, R., and DuVarney, D. C. April, 2005. Automatic synthesis of filters to discard buffer overflow attacks: A step towards realizing self-healing systems. In Proceedings of the USENIX Annual Technical Conference (USENIX’05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Locasto, M. E., Sidiroglou, S., and Keromytis, A. D. 2006. Software self-healing using collaborative application communities. In Proceedings of the 13th Annual Network and Distributed Systems Security Symposium (NDSS’06).Google ScholarGoogle Scholar
  23. Locasto, M. E., Wang, K., Keromytis, A. D., and Stolfo, S. J. 2005. Flips: Hybrid adaptive intrusion prevention. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection (RAID’05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Marty, R. 2002. Thor: A tool to test intrusion detection systems by variations of attacks. Master thesis, ETH Zurich.Google ScholarGoogle Scholar
  25. MemView. 2006. http://www2.biglobe.ne.jp/ sota/memview-e.html.Google ScholarGoogle Scholar
  26. Microsoft. 2007. Microsoft debuging tools: Overview. http://www.microsoft.com/whdc/devtools/debugging/default.mspx.Google ScholarGoogle Scholar
  27. Mockapetris, P. 1987. DOMAIN NAMES - IMPLEMENTATION AND SPECIFICATION. RFC 3425. http://www.ietf.org/rfc/rfc1035.txt.Google ScholarGoogle Scholar
  28. Musa, J., Fuoco, G., Irving, N., Juhlin, B., and Kropfl, D. 1996. Handbook of Software Reliability Engineering. McGraw-Hill, New York, 167--216.Google ScholarGoogle Scholar
  29. Naumovich, G. and Memon, N. D. 2003. Preventing piracy, reverse engineering, and tampering. IEEE Comput. 36, 7, 64--71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Newsome, J., Brumley, D., and Song, D. 2005. Vulnerability-specific execution filtering for exploit prevention on commodity software. In Proceedings of the 13th Annual Network and Distributed Systems Security Symposium (NDSS’05).Google ScholarGoogle Scholar
  31. Newsome, J., Karp, B., and Song, D. 2005. Polygraph: Automatically generating signatures for polymorphic worms. In Proceedings of IEEE Symposium on Security and Privacy (SSP’05). 226--241. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Newsome, J. and Song, D. 2005. Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software. In Proceedings of the 12th Annual Network and Distributed System Security Symposium (NDSS’05). San Diego, CA.Google ScholarGoogle Scholar
  33. Pasupulati, A., Coit, J., Levitt, K., Wu, S., Li, S., Kuo, R., and Fan, K. 2004. Buttercup: On network-based detection of polymorphic buffer overflow vulnerabilities. In Proceedings of the IEEE/IFIP Network Operation and Management Symposium (NOMS’04).Google ScholarGoogle Scholar
  34. Perdisci, R., Dagon, D., Lee, W., Fogla, P., and Sharif, M. 2006. Misleading worm signature generators using deliberate noise injection. In Proceedings of the IEEE Symposium on Security and Privacy (SSP’06). 17--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Portokalidis, G. and Bos, H. 2005. SweetBait: Zero-hour worm detection and containment using honeypots. Tech. rep. IR-CS-015, Vrije Universiteit Amsterdam.Google ScholarGoogle Scholar
  36. Reynolds, J. C., Just, J., Clough, L., and Maglich, R. 2003. On-line intrusion detection and attack prevention using diversity, generate-and-test, and generalization. In Proceedings of the Annual Hawaii International Conference on System Sciences (HICSS’03). 335.2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Richardson, D. W., Gribble, S. D., and Lazowska, E. D. 2005. The limits of global scanning worm detectors in the presence of background noise. In Proceedings of the 2005 ACM workshop on Rapid malcode (WORM’05). ACM Press, 60--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. SecurityFocus. 2006. http://www.securityfocus.com.Google ScholarGoogle Scholar
  39. Shacham, H., Page, M., Pfaff, B., Goh, E.-J., Modadugu, N., and Boneh, D. 2004. On the effectiveness of address-space randomization. In Proceedings of the ACM Conference on Computer and Communications Security (CCS’04). 298--307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Shannon, C. and Moore, D. 2004. The spread of the witty worm. IEEE Secur. Privacy 2, 4 (July/August), 46--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Sidiroglou, S., Locasto, M. E., Boyd, S. W., and Keromytis, A. D. 2005. Building a reactive immune system for software services. In Proceedings of the USENIX Annual Technical Conference (USENIX’05). 149--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Singh, S., Estan, C., Varghese, G., and Savage, S. 2004. Automated worm fingerprinting. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI’04). 45--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Spitzner, L. 2003. Honeypots: Catching the insider threat. In Proceedings of the Annual Computer Security Applications Conference (CSAC’03). 170--181. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Srinivasan, S. M., Kandula, S., Andrews, C. R., and Zhou, Y. 2004. Flashback: A lightweight extension for rollback and deterministic replay for software debugging. In Proceedings of USENIX Annual Technical Conference, General Track (USENIX’04). 29--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Suh, G. E., Lee, J. W., Zhang, D., and Devadas, S. 2004. Secure program execution via dynamic information flow tracking. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’04). 85--96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Tang, Y. and Chen, S. 2005. Defending against internet worms: A signature-based approach. In Proceedings of the Annual IEEE Conference on Computer Communications (INFOCOM’05). Miami, FL.Google ScholarGoogle Scholar
  47. Telescope. 2006. http://www.caida.org/analysis/security/telescope/.Google ScholarGoogle Scholar
  48. Toth, T. and Krügel, C. 2002. Accurate buffer overflow detection via abstract payload execution. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection (RAID’02). 274--291. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Tsai, T. K. and Iyer, R. K. 1995. Measuring fault tolerance with the ftape fault injection tool. In Proceedings of the 8th International Conference on Modelling Techniques and Tools for Computer Performance Evaluation (MMB’95). Springer-Verlag, 26--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Tucek, J., Lu, S., Huang, C., Xanthos, S., Zhou, Y., Newsome, J., Brumley, D., and Song, D. 2007. Sweeper: A lightweight end-to-end system for defending against fast worms. SIGOPS Oper. Syst. Rev. 41, 3, 115--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. US-CERT. Microsoft windows metafile handler setabortproc gdi escape vulnerability. http://www.kb.cert.org/vuls/id/181038.Google ScholarGoogle Scholar
  52. van Oorschot, P. C. 2003. Revisiting software protection. In Proceedings of the Information Security Conference (ISC’03). 1--13.Google ScholarGoogle ScholarCross RefCross Ref
  53. Vigna, G., Robertson, W., and Balzarotti, D. 2004. Testing network-based intrusion detection signatures using mutant exploits. In Proceedings of the ACM Conference on Computer and Communication Security (CCS’04). Washington, DC, 21--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Vulnerabilities 2006. http://www.securityfocus.com/vulnerabilities.Google ScholarGoogle Scholar
  55. Wang, H. J., Guo, C., Simon, D. R., and Zugenmaier, A. 2004. Shield: Vulnerability-driven network filters for preventing known vulnerability exploits. In Proceedings of the ACM Conference on Applications, Technologies, Architectures, and Procotols for Computer Communications (SIGCOMM’04). 193--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Wang, K. and Stolfo, S. J. 2004. Anomalous payload-based network intrusion detection. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection (RAID’04). 203--222.Google ScholarGoogle Scholar
  57. Wang, X., Li, Z., Xu, J., Reiter, M. K., Kil, C., and Choi, J. Y. 2006. Packet vaccine: Black-box exploit detection and signature generation. In Proceedings of the ACM Conference on Computer and Communications Security (CCS’06). 37--46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Wang, X., Pan, C.-C., Liu, P., and Zhu, S. 2006. Sigfree: A signature-free buffer overflow attack blocker. In Proceedings of the 15th Conference on USENIX Security Symposium (SECURITY’06). Berkeley, CA, 215--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Wasson, S. 2004. The NX bit. http://techreport.com/reviews/2004q4/pentium4-570j/index.x?pg=1.Google ScholarGoogle Scholar
  60. Whyte, D., Kranakis, E., and van Oorschot, P. 2005. DNS-based detection of scanning worms in an enterprise network. In Proceedings of the 12th Network and Distributed System Security Symposium (NDSS). 181--195.Google ScholarGoogle Scholar
  61. Xu, J., Ning, P., Kil, C., Zhai, Y., and Bookholt, C. 2005. Automatic diagnosis and response to memory corruption vulnerabilities. In Proceedings of the 12th ACM Conference on Computer and Communications Security (CCS’05). ACM Press, New York, NY, 223--234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Yegneswaran, V., Giffin, J. T., Barford, P., and Jha, S. 2005. An architecture for generating semantics-aware signatures. In Proceedings of USENIX Security Symposium (SECURITY’05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Zou, C. C., Towsley, D., Gong, W., and Cai, S. 2005. Routing worm: A fast, selective attack worm based on ip address information. In Proceedings of the 19th Workshop on Principles of Advanced and Distributed Simulation (PADS’05). 199--206. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Fast and Black-box Exploit Detection and Signature Generation for Commodity Software

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Information and System Security
        ACM Transactions on Information and System Security  Volume 12, Issue 2
        December 2008
        202 pages
        ISSN:1094-9224
        EISSN:1557-7406
        DOI:10.1145/1455518
        Issue’s Table of Contents

        Copyright © 2008 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 1 December 2008
        • Revised: 1 September 2007
        • Accepted: 1 September 2007
        • Received: 1 February 2007
        Published in tissec Volume 12, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!