skip to main content
research-article

Checksum-Aware Fuzzing Combined with Dynamic Taint Analysis and Symbolic Execution

Published:01 September 2011Publication History
Skip Abstract Section

Abstract

Fuzz testing has proven successful in finding security vulnerabilities in large programs. However, traditional fuzz testing tools have a well-known common drawback: they are ineffective if most generated inputs are rejected at the early stage of program running, especially when target programs employ checksum mechanisms to verify the integrity of inputs. This article presents TaintScope, an automatic fuzzing system using dynamic taint analysis and symbolic execution techniques, to tackle the above problem. TaintScope has several novel features: (1) TaintScope is a checksum-aware fuzzing tool. It can identify checksum fields in inputs, accurately locate checksum-based integrity checks by using branch profiling techniques, and bypass such checks via control flow alteration. Furthermore, it can fix checksum values in generated inputs using combined concrete and symbolic execution techniques. (2) TaintScope is a taint-based fuzzing tool working at the x86 binary level. Based on fine-grained dynamic taint tracing, TaintScope identifies the “hot bytes” in a well-formed input that are used in security-sensitive operations (e.g., invoking system/library calls), and then focuses on modifying such bytes with random or boundary values. (3) TaintScope is also a symbolic-execution-based fuzzing tool. It can symbolically evaluate a trace, reason about all possible values that can execute the trace, and then detect potential vulnerabilities on the trace.

We evaluate TaintScope on a number of large real-world applications. Experimental results show that TaintScope can accurately locate the checksum checks in programs and dramatically improve the effectiveness of fuzz testing. TaintScope has already found 30 previously unknown vulnerabilities in several widely used applications, including Adobe Acrobat, Flash Player, Google Picasa, and Microsoft Paint. Most of these severe vulnerabilities have been confirmed by Secunia and oCERT, and assigned CVE identifiers (such as CVE-2009-1882, CVE-2009-2688). Vendor patches have been released or are in preparation based on our reports.

References

  1. Balakrishnan, G. and Reps, T. 2010. Wysinwyx: What you see is not what you execute. ACM Trans. Program. Lang. Syst. 32, 23:1--23:84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Banks, G., Cova, M., Felmetsger, V., Almeroth, K., Kemmerer, R., and Vigna, G. 2006. SNOOZE: toward a Stateful NetwOrk prOtocol fuzZEr. In Proceedings of the Information Security Conference (ISC). Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Boutell, T. 1997. PNG Specification. RFC 2083, Internet Engineering Task Force.Google ScholarGoogle Scholar
  4. Brumley, D., Hartwig, C., Kang, M. G., Liang, Z., Newsome, J., Poosankam, P., Song, D., and Yin, H. 2007. Bitscope: Automatically dissecting malicious binaries. Tech. rep. CMU-CS-07-133, Carnegie Mellon University.Google ScholarGoogle Scholar
  5. Brumley, D., Wang, H., Jha, S., and Song, D. 2007. Creating vulnerability signatures using weakest preconditions. In Proceedings of the 20th IEEE Computer Security Foundations Symposium (CSF’07). IEEE Computer Society, Los Alamitos, CA, 311--325. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Brumley, D., Poosankam, P., Song, D., and Zheng, J. 2008. Automatic patch-based exploit generation is possible: Techniques and implications. In Proceedings of the IEEE Symposium on Security and Privacy (SP’08). IEEE Computer Society, Los Alamitos, CA, 143--157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Caballero, J., Yin, H., Liang, Z., and Song, D. 2007. Polyglot: Automatic extraction of protocol message format using dynamic binary analysis. In Proceedings of the 14th ACM Conference on Computer and Communications Security. ACM, New York, NY, 317--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Caballero, J., Poosankam, P., Kreibich, C., and Song, D. 2009. Dispatcher: Enabling active botnet infiltration using automatic protocol reverse-engineering. In Proceedings of the 16th ACM Conference on Computer and Communications Security. ACM, New York, NY, 621--634. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Caballero, J., Poosankam, P., McCamant, S., Babic, D., and Song, D. 2010. Input generation via decomposition and re-stitching: Finding bugs in malware. In Proceedings of the 17th ACM Conference on Computer and Communication Security. ACM, New York. NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Cadar, C., Dunbar, D., and Engler, D. 2008. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI’08). USENIX Association, Berkeley, CA, 209--224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Cadar, C., Ganesh, V., Pawlowski, P. M., Dill, D. L., and Engler, D. R. 2008. Exe: Automatically generating inputs of death. ACM Trans. Info. Syst. Sec. 12, 2, 1--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Cifuentes, C. 1994. Reverse compilation techniques. Ph.D. thesis, Queensland University of Technology, Australia.Google ScholarGoogle Scholar
  13. Clause, J. and Orso, A. 2009. Penumbra: Automatically identifying failure-relevant inputs using dynamic tainting. In Proceedings of the 18th International Symposium on Software Testing and Analysis (ISSTA’09). ACM, New York, NY, 249--260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Clause, J., Li, W., and Orso, A. 2007. Dytan: A generic dynamic taint analysis framework. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’07). ACM, New York, NY, 196--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Comparetti, P. M., Wondracek, G., Kruegel, C., and Kirda, E. 2009. Prospex: Protocol specification extraction. In Proceedings of the 30th IEEE Symposium on Security and Privacy (SP’09). IEEE Computer Society, Los Alamitos, CA, 110--125. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Cova, M., Felmetsger, V., Banks, G., and Vigna, G. 2006. Static detection of vulnerabilities in x86 executables. In Proceedings of the 22nd Annual Computer Security Applications Conference (ACSAC’06). IEEE Computer Society, Los Alamitos, CA, 269--278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Cui, W., Kannan, J., and Wang, H. J. 2007. Discoverer: Automatic protocol reverse engineering from network traces. In Proceedings of 16th USENIX Security Symposium (SS’07). USENIX Association, Berkeley, CA, 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Cui, W., Peinado, M., Chen, K., Wang, H. J., and Irun-Briz, L. 2008. Tupni: Automatic reverse engineering of input formats. In Proceedings of the 15th ACM Conference on Computer and Communications Security (CCS’08). ACM, New York, NY, 391--402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Deutsch, P. and Gailly, J.-L. 1996. ZLIB compressed data format specification version 3.3. RFC 1950, Internet Engineering Task Force. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Drewry, W. and Ormandy, T. 2007. Flayer: Exposing application internals. In Proceedings of the 1st USENIX Workshop on Offensive Technologies (WOOT’07). USENIX Association, Berkeley, CA, 1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Egele, M., Kruegel, C., Kirda, E., Yin, H., and Song, D. 2007. Dynamic spyware analysis. In Proceedings of the USENIX Annual Technical Conference (ATC’07). USENIX Association, Berkeley, CA, 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Elkarablieh, B., Godefroid, P., and Levin, M. Y. 2009. Precise pointer reasoning for dynamic test generation. In Proceedings of the 18th International Symposium on Software Testing and Analysis (ISSTA’09). ACM, New York, NY, 129--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Ganesh, V., Leek, T., and Rinard, M. 2009. Taint-based directed whitebox fuzzing. In Proceedings of the 31st International Conference on Software Engineering (ICSE’09). IEEE Computer Society, Los Alamitos, CA, 474--484. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Godefroid, P., Klarlund, N., and Sen, K. 2005. Dart: Directed automated random testing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’05). ACM, New York, NY, 213--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Godefroid, P., Kiezun, A., and Levin, M. Y. 2008a. Grammar-based whitebox fuzzing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’08). ACM, New York, NY, 206--215. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Godefroid, P., Levin, M., and Molnar, D. 2008b. Automated whitebox fuzz testing. In Proceedings of the 15th Annual Network and Distributed System Security Symposium.Google ScholarGoogle Scholar
  27. Johnson, N., Caballero, J., Chen, K. Z., McCamant, S., Poosankam, P., Reynaud, D., and Song, D. 2011. Differential slicing: Identifying causal execution differences for security applications. In Proceedings of the 32nd IEEE Symposium on Security and Privacy (Oakland’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Kang, M. G., Yin, H., Hanna, S., McCamant, S., and Song, D. 2009. Emulating emulation-resistant malware. In Proceedings of the 2nd Workshop on Virtual Machine Security (VMSec). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Korn, D., MacDonald, J., Mogul, J., and Vo, K. 2002. The VCDIFF generic differencing and compression data format. RFC 3284, Internet Engineering Task Force. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Lin, Z., Jiang, X., Xu, D., and Zhang, X. 2008. Automatic protocol format reverse engineering through context-aware monitored execution. In Proceedings of the 15th Annual Network and Distributed System Security Symposium.Google ScholarGoogle Scholar
  31. Luk, C.-K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V. J., and Hazelwood, K. 2005. Pin: Building customized program analysis tools with dynamic instrumentation. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, NY, 190--200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Majumdar, R. and Xu, R.-G. 2007. Directed test generation using symbolic grammars. In Proceedings of the 6th Joint Meeting on European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC-FSE Companion’07). ACM, New York, NY, 553--556. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Miller, B. P., Fredriksen, L., and Bryan, S. 1990. An empirical study of the reliability of UNIX utilities. Comm. ACM 33, 12, 32--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. MoBB. 2006. http://browserfun.blogspot.com.Google ScholarGoogle Scholar
  35. MoKB. 2006. Month of Kernel Bugs. http://projects.info-pull.com/mokb/.Google ScholarGoogle Scholar
  36. Molnar, D., Li, X. C., and Wagner, D. A. 2009. Dynamic test generation to find integer bugs in x86 binary linux programs. In Proceedings of the 18th USENIX Security Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Moser, A., Kruegel, C., and Kirda, E. 2007. Exploring multiple execution paths for malware analysis. In Proceedings of the IEEE Symposium on Security and Privacy (SP’07). IEEE Computer Society, Los Alamitos, CA, 231--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Nethercote, N. and Seward, J. 2007. Valgrind: A framework for heavyweight dynamic binary instrumentation. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’07). ACM, New York, NY, 89--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Newsome, J., Brumley, D., Franklin, J., and Song, D. 2006. Replayer: Automatic protocol replay by binary analysis. In Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS’06). ACM, New York, NY, 311--321. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Oehlert, P. 2005. Violating assumptions with fuzzing. IEEE Sec. Priv. 3, 2, 58--62. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Postel, J. 1981. Internet protocol. RFC 791, Internet Engineering Task Force.Google ScholarGoogle Scholar
  42. Sen, K., Marinov, D., and Agha, G. 2005. Cute: A concolic unit testing engine for c. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13). ACM, New York, NY, 263--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Sharif, M., Lanzi, A., Giffin, J., and Lee, W. 2008. Impeding malware analysis using conditional code obfuscation. In Proceedings of the 15th Annual Network and Distributed System Security Symposium.Google ScholarGoogle Scholar
  44. Stallings, W. 2005. Cryptography and Network Security 4th Ed. Prentice-Hall, Inc., Upper Saddle River, NJ, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Sutton, M., Greene, A., and Amini, P. 2007. Fuzzing: Brute Force Vulnerability Discovery. Addison-Wesley Professional. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Wang, T., Wei, T., Lin, Z., and Zou, W. 2009a. IntScope: Automatically detecting integer overflow vulnerability in X86 binary using symbolic execution. In Proceedings of the 16th Annual Network and Distributed System Security Symposium.Google ScholarGoogle Scholar
  47. Wang, Z., Jiang, X., Cui, W., Wang, X., and Grace, M. 2009b. Reformat: Automatic reverse engineering of encrypted messages. In Proceedings of the 14th European Conference on Research in Computer Security (ESORICS’09). Springer-Verlag, Berlin, 200--215. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Wei, T., Mao, J., Zou, W., and Chen, Y. 2007. Structuring 2-way branches in binary executables. In Proceedings of the 31st Annual International Computer Software and Applications Conference (COMPSAC’07). IEEE Computer Society, Los Alamitos, CA, 115--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Wilhelm, J. and Chiueh, T.-c. 2007. A forced sampled execution approach to kernel rootkit identification. In Proceedings of the 10th International Conference on Recent Advances in Intrusion Detection (RAID’07). Springer-Verlag, Berlin, 219--235. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Wondracek, G., Milani Comparetti, P., Kruegel, C., and Kirda, E. 2008. Automatic network protocol analysis. In Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS’08).Google ScholarGoogle Scholar
  51. Zalewski, M. 2007. Bunny-the-fuzzer: Instrumented c code security fuzzer. http://code.google.com/p/bunny-the-fuzzer/.Google ScholarGoogle Scholar

Index Terms

  1. Checksum-Aware Fuzzing Combined with Dynamic Taint Analysis and Symbolic Execution

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Information and System Security
      ACM Transactions on Information and System Security  Volume 14, Issue 2
      September 2011
      199 pages
      ISSN:1094-9224
      EISSN:1557-7406
      DOI:10.1145/2019599
      Issue’s Table of Contents

      Copyright © 2011 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 September 2011
      • Accepted: 1 April 2011
      • Revised: 1 March 2011
      • Received: 1 August 2010
      Published in tissec Volume 14, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!