skip to main content
research-article
Open Access
Artifacts Available
Artifacts Evaluated & Functional

FuzzFactory: domain-specific fuzzing with waypoints

Published:10 October 2019Publication History
Skip Abstract Section

Abstract

Coverage-guided fuzz testing has gained prominence as a highly effective method of finding security vulnerabilities such as buffer overflows in programs that parse binary data. Recently, researchers have introduced various specializations to the coverage-guided fuzzing algorithm for different domain-specific testing goals, such as finding performance bottlenecks, generating valid inputs, handling magic-byte comparisons, etc. Each such solution can require non-trivial implementation effort and produces a distinct variant of a fuzzing tool. We observe that many of these domain-specific solutions follow a common solution pattern.

In this paper, we present FuzzFactory, a framework for developing domain-specific fuzzing applications without requiring changes to mutation and search heuristics. FuzzFactory allows users to specify the collection of dynamic domain-specific feedback during test execution, as well as how such feedback should be aggregated. FuzzFactory uses this information to selectively save intermediate inputs, called waypoints, to augment coverage-guided fuzzing. Such waypoints always make progress towards domain-specific multi-dimensional objectives. We instantiate six domain-specific fuzzing applications using FuzzFactory: three re-implementations of prior work and three novel solutions, and evaluate their effectiveness on benchmarks from Google's fuzzer test suite. We also show how multiple domains can be composed to perform better than the sum of their parts. For example, we combine domain-specific feedback about strict equality comparisons and dynamic memory allocations, to enable the automatic generation of LZ4 bombs and PNG bombs.

Skip Supplemental Material Section

Supplemental Material

a174-padhye

Presentation at OOPSLA '19

References

  1. Cornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig, Ahmad-Reza Sadeghi, and Daniel Teuchert. 2019. Nautilus: Fishing for Deep Bugs with Grammars. In 26th Annual Network and Distributed System Security Symposium (NDSS ’19) .Google ScholarGoogle Scholar
  2. Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed Greybox Fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17) .Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. Coverage-based Greybox Fuzzing As Markov Chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16) .Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Peng Chen and Hao Chen. 2018. Angora: Efficient Fuzzing by Principled Search. In Proceedings of the 39th IEEE Symposium on Security and Privacy .Google ScholarGoogle ScholarCross RefCross Ref
  5. Yuanliang Chen, Yu Jiang, Fuchen Ma, Jie Liang, Mingzhe Wang, Chijin Zhou, Xun Jiao, and Zhuo Su. 2019. EnFuzz: Ensemble Fuzzing with Seed Synchronization among Diverse Fuzzers. In 28th USENIX Security Symposium (USENIX Security 19) . USENIX Association, Santa Clara, CA. https://www.usenix.org/conference/usenixsecurity19/presentation/chenyuanliangGoogle ScholarGoogle Scholar
  6. Nicolas Coppik, Oliver Schwahn, and Neeraj Suri. 2019. MemFuzz: Using Memory Accesses to Guide Fuzzing. In 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST) . IEEE, 48–58.Google ScholarGoogle ScholarCross RefCross Ref
  7. Google. 2019a. Continuous fuzzing of open source software. https://opensource.google.com/projects/oss-fuzz . Accessed March 26, 2019.Google ScholarGoogle Scholar
  8. Google. 2019b. Set of tests for fuzzing engines. https://github.com/google/fuzzer-test-suite . Accessed March 20, 2019.Google ScholarGoogle Scholar
  9. Lei Wei Junjie Wang, Bihuan Chen and Yang Liu. 2019. Superion: Grammar-Aware Greybox Fuzzing. In 41st International Conference on Software Engineering (ICSE ’19) .Google ScholarGoogle Scholar
  10. Kevin Laeufer, Jack Koenig, Donggyu Kim, Jonathan Bachrach, and Koushik Sen. 2018. RFUZZ: Coverage-directed Fuzz Testing of RTL on FPGAs. In Proceedings of the International Conference on Computer-Aided Design (ICCAD ’18). ACM, New York, NY, USA, Article 28, 8 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. LafIntel. 2016. Circumventing Fuzzing Roadblocks with Compiler Transformations. https://lafintel.wordpress.com/2016/08/ 15/circumventing-fuzzing-roadblocks-with-compiler-transformations/ . Accessed March 20, 2019.Google ScholarGoogle Scholar
  12. Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization (CGO ’04) . IEEE Computer Society, Washington, DC, USA, 75–. http://dl.acm.org/citation.cfm?id=977395. 977673Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Caroline Lemieux, Rohan Padhye, Koushik Sen, and Dawn Song. 2018. PerfFuzz: Automatically Generating Pathological Inputs. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2018). ACM, New York, NY, USA, 254–265. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Caroline Lemieux and Koushik Sen. 2018. FairFuzz: A Targeted Mutation Strategy for Increasing Greybox Fuzz Testing Coverage. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE ’18).Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Yuekang Li, Bihuan Chen, Mahinthan Chandramohan, Shang-Wei Lin, Yang Liu, and Alwen Tiu. 2017. Steelix: Program-state Based Binary Fuzzing. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017) .Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. LLVM Developer Group. 2016. libFuzzer. http://llvm.org/docs/LibFuzzer.html . Accessed March 20, 2019.Google ScholarGoogle Scholar
  17. Chi-Keung Luk, Robert Cohn, Robert Muth, Harish Patil, Artur Klauser, Geoff Lowney, Steven Wallace, Vijay Janapa Reddi, and Kim Hazelwood. 2005. Pin: Building Customized Program Analysis Tools with Dynamic Instrumentation. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’05) . ACM, New York, NY, USA, 190–200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Valentin J. M. Manès, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J. Schwartz, and Maverick Woo. 2018. Fuzzing: Art, Science, and Engineering. CoRR abs/1812.00140 (2018). arXiv: 1812.00140 http://arxiv.org/abs/ 1812.00140Google ScholarGoogle Scholar
  19. Shirin Nilizadeh, Yannic Noller, and Corina S. Păsăreanu. 2019. DifFuzz: Differential Fuzzing for Side-channel Analysis. In Proceedings of the 41st International Conference on Software Engineering (ICSE ’19). IEEE Press, Piscataway, NJ, USA, 176–187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Saahil Ognawala, Thomas Hutzelmann, Eirini Psallida, and Alexander Pretschner. 2018. Improving Function Coverage with Munch: A Hybrid Fuzzing and Directed Symbolic Execution Approach. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing (SAC ’18) . ACM, New York, NY, USA, 1475–1482. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Rohan Padhye, Caroline Lemieux, and Koushik Sen. 2019a. JQF: Coverage-guided Property-based Testing in Java. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2019) . ACM, New York, NY, USA, 398–401. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Rohan Padhye, Caroline Lemieux, Koushik Sen, Mike Papadakis, and Yves Le Traon. 2019b. Semantic Fuzzing with Zest. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2019) . ACM, New York, NY, USA, 329–340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Rohan Padhye, Caroline Lemieux, Koushik Sen, Mike Papadakis, and Yves Le Traon. 2019c. Validity Fuzzing and Parametric Generators for Effective Random Testing. In Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings (ICSE ’19) . IEEE Press, Piscataway, NJ, USA, 266–267. https://dl.acm.org/citation.cfm?id=3339777Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Hui Peng, Yan Shoshitaishvili, and Mathias Payer. 2018. T-Fuzz: fuzzing by program transformation. In 2018 IEEE Symposium on Security and Privacy (SP) . IEEE, 697–710.Google ScholarGoogle ScholarCross RefCross Ref
  25. Theofilos Petsios, Adrian Tang, Salvatore Stolfo, Angelos D Keromytis, and Suman Jana. 2017a. Nezha: Efficient domainindependent differential testing. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 615–632.Google ScholarGoogle ScholarCross RefCross Ref
  26. Theofilos Petsios, Jason Zhao, Angelos D. Keromytis, and Suman Jana. 2017b. SlowFuzz: Automated Domain-Independent Detection of Algorithmic Complexity Vulnerabilities. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17) . ACM, New York, NY, USA, 2155–2168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Van-Thuan Pham, Marcel Böhme, Andrew E. Santosa, Alexandru Razvan Caciulescu, and Abhik Roychoudhury. 2018. Smart Greybox Fuzzing. CoRR abs/1811.09447 (2018). arXiv: 1811.09447 http://arxiv.org/abs/1811.09447Google ScholarGoogle Scholar
  28. Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar, Cristiano Giuffrida, and Herbert Bos. 2017. VUzzer: Applicationaware Evolutionary Fuzzing. In Proceedings of the 2017 Network and Distributed System Security Symposium (NDSS ’17) .Google ScholarGoogle ScholarCross RefCross Ref
  29. Kostya Serebryany, Vitaly Buka, and Matt Morehouse. 2017. Structure-aware fuzzing for Clang and LLVM with libprotobufmutator.Google ScholarGoogle Scholar
  30. Richard M. Stallman et al. 2009. Using The Gnu Compiler Collection: A Gnu Manual For Gcc Version 4.3.3. CreateSpace, Paramount, CA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2016. Driller: Augmenting Fuzzing Through Selective Symbolic Execution. In Proceedings of the 2016 Network and Distributed System Security Symposium (NDSS ’16) .Google ScholarGoogle ScholarCross RefCross Ref
  32. Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018. QSYM: A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing. In Proceedings of the 27th USENIX Conference on Security Symposium (SEC’18). USENIX Association, Berkeley, CA, USA, 745–761. http://dl.acm.org/citation.cfm?id=3277203.3277260Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Michał Zalewski. 2014. American Fuzzy Lop. http://lcamtuf.coredump.cx/afl . Accessed March 20, 2019.Google ScholarGoogle Scholar
  34. Michał Zalewski. 2017. American Fuzzy Lop Technical Details. http://lcamtuf.coredump.cx/afl/technical_details.txt . Accessed March 20, 2019.Google ScholarGoogle Scholar

Index Terms

  1. FuzzFactory: domain-specific fuzzing with waypoints

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!