research-article

Directed Greybox Fuzzing

Authors:

Van-Thuan Pham,

Manh-Dung Nguyen,

Abhik RoychoudhuryAuthors Info & Claims

CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security

Pages 2329 - 2344

https://doi.org/10.1145/3133956.3134020

Published: 30 October 2017 Publication History

Abstract

Existing Greybox Fuzzers (GF) cannot be effectively directed, for instance, towards problematic changes or patches, towards critical system calls or dangerous locations, or towards functions in the stack-trace of a reported vulnerability that we wish to reproduce. In this paper, we introduce Directed Greybox Fuzzing (DGF) which generates inputs with the objective of reaching a given set of target program locations efficiently. We develop and evaluate a simulated annealing-based power schedule that gradually assigns more energy to seeds that are closer to the target locations while reducing energy for seeds that are further away. Experiments with our implementation AFLGo demonstrate that DGF outperforms both directed symbolic-execution-based whitebox fuzzing and undirected greybox fuzzing. We show applications of DGF to patch testing and crash reproduction, and discuss the integration of AFLGo into Google's continuous fuzzing platform OSS-Fuzz. Due to its directedness, AFLGo could find 39 bugs in several well-fuzzed, security-critical projects like LibXML2. 17 CVEs were assigned.

Supplemental Material

MP4 File

Download
2374.01 MB

References

[1]

Andrea Arcuri and Lionel Briand. 2014. A Hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability 24, 3 (2014), 219--250.

Digital Library

[2]

Hanno Böck. 2015. Wie man Heartbleed hätte finden können. Golem.de (April 2015). http://www.golem.de/news/fuzzing-wie-man-heartbleedhaette- finden-koennen-1504--113345.html (DE); https://blog.hboeck.de/archives/868- How-Heartbleed-couldve-been-found.html (EN).

[3]

Marcel Böhme, Bruno C. d. S. Oliveira, and Abhik Roychoudhury. 2013. Partition- based Regression Verification. In Proceedings of the 2013 International Conference on Software Engineering (ICSE '13). 302--311.

[4]

Marcel Böhme, Bruno C. d. S. Oliveira, and Abhik Roychoudhury. 2013. Regression Tests to Expose Change Interaction Errors. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2013). 334--344.

Digital Library

[5]

Marcel Böhme and Soumya Paul. 2016. A Probabilistic Analysis of the Efficiency of Automated Software Testing. IEEE Transactions on Software Engineering 42, 4 (April 2016), 345--360.

Digital Library

[6]

Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. Coverage- based Greybox Fuzzing As Markov Chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16). 1032--1043.

Digital Library

[7]

Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-coverage Tests for Complex Systems Programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI'08). 209--224.

Digital Library

[8]

Vitaly Chipounov, Volodymyr Kuznetsov, and George Candea. 2011. S2E: A Platform for In-vivo Multi-path Analysis of Software Systems. In ASPLOS XVI. 265--278.

[9]

Maria Christakis, Peter Müller, and Valentin Wüstholz. 2016. Guiding Dynamic Symbolic Execution Toward Unverified Program Executions. In Proceedings of the 38th International Conference on Software Engineering (ICSE '16). 144--155.

Digital Library

[10]

Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'08/ETAPS'08). 337--340.

Digital Library

[11]

Vijay Ganesh, Tim Leek, and Martin Rinard. 2009. Taint-based Directed Whitebox Fuzzing. In Proceedings of the 31st International Conference on Software Engineering (ICSE '09). 474--484.

Digital Library

[12]

Patrice Godefroid, Michael Y. Levin, and David Molnar. 2012. SAGE: Whitebox Fuzzing for Security Testing. Queue 10, 1, Article 20 (Jan. 2012), 8 pages.

[13]

Patrice Godefroid, Michael Y. Levin, and David A. Molnar. 2008. Automated Whitebox Fuzz Testing. In NDSS '08 (2009-06--18). The Internet Society.

[14]

Istvan Haller, Yuseok Jeon, Hui Peng, Mathias Payer, Cristiano Giuffrida, Her- bert Bos, and Erik van der Kouwe. 2016. TypeSan: Practical Type Confusion Detection. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16). 517--528.

Digital Library

[15]

Istvan Haller, Asia Slowinska, Matthias Neugschwandtner, and Herbert Bos. 2013. Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations. In Proceedings of the 22Nd USENIX Conference on Security (SEC'13). 49--64.

Digital Library

[16]

Mark Harman, Yue Jia, and William B. Langdon. 2011. Strong Higher Order Mutation-based Test Data Generation. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE '11). 212--222.

Digital Library

[17]

Andrew Henderson, Lok Kwong Yan, Xunchao Hu, Aravind Prakash, Heng Yin, Stephen McCamant, undefined, undefined, undefined, and undefined. 2017. DECAF: A Platform-Neutral Whole-System Dynamic Binary Analysis Platform. IEEE Transactions on Software Engineering 43, 2 (2017), 164--184.

Digital Library

[18]

Wei Jin and Alessandro Orso. 2012. BugRedux: Reproducing Field Failures for In- house Debugging. In Proceedings of the 34th International Conference on Software Engineering (ICSE '12). 474--484.

[19]

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 1983. Optimization by simulated annealing. SCIENCE 220, 4598 (1983), 671--680.

[20]

Kin-Keung Ma, Khoo Yit Phang, Jeffrey S. Foster, and Michael Hicks. 2011. Di- rected Symbolic Execution. In Proceedings of the 18th International Conference on Static Analysis (SAS'11). 95--111.

[21]

Paul Dan Marinescu and Cristian Cadar. 2013. KATCH: High-coverage Testing of Software Patches. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2013). 235--245.

Digital Library

[22]

Björn Matthis, Vitalii Avdiienko, Ezekiel Soremekun, Marcel Böhme, and Andreas Zeller. 2017. Detecting Information Flow by Mutating Input Data. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE '17). 1--11.

[23]

Kurt Mehlhorn. 1984. Data structures and algorithms: 1. Searching and sorting. Springer 84 (1984), 90.

[24]

Barton P. Miller, Louis Fredriksen, and Bryan So. 1990. An Empirical Study of the Reliability of UNIX Utilities. Commun. ACM 33, 12 (Dec. 1990), 32--44.

Digital Library

[25]

James Newsome, Dawn Song, James Newsome, and Dawn Song. 2005. Dynamic taint analysis: Automatic detection, analysis, and signature generation of exploit attacks on commodity software. In Proceedings of the 12th Network and Distributed Systems Security Symposium (NDSS).

[26]

Brian S. Pak. 2012. Hybrid Fuzz Testing: Discovering Software Bugs via Fuzzing and Symbolic Execution. Ph.D. Dissertation. Carnegie Mellon University Pittsburgh.

[27]

Suzette Person, Guowei Yang, Neha Rungta, and Sarfraz Khurshid. 2011. Directed Incremental Symbolic Execution. In Proceedings of the 32Nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '11). 504-- 515.

Digital Library

[28]

V. T. Pham, M. Böhme, and A. Roychoudhury. 2016. Model-based whitebox fuzzing for program binaries. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016). 543--553.

Digital Library

[29]

Van-Thuan Pham, Wei Boon Ng, Konstantin Rubinov, and Abhik Roychoudhury. 2015. Hercules: Reproducing Crashes in Real-world Application Binaries. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (ICSE '15). 891--901.

[30]

Dawei Qi, Abhik Roychoudhury, and Zhenkai Liang. 2010. Test Generation to Expose Changes in Evolving Programs. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE '10). 397--406.

Digital Library

[31]

Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar, Cristiano Giuffrida, and Herbert Bos. 2017. VUzzer: Application-aware Evolutionary Fuzzing. In NDSS '17. 1--14.

[32]

Eric F. Rizzi, Sebastian Elbaum, and Matthew B. Dwyer. 2016. On the Techniques We Create, the Tools We Build, and Their Misalignments: A Study of KLEE. In Proceedings of the 38th International Conference on Software Engineering (ICSE '16). 132--143.

Digital Library

[33]

J. Rößler, A. Zeller, G. Fraser, C. Zamfir, and G. Candea. 2013. Reconstructing Core Dumps. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation. 114--123.

Digital Library

[34]

R. Santelices, P. K. Chittimalli, T. Apiwattanapong, A. Orso, and M. J. Harrold. 2008. Test-Suite Augmentation for Evolving Software. In Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE '08). 218--227.

Digital Library

[35]

Konstantin Serebryany, Derek Bruening, Alexander Potapenko, and Dmitry Vyukov. 2012. AddressSanitizer: A Fast Address Sanity Checker. In Proceedings of the 2012 USENIX Conference on Annual Technical Conference (USENIX ATC'12). 28--28.

[36]

S. Sparks, S. Embleton, R. Cunningham, and C. Zou. 2007. Automated Vulner- ability Analysis: Leveraging Control Flow for Evolutionary Input Crafting. In Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007). 477--486.

[37]

E. Stepanov and K. Serebryany. 2015. MemorySanitizer: Fast detector of unini- tialized memory use in C. In 2015 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). 46--55.

[38]

Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2016. Driller: Augmenting Fuzzing Through Selective Symbolic Execution. In NDSS '16. 1--16.

[39]

András Vargha and Harold D. Delaney. 2000. A Critique and Improvement of the "CL" Common Language Effect Size Statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics 25, 2 (2000), 101--132.

[40]

Tielei Wang, Tao Wei, Guofei Gu, and Wei Zou. 2010. TaintScope: A Checksum- Aware Directed Fuzzing Tool for Automatic Software Vulnerability Detection. In Proceedings of the 2010 IEEE Symposium on Security and Privacy (SP '10). 497--512.

Digital Library

[41]

Website. 2017. AFL - Pulling Jpegs out of Thin Air, Michael Zalewski. https: //lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thin-air.html. (2017). Accessed: 2017-05--13.

[42]

Website. 2017. AFL Vulnerability Trophy Case. http://lcamtuf.coredump.cx/afl/ #bugs. (2017). Accessed: 2017-05-13.

[43]

Website. 2017. American Fuzzy Lop (AFL) Fuzzer. http://lcamtuf.coredump.cx/ afl/technical_details.txt. (2017). Accessed: 2017-05--13.

[44]

Website. 2017. Announcing OSS-Fuzz. https://testing.googleblog.com/2016/12/ announcing-oss-fuzz-continuous-fuzzing.html. (2017). Accessed: 2017-05-13.

[45]

Website. 2017. BoringSSL -- Google's fork of OpenSSL. https://boringssl. googlesource.com/. (2017). Accessed: 2017-05-13.

[46]

Website. 2017. Commit to OpenSSL that introduced Heartbleed. https://git. openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=4817504. (2017). Accessed: 2017-05-13.

[47]

Website. 2017. Descriptive statistics of OpenSSL library. https://www.openhub. net/p/openssl. (2017). Accessed: 2017-05--13.

[48]

Website. 2017. Expat XML Parser. https://libexpat.github.io/. (2017). Accessed: 2017-05-13.

[49]

Website. 2017. Heartbleed - A vulnerability in OpenSSL. http://heartbleed.com/. (2017). Accessed: 2017-05-13.

[50]

Website. 2017. Libav Open source audio and video processing tools. https: //libav.org/. (2017). Accessed: 2017-05-13.

[51]

Website. 2017. "libc++abi" C++ Standard Library Support. https://libcxxabi.llvm. org/. (2017). Accessed: 2017-05--13.

[52]

Website. 2017. LibDwarf is parser for the DWARF information used by compilers and debuggers. https://www.prevanders.net/dwarf.html/. (2017). Accessed: 2017-05-13.

[53]

Website. 2017. LibFuzzer: A library for coverage-guided fuzz testing. http: //llvm.org/docs/LibFuzzer.html. (2017). Accessed: 2017-05-13.

[54]

Website. 2017. LibPNG - A library for processing PNG files. http://www.libpng. org/pub/png/libpng.html. (2017). Accessed: 2017-05-13.

[55]

Website. 2017. Libxml2 is the XML C parser and toolkit developed for the Gnome project. http://xmlsoft.org/. (2017). Accessed: 2017-05-13.

[56]

Website. 2017. Ming is a library for generating Macromedia Flash files. http: //www.libming.org/. (2017). Accessed: 2017-05--13.

[57]

Website. 2017. MITRE -- Common Vulnerabilities and Exposures. https://cve. mitre.org/. (2017). Accessed: 2017-05-13.

[58]

Website. 2017. OSS-Fuzz: Continuous Fuzzing Framework for Open-Source Projects. https://github.com/google/oss-fuzz. (2017). Accessed: 2017-05-13.

[59]

Website. 2017. OSS-Fuzz: Five Months Later. https://testing.googleblog.com/ 2017/05/oss-fuzz-five-months-later-and.html. (2017). Accessed: 2017-05-13.

[60]

Website. 2017. Peach Fuzzer Platform. http://www.peachfuzzer.com/products/ peach-platform/. (2017). Accessed: 2017-05--13.

[61]

Website. 2017. Search engine for the internet of things -- devices still vulnerable to Heartbleed. https://www.shodan.io/report/89bnfUyJ. (2017). Accessed: 2017-05-13.

[62]

Website. 2017. SPIKE Fuzzer Platform. http://www.immunitysec.com. (2017). Accessed: 2017-05-13.

[63]

Website. 2017. US National Vulnerability Database. https://nvd.nist.gov/vuln/ search. (2017). Accessed: 2017-05-13.

[64]

Website. 2017. Video Lan Client -- Open-source Media Player. https://www. videolan.org. (2017). Accessed: 2017-05-13.

[65]

Website. 2017. Zzuf: multi-purpose fuzzer. http://caca.zoy.org/wiki/zzuf. (2017). Accessed: 2017-05-13.

[66]

Zhihong Xu, Yunho Kim, Moonzoo Kim, Gregg Rothermel, and Myra B. Cohen. 2010. Directed Test Suite Augmentation: Techniques and Tradeoffs. In Proceedings of the Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE '10). 257--266.

Digital Library

Cited By

Du CXu GGuo YWang ZYu W(2024)A Novel Seed Generation Approach for Vulnerability Mining Based on Generative Adversarial Networks and Attention MechanismsMathematics10.3390/math1205074512:5(745)Online publication date: 1-Mar-2024
https://doi.org/10.3390/math12050745
Koffi KKampourakis VSong JKolias CIvans R(2024)StructuredFuzzer: Fuzzing Structured Text-Based Control Logic ApplicationsElectronics10.3390/electronics1313247513:13(2475)Online publication date: 25-Jun-2024
https://doi.org/10.3390/electronics13132475
Zeng QXiong DWu ZQian KWang YSu Y(2024)WolfFuzz: A Dynamic, Adaptive, and Directed Greybox FuzzerElectronics10.3390/electronics1311209613:11(2096)Online publication date: 28-May-2024
https://doi.org/10.3390/electronics13112096
Show More Cited By

Index Terms

Directed Greybox Fuzzing
1. Security and privacy
  1. Systems security
    1. Vulnerability management
      1. Penetration testing
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Sequence coverage directed greybox fuzzing
ICPC '19: Proceedings of the 27th International Conference on Program Comprehension

Existing directed fuzzers are not efficient enough. Directed symbolic-execution-based whitebox fuzzers, e.g. BugRedux, spend lots of time on heavyweight program analysis and constraints solving at runtime. Directed greybox fuzzers, such as AFLGo, ...
TargetFuzz: Using DARTs to Guide Directed Greybox Fuzzers
ASIA CCS '22: Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security

Software development is a continuous and incremental process. Developers continuously improve their software in small batches rather than in one large batch. The high frequency of small batches makes it essential to use effective testing methods that ...
Guiding Greybox Fuzzing with Mutation Testing
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Greybox fuzzing and mutation testing are two popular but mostly independent fields of software testing research that have so far had limited overlap. Greybox fuzzing, generally geared towards searching for new bugs, predominantly uses code coverage ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security

October 2017

2682 pages

ISBN:9781450349468

DOI:10.1145/3133956

General Chair:
Bhavani Thuraisingham
The University of Texas at Dallas, USA
,
Program Chairs:
David Evans
University of Virginia
,
Tal Malkin
Columbia University
,
Dongyan Xu
Purdue University

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Research Foundation Singapore

Conference

CCS '17

Sponsor:

SIGSAC

CCS '17: 2017 ACM SIGSAC Conference on Computer and Communications Security

October 30 - November 3, 2017

Texas, Dallas, USA

Acceptance Rates

CCS '17 Paper Acceptance Rate 151 of 836 submissions, 18%;

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '24

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 14 - 18, 2024

Salt Lake City , UT , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

458
Total Citations
View Citations
3,852
Total Downloads

Downloads (Last 12 months)840
Downloads (Last 6 weeks)52

Reflects downloads up to 23 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Du CXu GGuo YWang ZYu W(2024)A Novel Seed Generation Approach for Vulnerability Mining Based on Generative Adversarial Networks and Attention MechanismsMathematics10.3390/math1205074512:5(745)Online publication date: 1-Mar-2024
https://doi.org/10.3390/math12050745
Koffi KKampourakis VSong JKolias CIvans R(2024)StructuredFuzzer: Fuzzing Structured Text-Based Control Logic ApplicationsElectronics10.3390/electronics1313247513:13(2475)Online publication date: 25-Jun-2024
https://doi.org/10.3390/electronics13132475
Zeng QXiong DWu ZQian KWang YSu Y(2024)WolfFuzz: A Dynamic, Adaptive, and Directed Greybox FuzzerElectronics10.3390/electronics1311209613:11(2096)Online publication date: 28-May-2024
https://doi.org/10.3390/electronics13112096
Du CGuo YFeng YZheng S(2024)HotCFuzz: Enhancing Vulnerability Detection through Fuzzing and Hotspot Code Coverage AnalysisElectronics10.3390/electronics1310190913:10(1909)Online publication date: 13-May-2024
https://doi.org/10.3390/electronics13101909
Shen QMeng GChen K(2024)Revealing the exploitability of heap overflow through PoC analysisCybersecurity10.1186/s42400-024-00244-67:1Online publication date: 18-Jul-2024
https://doi.org/10.1186/s42400-024-00244-6
Yang JLiu CFang B(2024)Adaptive scheduling-based fine-grained greybox fuzzing for cloud-native applicationsJournal of Cloud Computing10.1186/s13677-024-00681-113:1Online publication date: 26-Jun-2024
https://doi.org/10.1186/s13677-024-00681-1
Sharma ACadar CMetzman JBöhme MNoller YSzekeres L(2024)Effective Fuzzing within CI/CD Pipelines (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685534(52-60)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685534
Huang MLemieux CBöhme MNoller YSzekeres L(2024)Directed or Undirected: Investigating Fuzzing Strategies in a CI/CD Setup (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685532(33-41)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685532
Ye MNan YDai HYang SLuo XZheng Z(2024)FunFuzz: A Function-Oriented Fuzzer for Smart Contract Vulnerability Detection with High Effectiveness and EfficiencyACM Transactions on Software Engineering and Methodology10.1145/367472533:7(1-20)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674725
Wan YWang WSun JTian D(2024)Directed Fuzzing Based on Bottleneck DetectionProceedings of the 2024 5th International Conference on Computing, Networks and Internet of Things10.1145/3670105.3670111(32-37)Online publication date: 24-May-2024
https://dl.acm.org/doi/10.1145/3670105.3670111
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents