skip to main content
research-article

Verifying systems rules using rule-directed symbolic execution

Published:16 March 2013Publication History
Skip Abstract Section

Abstract

Systems code must obey many rules, such as "opened files must be closed." One approach to verifying rules is static analysis, but this technique cannot infer precise runtime effects of code, often emitting many false positives. An alternative is symbolic execution, a technique that verifies program paths over all inputs up to a bounded size. However, when applied to verify rules, existing symbolic execution systems often blindly explore many redundant program paths while missing relevant ones that may contain bugs.

Our key insight is that only a small portion of paths are relevant to rules, and the rest (majority) of paths are irrelevant and do not need to be verified. Based on this insight, we create WOODPECKER, a new symbolic execution system for effectively checking rules on systems programs. It provides a set of builtin checkers for common rules, and an interface for users to easily check new rules. It directs symbolic execution toward the program paths relevant to a checked rule, and soundly prunes redundant paths, exponentially speeding up symbolic execution. It is designed to be heuristic-agnostic, enabling users to leverage existing powerful search heuristics.

Evaluation on 136 systems programs totaling 545K lines of code, including some of the most widely used programs, shows that, with a time limit of typically just one hour for each verification run, WOODPECKER effectively verifies 28.7% of the program and rule combinations over bounded input, whereas an existing symbolic execution system KLEE verifies only 8.5%. For the remaining combinations, WOODPECKER verifies 4.6 times as many relevant paths as KLEE. With a longer time limit, WOODPECKER verifies much more paths than KLEE, e.g., 17 times as many with a fourhour limit. WOODPECKER detects 113 rule violations, including 10 serious data loss errors with 2 most serious ones already confirmed by the corresponding developers.

References

  1. The LLVM compiler framework. http://llvm.org.Google ScholarGoogle Scholar
  2. STP Constraint Solver. https://sites.google.com/site/stpfastprover/.Google ScholarGoogle Scholar
  3. H. Agrawal and J. R. Horgan. Dynamic program slicing. In Proceedings of the ACM SIGPLAN '90 Conference on Programming Language Design and Implementation (PLDI '90), pages 246--256, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. G. Altekar and I. Stoica. ODR: output-deterministic replay for multicore debugging. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (SOSP '09), pages 193--206, Oct. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. A. Andoni, D. Daniliuc, S. Khurshid, and D. Marinov. Evaluating the "small scope hypothesis". Technical report, MIT CSAIL, 2002.Google ScholarGoogle Scholar
  6. D. Avots, M. Dalton, V. B. Livshits, and M. S. Lam. Improving software security with a C pointer analysis. In Proceedings of the 27th International Conference on Software Engineering (ICSE '05), pages 332--341, May 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. T. Ball and S. K. Rajamani. Automatically validating temporal safety properties of interfaces. In Proceedings of the Eighth International SPIN Workshop on Model Checking of Software (SPIN '01), pages 103--122, May 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. BBC. The life of birds documentary.Google ScholarGoogle Scholar
  9. S. Bucur, V. Ureche, C. Zamfir, and G. Candea. Parallel symbolic execution for automated real-world software testing. In Proceedings of the 6th ACM European Conference on Computer Systems (EUROSYS '11), pages 183--198, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. C. Cadar. Private email communication, Mar. 2012.Google ScholarGoogle Scholar
  11. C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: automatically generating inputs of death. In Proceedings of the 13th ACM conference on Computer and communications security (CCS '06), pages 322--335, Oct.--Nov. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. C. Cadar, D. Dunbar, and D. Engler. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the Eighth Symposium on Operating Systems Design and Implementation (OSDI '08), pages 209--224, Dec. 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. V. Chipounov, V. Kuznetsov, and G. Candea. S2E: a platform for in-vivo multi-path analysis of software systems. In Sixteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '11), pages 265--278, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An empirical study of operating systems errors. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), pages 73--88, Nov. 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. Chow, B. Pfaff, T. Garfinkel, K. Christopher, and M. Rosenblum. Understanding Data Lifetime via Whole System Simulation. In Proceedings of the 13th USENIX Security Symposium, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. J. Chow, B. Pfaff, T. Garfinkel, and M. Rosenblum. Shredding Your Garbage: Reducing Data Lifetime Through Secure Deallocation. In Proceedings of the 14th USENIX Security Symposium, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. E. Clarke and D. Kroening. Hardware verification using ANSI-C programs as a reference. In Proceedings of ASP-DAC 2003, pages 308--311, January 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. symver:fse9A. Coen-Porisini, G. Denaro, C. Ghezzi, and M. Pezzé. Using symbolic execution for verifying safety-critical systems. In Proceedings of the Eighth European Software Engineering Conference held jointly with the Ninth ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-9), pages 142--151, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. P. Collingbourne, C. Cadar, and P. H. Kelly. Symbolic crosschecking of floating-point and SIMD code. In Proceedings of the 6th ACM European Conference on Computer Systems (EUROSYS '11), pages 315--328, Apr. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Coreutils - GNU core utilities. http://www.gnu.org/software/coreutils.Google ScholarGoogle Scholar
  21. M. Costa, M. Castro, L. Zhou, L. Zhang, and M. Peinado. Bouncer: securing software by blocking bad input. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP '07), pages 117--130, Oct. 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. H. Cui, J. Wu, C.-C. Tsai, and J. Yang. Stable deterministic multithreading through schedule memoization. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), Oct. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. H. Cui, J. Wu, J. Gallagher, H. Guo, and J. Yang. Efficient deterministic multithreading through schedule relaxation. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP '11), Oct. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. CVS. http://www.cvshome.org.Google ScholarGoogle Scholar
  25. M. Das, S. Lerner, and M. Seigle. Esp: path-sensitive program verification in polynomial time. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI '02), pages 57--68, June 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. D. Engler, B. Chelf, A. Chou, and S. Hallem. Checking system rules using system-specific, programmer-written compiler extensions. In Proceedings of the Fourth Symposium on Operating Systems Design and Implementation (OSDI '00), Sept. 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Hallem, Chou, and Chelf}sosp:deriveD. Engler, D. Yu Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: A general approach to inferring errors in systems code. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. V. Ganesh and D. L. Dill. A decision procedure for bit-vectors and arrays. In Proceedings of the 19th International Conference On Computer Aided Verification (CAV' 07), pages 519--531, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. X. Ge, K. Taneja, T. Xie, and N. Tillmann. Dyta: dynamic symbolic execution guided with static verification results. In Proceedings of the 33rd International Conference on Software Engineering, ICSE '11, pages 992--994, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Git. http://git-scm.com/.Google ScholarGoogle Scholar
  31. GNU sed. http://www.gnu.org/software/sed.Google ScholarGoogle Scholar
  32. P. Godefroid. Model Checking for Programming Languages using VeriSoft. In Proceedings of the 24th Annual Symposium on Principles of Programming Languages (POPL '97), pages 174--186, Jan. 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. P. Godefroid, N. Klarlund, and K. Sen. Dart: Directed automated random testing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI '05), pages 213--223, June 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. H. Guo, M. Wu, L. Zhou, G. Hu, J. Yang, and L. Zhang. Practical software model checking via dynamic interface reduction. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP '11), Oct. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. B. Hackett and A. Aiken. How is aliasing used in systems software? In Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT '06/FSE-14), pages 69--80, Nov. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. S. Hallem, B. Chelf, Y. Xie, and D. Engler. A system and language for building system-specific, static analyses. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI '02), 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. T. A. Henzinger, R. Jhala, R. Majumdar, , and G. Sutre. Lazy abstraction. In Proceedings of the 29th Annual Symposium on Principles of Programming Languages, pages pp. 58--70, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. D. Jackson and C. A. Damon. Elements of style: Analyzing a software design feature with a counterexample detector. IEEE Trans. Softw. Eng., 22 (7): 484--495, July 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. J. Jaffar, V. Murali, J. A. Navas, and A. E. Santosa. Tracer: A symbolic execution tool for verification. In Proceedings of the 24th international conference on Computer aided verification, CAV'12, July 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. R. Jhala and R. Majumdar. Path slicing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI '05), pages 38--47, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. C. Killian, J. W. Anderson, R. Jhala, and A. Vahdat. Life, death, and the critical transition: Finding liveness bugs in systems code. In Proceedings of the Fourth Symposium on Networked Systems Design and Implementation (NSDI '07), pages 243--256, April 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. J. C. King. A new approach to program testing. In Proceedings of the international conference on Reliable software, pages 228--233, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. V. Kuznetsov, J. Kinder, S. Bucur, and G. Candea. Efficient state merging in symbolic execution. In Proceedings of the ACM SIGPLAN 2012 Conference on Programming Language Design and Implementation (PLDI '12), pages 193--204, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. K.-K. Ma, Y. P. Khoo, J. S. Foster, and M. Hicks. Directed symbolic execution. In The 18th International Static Analysis Symposium, SAS '11, Sept. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. R. Majumdar and R.-G. Xu. Directed test generation using symbolic grammars. In Proceedings of the Seventh European Software Engineering Conference held jointly with the Seventh ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-7), pages 553--556, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. J. S. Metos and J. V. Oldfield. Binary decision diagrams: From abstract representations to physical implementations. In DAC '83: Proceedings of the 20th conference on Design automation, pages 567--570, 1983. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. S. Person, G. Yang, N. Rungta, and S. Khurshid. Directed incremental symbolic execution. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, PLDI '11, pages 504--515, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. C. S. Pasareanu and N. Rungta. Symbolic pathfinder: symbolic execution of java bytecode. In Proceedings of the IEEE/ACM international conference on Automated software engineering, ASE '10, pages 179--180, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. C. S. Pasareanu, N. Rungta, and W. Visser. Symbolic execution with mixed concrete-symbolic solving. In Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA '11, pages 34--44, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. QEMU. http://www.qemu.org.Google ScholarGoogle Scholar
  51. D. P. Quigley, J. Sipek, C. P. Wright, and E. Zadok. UnionFS: User- and Community-oriented Development of a Unification Filesystem. In Proceedings of the 2006 Linux Symposium, volume 2, pages 349--362, Ottawa, Canada, July 2006.Google ScholarGoogle Scholar
  52. D. A. Ramos and D. R. Engler. Practical, low-effort equivalence verification of real code. In Proceedings of the 23rd international conference on Computer aided verification, CAV'11, pages 669--685, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference held jointly with the 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13), pages 263--272, Sept. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. shadow. http://pkg-shadow.alioth.debian.org/.Google ScholarGoogle Scholar
  55. S. Smith. Eat my data: How everybody gets file io wrong. In OSCON 2008, July 2008.Google ScholarGoogle Scholar
  56. tar. http://www.gnu.org/software/tar/.Google ScholarGoogle Scholar
  57. M. Weiser. Program slicing. In Fifth International Conference on Software Engineering, pages 439--449, 1981. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. J. Whaley. bddbddb Project. http://bddbddb.sourceforge.net.Google ScholarGoogle Scholar
  59. J. Whaley and M. S. Lam. Cloning-based context-sensitive pointer alias analysis using binary decision diagrams. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI '04), pages 131--144, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. M. Yabandeh, N. Knezevic, D. Kostic, and V. Kuncak. CrystalBall: Predicting and preventing inconsistencies in deployed distributed systems. In Proceedings of the Sixth Symposium on Networked Systems Design and Implementation (NSDI '09), Apr. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. J. Yang, T. Kremenek, Y. Xie, and D. Engler. MECA: an extensible, expressive system and language for statically checking security properties. In Proceedings of the 10th ACM conference on Computer and communications security (CCS '03), pages 321--334, Oct. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. J. Yang, P. Twohey, D. Engler, and M. Musuvathi. Using model checking to find serious file system errors. In Proceedings of the Sixth Symposium on Operating Systems Design and Implementation (OSDI '04), pages 273--288, Dec. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. J. Yang, C. Sar, and D. Engler. Explode: a lightweight, general system for finding serious storage system errors. In Proceedings of the Seventh Symposium on Operating Systems Design and Implementation (OSDI '06), pages 131--146, Nov. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. J. Yang, C. Sar, P. Twohey, C. Cadar, and D. Engler. Automatically generating malicious disks using symbolic execution. In Proceedings of the 2006 IEEE Symposium on Security and Privacy (SP '06), pages 243--257, May 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. J. Yang, T. Chen, M. Wu, Z. Xu, X. Liu, H. Lin, M. Yang, F. Long, L. Zhang, and L. Zhou. MODIST: Transparent model checking of unmodified distributed systems. In Proceedings of the Sixth Symposium on Networked Systems Design and Implementation (NSDI '09), pages 213--228, Apr. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. C. Zamfir and G. Candea. Execution synthesis: a technique for automated software debugging. In Proceedings of the 5th ACM European Conference on Computer Systems (EUROSYS '10), pages 321--334, Apr. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. X. Zhang and R. Gupta. Cost effective dynamic program slicing. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI '04), pages 94--106, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Verifying systems rules using rule-directed symbolic execution

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGPLAN Notices
            ACM SIGPLAN Notices  Volume 48, Issue 4
            ASPLOS '13
            April 2013
            540 pages
            ISSN:0362-1340
            EISSN:1558-1160
            DOI:10.1145/2499368
            Issue’s Table of Contents
            • cover image ACM Conferences
              ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems
              March 2013
              574 pages
              ISBN:9781450318709
              DOI:10.1145/2451116

            Copyright © 2013 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 16 March 2013

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!