skip to main content

Finding real bugs in big programs with incorrectness logic

Published:29 April 2022Publication History
Skip Abstract Section

Abstract

Incorrectness Logic (IL) has recently been advanced as a logical theory for compositionally proving the presence of bugs—dual to Hoare Logic, which is used to compositionally prove their absence. Though IL was motivated in large part by the aim of providing a logical foundation for bug-catching program analyses, it has remained an open question: is IL useful only retrospectively (to explain existing analyses), or can it actually be useful in developing new analyses which can catch real bugs in big programs?

In this work, we develop Pulse-X, a new, automatic program analysis for catching memory errors, based on ISL, a recent synthesis of IL and separation logic. Using Pulse-X, we have found 15 new real bugs in OpenSSL, which we have reported to OpenSSL maintainers and have since been fixed. In order not to be overwhelmed with potential but false error reports, we develop a compositional bug-reporting criterion based on a distinction between latent and manifest errors, which references the under-approximate ISL abstractions computed by Pulse-X, and we investigate the fix rate resulting from application of this criterion. Finally, to probe the potential practicality of our bug-finding method, we conduct a comparison to Infer, a widely used analyzer which has proven useful in industrial engineering practice.

References

  1. Ali Asadi, Krishnendu Chatterjee, Hongfei Fu, Amir Kafshdar Goharshady, and Mohammad Mahdavi. 2021. Polynomial reachability witnesses via Stellensätze. In PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 20211, Stephen N. Freund and Eran Yahav (Eds.). ACM, 772–787. https://doi.org/10.1145/3453483.3454076 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Thomas Ball, Orna Kupferman, and Greta Yorsh. 2005. Abstraction for Falsification. In Computer Aided Verification, 17th International Conference, CAV 2005, Edinburgh, Scotland, UK, July 6-10, 2005, Proceedings, Kousha Etessami and Sriram K. Rajamani (Eds.) (Lecture Notes in Computer Science, Vol. 3576). Springer, 67–81. https://doi.org/10.1007/11513988_8 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Fraser Brown, Deian Stefan, and Dawson Engler. 2020. Sys: A Static/Symbolic Tool for Finding Good Bugs in Good (Browser) Code. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 199–216. isbn:978-1-939133-17-5 https://www.usenix.org/conference/usenixsecurity20/presentation/brownGoogle ScholarGoogle Scholar
  4. Cristiano Calcagno, Dino Distefano, Peter O’Hearn, and Hongseok Yang. 2009. Compositional Shape Analysis by Means of Bi-Abduction. In Proceedings of the 36th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’09). Association for Computing Machinery, New York, NY, USA. 289–300. isbn:9781605583792 https://doi.org/10.1145/1480881.1480917 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cristiano Calcagno, Dino Distefano, Peter W. O’Hearn, and Hongseok Yang. 2011. Compositional Shape Analysis by Means of Bi-Abduction. J. ACM, 58, 6, Article 26, 66 pages. issn:0004-5411 https://doi.org/10.1145/2049697.2049700 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Ankush Das, Shuvendu K. Lahiri, Akash Lal, and Yi Li. 2015. Angelic Verification: Precise Verification Modulo Unknowns. In Computer Aided Verification - 27th International Conference, CAV 2015, San Francisco, CA, USA, July 18-24, 2015, Proceedings, Part I, Daniel Kroening and Corina S. Pasareanu (Eds.) (Lecture Notes in Computer Science, Vol. 9206). Springer, 324–342. https://doi.org/10.1007/978-3-319-21690-4_19 Google ScholarGoogle ScholarCross RefCross Ref
  7. Edsko de Vries and Vasileios Koutavas. 2011. Reverse Hoare Logic. In Software Engineering and Formal Methods, Gilles Barthe, Alberto Pardo, and Gerardo Schneider (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 155–171. isbn:978-3-642-24690-6Google ScholarGoogle Scholar
  8. Isil Dillig, Thomas Dillig, and Alex Aiken. 2012. Automated Error Diagnosis Using Abductive Inference. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’12). Association for Computing Machinery, New York, NY, USA. 181–192. isbn:9781450312059 https://doi.org/10.1145/2254064.2254087 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Dino Distefano, Manuel Fähndrich, Francesco Logozzo, and Peter W. O’Hearn. 2019. Scaling Static Analyses at Facebook. Commun. ACM, 62, 8 (2019), jul, 62–70. issn:0001-0782 https://doi.org/10.1145/3338112 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Dino Distefano, Peter W. O’Hearn, and Hongseok Yang. 2006. A Local Shape Analysis Based on Separation Logic. In Tools and Algorithms for the Construction and Analysis of Systems, 12th International Conference, TACAS 2006 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2006, Vienna, Austria, March 25 - April 2, 2006, Proceedings, Holger Hermanns and Jens Palsberg (Eds.) (Lecture Notes in Computer Science, Vol. 3920). Springer, 287–302. https://doi.org/10.1007/11691372_19 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Facebook. 2021. https://fbinfer.com/Google ScholarGoogle Scholar
  12. José Fragoso Santos, Petar Maksimović, Sacha-Élie Ayoun, and Philippa Gardner. 2020. Gillian, Part i: A Multi-Language Platform for Symbolic Execution. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020). Association for Computing Machinery, New York, NY, USA. 927–942. isbn:9781450376136 https://doi.org/10.1145/3385412.3386014 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Patrice Godefroid. 2007. Compositional Dynamic Test Generation. In Proceedings of the 34th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’07). Association for Computing Machinery, New York, NY, USA. 47–54. isbn:1595935754 https://doi.org/10.1145/1190216.1190226 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Patrice Godefroid, Aditya V. Nori, Sriram K. Rajamani, and SaiDeep Tetali. 2010. Compositional may-must program analysis: unleashing the power of alternation. In Proceedings of the 37th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2010, Madrid, Spain, January 17-23, 2010, Manuel V. Hermenegildo and Jens Palsberg (Eds.). ACM, 43–56.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mark Harman, Yue Jia, and Yuanyuan Zhang. 2015. Achievements, Open Problems and Challenges for Search Based Software Testing. In 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, April 13-17, 2015. IEEE Computer Society, 1–12. https://doi.org/10.1109/ICST.2015.7102580 Google ScholarGoogle ScholarCross RefCross Ref
  16. Kyriakos K. Ispoglou, Daniel Austin, Vishwath Mohan, and Mathias Payer. 2020. FuzzGen: Automatic Fuzzer Generation. In 29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020, Srdjan Capkun and Franziska Roesner (Eds.). USENIX Association, 2271–2287. https://www.usenix.org/conference/usenixsecurity20/presentation/ispoglouGoogle ScholarGoogle Scholar
  17. Ranjit Jhala and Rupak Majumdar. 2009. Software Model Checking. ACM Comput. Surv., 41, 4 (2009), Article 21, Oct., 54 pages. issn:0360-0300 https://doi.org/10.1145/1592434.1592438 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sarfraz Khurshid, Corina S. Păsăreanu, and Willem Visser. 2003. Generalized Symbolic Execution for Model Checking and Testing. In Proceedings of the 9th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’03). Springer-Verlag, Berlin, Heidelberg. 553–568. isbn:3540008985Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Quang Loc Le, Cristian Gherghina, Shengchao Qin, and Wei-Ngan Chin. 2014. Shape Analysis via Second-Order Bi-Abduction. In Computer Aided Verification, Armin Biere and Roderick Bloem (Eds.). Springer International Publishing, Cham. 52–68. isbn:978-3-319-08867-9Google ScholarGoogle Scholar
  20. Quang Loc Le, Azalea Raad, Jules Villard, Josh Berdine, Derek Dreyer, and Peter O’Hearn. 2022. Artifact and Appendix of Finding Real Bugs in Big Programs with Incorrectness Logic (supplementary material). https://doi.org/10.5281/zenodo.6342311 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Bernhard Möller, Peter O’Hearn, and Tony Hoare. 2021. On Algebra of Program Correctness and Incorrectness. In Relational and Algebraic Methods in Computer Science - 19th International Conference, RAMiCS 2021.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Peter W. OHearn. 2007. Resources, Concurrency, and Local Reasoning. Theor. Comput. Sci., 375, 1-3 (2007), apr, 271–307. issn:0304-3975 https://doi.org/10.1016/j.tcs.2006.12.035 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Peter W. O’Hearn. 2015. https://mailing.openssl.dev.narkive.com/2DbkkYzD/openssl-org-3403-null-dereference-and-memory-leak-reports-for-openssl-1-0-1h-from-facebook-s-inferGoogle ScholarGoogle Scholar
  24. Peter W. O’Hearn. 2019. Incorrectness Logic. Proc. ACM Program. Lang., 4, POPL (2019), Article 10, Dec., 32 pages. https://doi.org/10.1145/3371078 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Peter W. O’Hearn, John C. Reynolds, and Hongseok Yang. 2001. Local Reasoning about Programs that Alter Data Structures. In Computer Science Logic, 15th International Workshop, CSL 2001. 10th Annual Conference of the EACSL, Paris, France, September 10-13, 2001, Proceedings. 1–19. https://doi.org/10.1007/3-540-44802-0_1 Google ScholarGoogle ScholarCross RefCross Ref
  26. J.A. Pendergrass, S.C. Lee, and C.D. McConnell. 2013. Theory and Practice of Mechanized Software Analysis. In Johns Hopkins APL Technical Digest, Volume 32, Number 2. 499–508.Google ScholarGoogle Scholar
  27. Long H. Pham, Quang Loc Le, Quoc-Sang Phan, Jun Sun, and Shengchao Qin. 2019. Enhancing Symbolic Execution of Heap-Based Programs with Separation Logic for Test Input Generation. In Automated Technology for Verification and Analysis, Yu-Fang Chen, Chih-Hong Cheng, and Javier Esparza (Eds.). Springer International Publishing, Cham. 209–227. isbn:978-3-030-31784-3Google ScholarGoogle Scholar
  28. Azalea Raad, Josh Berdine, Hoang-Hai Dang, Derek Dreyer, Peter O’Hearn, and Jules Villard. 2020. Local Reasoning About the Presence of Bugs: Incorrectness Separation Logic. In Computer Aided Verification: 32nd International Conference, CAV 2020, Los Angeles, CA, USA, July 21-24, 2020, Proceedings, Part II. Springer-Verlag, Berlin, Heidelberg. 225–252. isbn:978-3-030-53290-1 https://doi.org/10.1007/978-3-030-53291-8_14 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Azalea Raad, Josh Berdine, Derek Dreyer, and Peter O’Hearn. 2022. Concurrent Incorrectness Separation Logic. Proc. ACM Program. Lang., 6, POPL (2022), Conditionally accepted.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. David A. Ramos and Dawson Engler. 2015. Under-Constrained Symbolic Execution: Correctness Checking for Real Code. In 24th USENIX Security Symposium (USENIX Security 15). USENIX Association, Washington, D.C.. 49–64. isbn:978-1-939133-11-3 https://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/ramosGoogle ScholarGoogle Scholar
  31. Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from Building Static Analysis Tools at Google. Commun. ACM, 61, 4 (2018), March, 58–66. issn:0001-0782 https://doi.org/10.1145/3188720 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Finding real bugs in big programs with incorrectness logic

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!