skip to main content

Symbolic execution for randomized programs

Published:31 October 2022Publication History
Skip Abstract Section

Abstract

We propose a symbolic execution method for programs that can draw random samples. In contrast to existing work, our method can verify randomized programs with unknown inputs and can prove probabilistic properties that universally quantify over all possible inputs. Our technique augments standard symbolic execution with a new class of probabilistic symbolic variables, which represent the results of random draws, and computes symbolic expressions representing the probability of taking individual paths. We implement our method on top of the KLEE symbolic execution engine alongside multiple optimizations and use it to prove properties about probabilities and expected values for a range of challenging case studies written in C++, including Freivalds’ algorithm, randomized quicksort, and a randomized property-testing algorithm for monotonicity. We evaluate our method against Psi, an exact probabilistic symbolic inference engine, and Storm, a probabilistic model checker, and show that our method significantly outperforms both tools.

References

  1. Aws Albarghouthi, Loris D’Antoni, Samuel Drews, and Aditya V. Nori. 2017. FairSquare: Probabilistic Verification of Program Fairness. Proceedings of the ACM on Programming Languages, 1, OOPSLA (2017), Article 80, https://doi.org/10.1145/3133904 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Aws Albarghouthi and Justin Hsu. 2018. Constraint-Based Synthesis of Coupling Proofs. In International Conference on Computer Aided Verification (CAV), Oxford, England. https://doi.org/10.1007/978-3-319-96145-3_18 arxiv:1804.04052. Google ScholarGoogle ScholarCross RefCross Ref
  3. Aws Albarghouthi and Justin Hsu. 2018. Synthesizing Coupling Proofs of Differential Privacy. Proceedings of the ACM on Programming Languages, 2, POPL (2018), Article 58, Jan., https://doi.org/10.1145/3158146 arxiv:1709.05361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Christel Baier, Edmund M. Clarke, Vasiliki Hartonas-Garmhausen, Marta Z. Kwiatkowska, and Mark Ryan. 1997. Symbolic Model Checking for Probabilistic Processes. In International Colloquium on Automata, Languages and Programming (ICALP), Bologna, Italy (Lecture Notes in Computer Science, Vol. 1256). Springer, 430–440. https://doi.org/10.1007/3-540-63165-8_199 Google ScholarGoogle ScholarCross RefCross Ref
  5. Christel Baier, Luca de Alfaro, Vojtech Forejt, and Marta Kwiatkowska. 2018. Model Checking Probabilistic Systems. In Handbook of Model Checking. Springer-Verlag, 963–999. https://doi.org/10.1007/978-3-319-10575-8_28 Google ScholarGoogle ScholarCross RefCross Ref
  6. Jialu Bao, Nitesh Trivedi, Drashti Pathak, Justin Hsu, and Subhajit Roy. 2022. Data-Driven Invariant Learning for Probabilistic Programs. In International Conference on Computer Aided Verification (CAV), Haifa, Israel. https://doi.org/10.1007/978-3-031-13185-1_3 arxiv:2106.05421. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2016. The Satisfiability Modulo Theories Library (SMT-LIB). www.SMT-LIB.org. Google ScholarGoogle Scholar
  8. Gilles Barthe, Rohit Chadha, Paul Krogmeier, A. Prasad Sistla, and Mahesh Viswanathan. 2021. Deciding Accuracy of Differential Privacy Schemes. Proceedings of the ACM on Programming Languages, 5, POPL (2021), Article 8, Jan., https://doi.org/10.1145/3434289 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. 2020. Foundations of Probabilistic Programming Languages, Gilles Barthe, Joost-Pieter Katoen, and Alexandra Silva (Eds.). Cambridge University Press. https://doi.org/10.1017/9781108770750.006 Google ScholarGoogle ScholarCross RefCross Ref
  10. Ezio Bartocci, Laura Kovács, and Miroslav Stankovic. 2019. Automatic Generation of Moment-Based Invariants for Prob-Solvable Loops. In International Symposium on Automated Technology for Verification and Analysis (ATVA), Taipei City, Taiwan (Lecture Notes in Computer Science, Vol. 11781). Springer-Verlag, 255–276. https://doi.org/10.1007/978-3-030-31784-3_15 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Ezio Bartocci, Laura Kovács, and Miroslav Stankovic. 2020. Mora – Automatic Generation of Moment-Based Invariants. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Dublin, Ireland (Lecture Notes in Computer Science, Vol. 12078). Springer-Verlag, 492–498. https://doi.org/10.1007/978-3-030-45190-5_28 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. 2010. A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World. Commun. ACM, 53, 2 (2010), Feb., 66–75. issn:0001-0782 https://doi.org/10.1145/1646353.1646374 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Burton H. Bloom. 1970. Space/Time Trade-offs in Hash Coding with Allowable Errors. Commun. ACM, 13, 7 (1970), July, 422–426. issn:0001-0782 https://doi.org/10.1145/362686.362692 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Mateus Borges, Antonio Filieri, Marcelo d’Amorim, Corina S. Păsăreanu, and Willem Visser. 2014. Compositional Solution Space Quantification for Probabilistic Software Analysis. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Edinburgh, Scotland. 123–132. https://doi.org/10.1145/2666356.2594329 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), San Diego, California. 209–224. http://www.usenix.org/events/osdi08/tech/full_papers/cadar/cadar.pdf Google ScholarGoogle Scholar
  16. Michael Carbin, Deokhwan Kim, Sasa Misailovic, and Martin C Rinard. 2012. Proving Acceptability Properties of Relaxed Nondeterministic Approximate Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Beijing, China. 169–180. https://doi.org/10.1145/2254064.2254086 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic Program Analysis with Martingales. In International Conference on Computer Aided Verification (CAV), Saint Petersburg, Russia (Lecture Notes in Computer Science, Vol. 8044). 511–526. https://doi.org/10.1007/978-3-642-39799-8_34 Google ScholarGoogle ScholarCross RefCross Ref
  18. Krishnendu Chatterjee, Hongfei Fu, Petr Novotný, and Rouzbeh Hasheminezhad. 2016. Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Saint Petersburg, Florida. 327–342. isbn:978-1-4503-3549-2 https://doi.org/10.1145/2837614.2837639 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Bihuan Chen, Yang Liu, and Wei Le. 2016. Generating Performance Distributions via Probabilistic Symbolic Execution. In International Conference on Software Engineering (ICSE), Austin, Texas. 49–60. isbn:9781450339001 https://doi.org/10.1145/2884781.2884794 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Guillaume Claret, Sriram K. Rajamani, Aditya V. Nori, Andrew D. Gordon, and Johannes Borgström. 2013. Bayesian Inference Using Data Flow Analysis. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), Saint Petersburg, Russia. 92–102. https://doi.org/10.1145/2491411.2491423 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Graham Cormode and S. Muthukrishnan. 2004. An Improved Data Stream Summary: The Count-Min Sketch and Its Applications. In Latin American Symposium on Theoretical Informatics (LATIN), Buenos Aires, Argentina (Lecture Notes in Computer Science, Vol. 2976). Springer-Verlag, 29–38. https://doi.org/10.1007/978-3-540-24698-5_7 Google ScholarGoogle ScholarCross RefCross Ref
  22. Patrick Cousot and Michael Monerau. 2012. Probabilistic Abstract Interpretation. In European Symposium on Programming (ESOP), Tallinn, Estonia (Lecture Notes in Computer Science, Vol. 7211). Springer-Verlag, 169–193. https://doi.org/10.1007/978-3-642-28869-2_9 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Budapest, Hungary (Lecture Notes in Computer Science, Vol. 4963). Springer-Verlag, 337–340. isbn:978-3-540-78800-3 https://doi.org/10.1007/978-3-540-78800-3_24 Google ScholarGoogle ScholarCross RefCross Ref
  24. Gian Pietro Farina, Stephen Chong, and Marco Gaboardi. 2019. Relational Symbolic Execution. In ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP), Porto, Portugal. Article 10, https://doi.org/10.1145/3354166.3354175 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gian Pietro Farina, Stephen Chong, and Marco Gaboardi. 2021. Coupled Relational Symbolic Execution for Differential Privacy. In European Symposium on Programming (ESOP), Luxembourg City, Luxembourg (Lecture Notes in Computer Science, Vol. 12648). Springer-Verlag, 207–233. https://doi.org/10.1007/978-3-030-72019-3_8 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Antonio Filieri, Corina S. Păsăreanu, and Willem Visser. 2013. Reliability Analysis in Symbolic Pathfinder. In International Conference on Software Engineering (ICSE), San Francisco, California. 622–631. https://doi.org/10.1109/ICSE.2013.6606608 Google ScholarGoogle ScholarCross RefCross Ref
  27. Antonio Filieri, Corina S. Păsăreanu, and Guowei Yang. 2015. Quantification of Software Changes through Probabilistic Symbolic Execution. In IEEE/ACM International Conference on Automated Software Engineering (ASE), Lincoln, Nebraska. 703–708. https://doi.org/10.1109/ASE.2015.78 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Rūsiņš Freivalds. 1977. Probabilistic Machines Can Use Less Running Time. In IFIP World Congress, Toronto, Canada. North-Holland, 839–842. Google ScholarGoogle Scholar
  29. Timon Gehr, Sasa Misailovic, and Martin Vechev. 2016. PSI: Exact Symbolic Inference for Probabilistic Programs. In International Conference on Computer Aided Verification (CAV), Toronto, Ontario (Lecture Notes in Computer Science, Vol. 9779). Springer-Verlag, 62–83. https://doi.org/10.1007/978-3-319-41528-4_4 Google ScholarGoogle ScholarCross RefCross Ref
  30. Timon Gehr, Samuel Steffen, and Martin Vechev. 2020. λ PSI: Exact Inference for Higher-Order Probabilistic Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), London, England. 883–897. https://doi.org/10.1145/3385412.3386006 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jaco Geldenhuys, Matthew B. Dwyer, and Willem Visser. 2012. Probabilistic Symbolic Execution. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), Minneapolis, Minnesota. 166–176. https://doi.org/10.1145/2338965.2336773 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Oded Goldreich. 2017. Introduction to Property Testing. Cambridge University Press. isbn:978-1-107-19405-2 https://doi.org/10.1017/9781108135252 Google ScholarGoogle ScholarCross RefCross Ref
  33. Friedrich Gretz, Joost-Pieter Katoen, and Annabelle McIver. 2013. Prinsys—On a Quest for Probabilistic Loop Invariants. In International Conference on Quantitative Evaluation of Systems (QEST), Buenos Aires, Argentina (Lecture Notes in Computer Science, Vol. 8054). Springer-Verlag, 193–208. https://doi.org/10.1007/978-3-642-40196-1_17 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Friedrich Gretz, Joost-Pieter Katoen, and Annabelle McIver. 2014. Operational versus weakest pre-expectation semantics for the probabilistic guarded command language. Performance Evaluation, 73 (2014), 110–132. https://doi.org/10.1016/j.peva.2013.11.004 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Christian Hensel, Sebastian Junges, Joost-Pieter Katoen, Tim Quatmann, and Matthias Volk. 2022. The probabilistic model checker Storm. International Journal on Software Tools for Technology Transfer, 24, 4 (2022), 589–610. https://doi.org/10.1007/s10009-021-00633-z arxiv:2002.07080. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. C. A. R. Hoare. 1961. Algorithms 63-64: partition and quicksort. Commun. ACM, 4, 7 (1961), July, 321. https://doi.org/10.1145/366622.366642 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Steven Holtzen, Guy Van den Broeck, and Todd Millstein. 2020. Scaling Exact Inference for Discrete Probabilistic Programs. Proceedings of the ACM on Programming Languages, 4, OOPSLA (2020), Article 140, Nov., https://doi.org/10.1145/3428208 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Keyur Joshi, Vimuth Fernando, and Sasa Misailovic. 2019. Statistical Algorithmic Profiling for Randomized Approximate Programs. In International Conference on Software Engineering (ICSE), Montréal, Québec. 608–618. https://doi.org/10.1109/ICSE.2019.00071 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Qiao Kang, Jiarong Xing, Yiming Qiu, and Ang Chen. 2021. Probabilistic Profiling of Stateful Data Planes for Adversarial Testing. In International Conference on Architectural Support for Programming Langauages and Operating Systems (ASPLOS). 286–301. https://doi.org/10.1145/3445814.3446764 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Timotej Kapus, Martin Nowack, and Cristian Cadar. 2019. Constraints in Dynamic Symbolic Execution: Bitvectors or Integers? In International Conference on Tests and Proofs (TAP), Porto, Portugal (Lecture Notes in Computer Science, Vol. 11823). Springer-Verlag, 41–54. https://doi.org/10.1007/978-3-030-31157-5_3 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Adam Kiezun, Vijay Ganesh, Shay Artzi, Philip J. Guo, Pieter Hooimeijer, and Michael D. Ernst. 2012. HAMPI: A Solver for Word Equations over Strings, Regular Expressions, and Context-Free Grammars. ACM Transactions on Software Engineering and Methodology, 21, 4 (2012), Article 25, Nov., https://doi.org/10.1145/2377656.2377662 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. James C. King. 1976. Symbolic Execution and Program Testing. Commun. ACM, 19, 7 (1976), July, 385–394. https://doi.org/10.1145/360248.360252 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Dexter Kozen. 1985. A Probabilistic PDL. J. Comput. System Sci., 30, 2 (1985), 162–178. https://doi.org/10.1016/0022-0000(85)90012-1 Google ScholarGoogle ScholarCross RefCross Ref
  44. Marta Kwiatkowska, Gethin Norman, and David Parker. 2011. PRISM 4.0: Verification of Probabilistic Real-Time Systems. In International Conference on Computer Aided Verification (CAV), Snowbird, Utah (Lecture Notes in Computer Science, Vol. 6806). Springer-Verlag, 585–591. https://doi.org/10.1007/978-3-642-22110-1_47 Google ScholarGoogle ScholarCross RefCross Ref
  45. Marcel Moosbrugger, Ezio Bartocci, Joost-Pieter Katoen, and Laura Kovács. 2021. Automated Termination Analysis of Polynomial Probabilistic Programs. In European Symposium on Programming (ESOP), Luxembourg City, Luxembourg (Lecture Notes in Computer Science, Vol. 12648). Springer-Verlag, 491–518. https://doi.org/10.1007/978-3-030-72019-3_18 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. ACM Transactions on Programming Languages and Systems, 18, 3 (1996), 325–353. https://doi.org/10.1145/229542.229547 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Adrian Sampson, Pavel Panchekha, Todd Mytkowicz, Kathryn S. McKinley, Dan Grossman, and Luis Ceze. 2014. Expressing and Verifying Probabilistic Assertions. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Edinburgh, Scotland. 112–122. https://doi.org/10.1145/2594291.2594294 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. 2013. Static Analysis for Probabilistic Programs: Inferring Whole Program Properties from Finitely Many Paths. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Seattle, Washington. 447–458. https://doi.org/10.1145/2499370.2462179 Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Raimondas Sasnauskas, Oscar Soria Dustmann, Benjamin Lucien Kaminski, Klaus Wehrle, Carsten Weise, and Stefan Kowalewski. 2011. Scalable Symbolic Execution of Distributed Systems. In International Conference on Distributed Computing Systems (ICDCS), Minneapolis, Minnesota. 333–342. https://doi.org/10.1109/ICDCS.2011.28 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Raimondas Sasnauskas, Olaf Landsiedel, Muhammad Hamad Alizai, Carsten Weise, Stefan Kowalewski, and Klaus Wehrle. 2010. KleeNet: Discovering Insidious Interaction Bugs in Wireless Sensor Networks before Deployment. In ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), Stockholm, Sweden. 186–196. https://doi.org/10.1145/1791212.1791235 Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Steve Selvin. 1975. Letters to the Editor. The American Statistician, 29, 1 (1975), 67–71. https://doi.org/10.1080/00031305.1975.10479121 arxiv:https://doi.org/10.1080/00031305.1975.10479121. Google ScholarGoogle ScholarCross RefCross Ref
  52. Calvin Smith, Justin Hsu, and Aws Albarghouthi. 2019. Trace Abstraction modulo Probability. Proceedings of the ACM on Programming Languages, 3, POPL (2019), Article 39, Jan., https://doi.org/10.1145/3290352 arxiv:1810.12396. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Zachary Susag, Sumit Lahiri, Justin Hsu, and Subhajit Roy. 2022. Artifact for Symbolic Execution for Randomized Programs. https://doi.org/10.5281/zenodo.7061819 Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Zachary Susag, Sumit Lahiri, Justin Hsu, and Subhajit Roy. 2022. Symbolic Execution for Randomized Programs. https://doi.org/10.48550/arXiv.2209.08046 arxiv:2209.08046. Google ScholarGoogle Scholar
  55. Jeffrey Scott Vitter. 1985. Random Sampling with a Reservoir. ACM Trans. Math. Software, 11, 1 (1985), March, 37–57. https://doi.org/10.1145/3147.3165 Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Di Wang, Jan Hoffmann, and Thomas W. Reps. 2018. PMAF: An Algebraic Framework for Static Analysis of Probabilistic Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Philadelphia, Pennsylvania. 513–528. https://doi.org/10.1145/3192366.3192408 Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Di Wang, Jan Hoffmann, and Thomas W. Reps. 2021. Central Moment Analysis for Cost Accumulators in Probabilistic Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). 559–573. https://doi.org/10.1145/3453483.3454062 arxiv:2001.10150. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Symbolic execution for randomized programs

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Article Metrics

              • Downloads (Last 12 months)247
              • Downloads (Last 6 weeks)34

              Other Metrics

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!