Abstract
We propose a symbolic execution method for programs that can draw random samples. In contrast to existing work, our method can verify randomized programs with unknown inputs and can prove probabilistic properties that universally quantify over all possible inputs. Our technique augments standard symbolic execution with a new class of probabilistic symbolic variables, which represent the results of random draws, and computes symbolic expressions representing the probability of taking individual paths. We implement our method on top of the KLEE symbolic execution engine alongside multiple optimizations and use it to prove properties about probabilities and expected values for a range of challenging case studies written in C++, including Freivalds’ algorithm, randomized quicksort, and a randomized property-testing algorithm for monotonicity. We evaluate our method against Psi, an exact probabilistic symbolic inference engine, and Storm, a probabilistic model checker, and show that our method significantly outperforms both tools.
- Aws Albarghouthi, Loris D’Antoni, Samuel Drews, and Aditya V. Nori. 2017. FairSquare: Probabilistic Verification of Program Fairness. Proceedings of the ACM on Programming Languages, 1, OOPSLA (2017), Article 80, https://doi.org/10.1145/3133904
Google Scholar
Digital Library
- Aws Albarghouthi and Justin Hsu. 2018. Constraint-Based Synthesis of Coupling Proofs. In International Conference on Computer Aided Verification (CAV), Oxford, England. https://doi.org/10.1007/978-3-319-96145-3_18 arxiv:1804.04052.
Google Scholar
Cross Ref
- Aws Albarghouthi and Justin Hsu. 2018. Synthesizing Coupling Proofs of Differential Privacy. Proceedings of the ACM on Programming Languages, 2, POPL (2018), Article 58, Jan., https://doi.org/10.1145/3158146 arxiv:1709.05361.
Google Scholar
Digital Library
- Christel Baier, Edmund M. Clarke, Vasiliki Hartonas-Garmhausen, Marta Z. Kwiatkowska, and Mark Ryan. 1997. Symbolic Model Checking for Probabilistic Processes. In International Colloquium on Automata, Languages and Programming (ICALP), Bologna, Italy (Lecture Notes in Computer Science, Vol. 1256). Springer, 430–440. https://doi.org/10.1007/3-540-63165-8_199
Google Scholar
Cross Ref
- Christel Baier, Luca de Alfaro, Vojtech Forejt, and Marta Kwiatkowska. 2018. Model Checking Probabilistic Systems. In Handbook of Model Checking. Springer-Verlag, 963–999. https://doi.org/10.1007/978-3-319-10575-8_28
Google Scholar
Cross Ref
- Jialu Bao, Nitesh Trivedi, Drashti Pathak, Justin Hsu, and Subhajit Roy. 2022. Data-Driven Invariant Learning for Probabilistic Programs. In International Conference on Computer Aided Verification (CAV), Haifa, Israel. https://doi.org/10.1007/978-3-031-13185-1_3 arxiv:2106.05421.
Google Scholar
Digital Library
- Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2016. The Satisfiability Modulo Theories Library (SMT-LIB). www.SMT-LIB.org.
Google Scholar
- Gilles Barthe, Rohit Chadha, Paul Krogmeier, A. Prasad Sistla, and Mahesh Viswanathan. 2021. Deciding Accuracy of Differential Privacy Schemes. Proceedings of the ACM on Programming Languages, 5, POPL (2021), Article 8, Jan., https://doi.org/10.1145/3434289
Google Scholar
Digital Library
- 2020. Foundations of Probabilistic Programming Languages, Gilles Barthe, Joost-Pieter Katoen, and Alexandra Silva (Eds.). Cambridge University Press. https://doi.org/10.1017/9781108770750.006
Google Scholar
Cross Ref
- Ezio Bartocci, Laura Kovács, and Miroslav Stankovic. 2019. Automatic Generation of Moment-Based Invariants for Prob-Solvable Loops. In International Symposium on Automated Technology for Verification and Analysis (ATVA), Taipei City, Taiwan (Lecture Notes in Computer Science, Vol. 11781). Springer-Verlag, 255–276. https://doi.org/10.1007/978-3-030-31784-3_15
Google Scholar
Digital Library
- Ezio Bartocci, Laura Kovács, and Miroslav Stankovic. 2020. Mora – Automatic Generation of Moment-Based Invariants. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Dublin, Ireland (Lecture Notes in Computer Science, Vol. 12078). Springer-Verlag, 492–498. https://doi.org/10.1007/978-3-030-45190-5_28
Google Scholar
Digital Library
- Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. 2010. A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World. Commun. ACM, 53, 2 (2010), Feb., 66–75. issn:0001-0782 https://doi.org/10.1145/1646353.1646374
Google Scholar
Digital Library
- Burton H. Bloom. 1970. Space/Time Trade-offs in Hash Coding with Allowable Errors. Commun. ACM, 13, 7 (1970), July, 422–426. issn:0001-0782 https://doi.org/10.1145/362686.362692
Google Scholar
Digital Library
- Mateus Borges, Antonio Filieri, Marcelo d’Amorim, Corina S. Păsăreanu, and Willem Visser. 2014. Compositional Solution Space Quantification for Probabilistic Software Analysis. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Edinburgh, Scotland. 123–132. https://doi.org/10.1145/2666356.2594329
Google Scholar
Digital Library
- Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), San Diego, California. 209–224. http://www.usenix.org/events/osdi08/tech/full_papers/cadar/cadar.pdf
Google Scholar
- Michael Carbin, Deokhwan Kim, Sasa Misailovic, and Martin C Rinard. 2012. Proving Acceptability Properties of Relaxed Nondeterministic Approximate Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Beijing, China. 169–180. https://doi.org/10.1145/2254064.2254086
Google Scholar
Digital Library
- Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic Program Analysis with Martingales. In International Conference on Computer Aided Verification (CAV), Saint Petersburg, Russia (Lecture Notes in Computer Science, Vol. 8044). 511–526. https://doi.org/10.1007/978-3-642-39799-8_34
Google Scholar
Cross Ref
- Krishnendu Chatterjee, Hongfei Fu, Petr Novotný, and Rouzbeh Hasheminezhad. 2016. Algorithmic Analysis of Qualitative and Quantitative Termination Problems for Affine Probabilistic Programs. In ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), Saint Petersburg, Florida. 327–342. isbn:978-1-4503-3549-2 https://doi.org/10.1145/2837614.2837639
Google Scholar
Digital Library
- Bihuan Chen, Yang Liu, and Wei Le. 2016. Generating Performance Distributions via Probabilistic Symbolic Execution. In International Conference on Software Engineering (ICSE), Austin, Texas. 49–60. isbn:9781450339001 https://doi.org/10.1145/2884781.2884794
Google Scholar
Digital Library
- Guillaume Claret, Sriram K. Rajamani, Aditya V. Nori, Andrew D. Gordon, and Johannes Borgström. 2013. Bayesian Inference Using Data Flow Analysis. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), Saint Petersburg, Russia. 92–102. https://doi.org/10.1145/2491411.2491423
Google Scholar
Digital Library
- Graham Cormode and S. Muthukrishnan. 2004. An Improved Data Stream Summary: The Count-Min Sketch and Its Applications. In Latin American Symposium on Theoretical Informatics (LATIN), Buenos Aires, Argentina (Lecture Notes in Computer Science, Vol. 2976). Springer-Verlag, 29–38. https://doi.org/10.1007/978-3-540-24698-5_7
Google Scholar
Cross Ref
- Patrick Cousot and Michael Monerau. 2012. Probabilistic Abstract Interpretation. In European Symposium on Programming (ESOP), Tallinn, Estonia (Lecture Notes in Computer Science, Vol. 7211). Springer-Verlag, 169–193. https://doi.org/10.1007/978-3-642-28869-2_9
Google Scholar
Digital Library
- Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Budapest, Hungary (Lecture Notes in Computer Science, Vol. 4963). Springer-Verlag, 337–340. isbn:978-3-540-78800-3 https://doi.org/10.1007/978-3-540-78800-3_24
Google Scholar
Cross Ref
- Gian Pietro Farina, Stephen Chong, and Marco Gaboardi. 2019. Relational Symbolic Execution. In ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP), Porto, Portugal. Article 10, https://doi.org/10.1145/3354166.3354175
Google Scholar
Digital Library
- Gian Pietro Farina, Stephen Chong, and Marco Gaboardi. 2021. Coupled Relational Symbolic Execution for Differential Privacy. In European Symposium on Programming (ESOP), Luxembourg City, Luxembourg (Lecture Notes in Computer Science, Vol. 12648). Springer-Verlag, 207–233. https://doi.org/10.1007/978-3-030-72019-3_8
Google Scholar
Digital Library
- Antonio Filieri, Corina S. Păsăreanu, and Willem Visser. 2013. Reliability Analysis in Symbolic Pathfinder. In International Conference on Software Engineering (ICSE), San Francisco, California. 622–631. https://doi.org/10.1109/ICSE.2013.6606608
Google Scholar
Cross Ref
- Antonio Filieri, Corina S. Păsăreanu, and Guowei Yang. 2015. Quantification of Software Changes through Probabilistic Symbolic Execution. In IEEE/ACM International Conference on Automated Software Engineering (ASE), Lincoln, Nebraska. 703–708. https://doi.org/10.1109/ASE.2015.78
Google Scholar
Digital Library
- Rūsiņš Freivalds. 1977. Probabilistic Machines Can Use Less Running Time. In IFIP World Congress, Toronto, Canada. North-Holland, 839–842.
Google Scholar
- Timon Gehr, Sasa Misailovic, and Martin Vechev. 2016. PSI: Exact Symbolic Inference for Probabilistic Programs. In International Conference on Computer Aided Verification (CAV), Toronto, Ontario (Lecture Notes in Computer Science, Vol. 9779). Springer-Verlag, 62–83. https://doi.org/10.1007/978-3-319-41528-4_4
Google Scholar
Cross Ref
- Timon Gehr, Samuel Steffen, and Martin Vechev. 2020. λ PSI: Exact Inference for Higher-Order Probabilistic Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), London, England. 883–897. https://doi.org/10.1145/3385412.3386006
Google Scholar
Digital Library
- Jaco Geldenhuys, Matthew B. Dwyer, and Willem Visser. 2012. Probabilistic Symbolic Execution. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), Minneapolis, Minnesota. 166–176. https://doi.org/10.1145/2338965.2336773
Google Scholar
Digital Library
- Oded Goldreich. 2017. Introduction to Property Testing. Cambridge University Press. isbn:978-1-107-19405-2 https://doi.org/10.1017/9781108135252
Google Scholar
Cross Ref
- Friedrich Gretz, Joost-Pieter Katoen, and Annabelle McIver. 2013. Prinsys—On a Quest for Probabilistic Loop Invariants. In International Conference on Quantitative Evaluation of Systems (QEST), Buenos Aires, Argentina (Lecture Notes in Computer Science, Vol. 8054). Springer-Verlag, 193–208. https://doi.org/10.1007/978-3-642-40196-1_17
Google Scholar
Digital Library
- Friedrich Gretz, Joost-Pieter Katoen, and Annabelle McIver. 2014. Operational versus weakest pre-expectation semantics for the probabilistic guarded command language. Performance Evaluation, 73 (2014), 110–132. https://doi.org/10.1016/j.peva.2013.11.004
Google Scholar
Digital Library
- Christian Hensel, Sebastian Junges, Joost-Pieter Katoen, Tim Quatmann, and Matthias Volk. 2022. The probabilistic model checker Storm. International Journal on Software Tools for Technology Transfer, 24, 4 (2022), 589–610. https://doi.org/10.1007/s10009-021-00633-z arxiv:2002.07080.
Google Scholar
Digital Library
- C. A. R. Hoare. 1961. Algorithms 63-64: partition and quicksort. Commun. ACM, 4, 7 (1961), July, 321. https://doi.org/10.1145/366622.366642
Google Scholar
Digital Library
- Steven Holtzen, Guy Van den Broeck, and Todd Millstein. 2020. Scaling Exact Inference for Discrete Probabilistic Programs. Proceedings of the ACM on Programming Languages, 4, OOPSLA (2020), Article 140, Nov., https://doi.org/10.1145/3428208
Google Scholar
Digital Library
- Keyur Joshi, Vimuth Fernando, and Sasa Misailovic. 2019. Statistical Algorithmic Profiling for Randomized Approximate Programs. In International Conference on Software Engineering (ICSE), Montréal, Québec. 608–618. https://doi.org/10.1109/ICSE.2019.00071
Google Scholar
Digital Library
- Qiao Kang, Jiarong Xing, Yiming Qiu, and Ang Chen. 2021. Probabilistic Profiling of Stateful Data Planes for Adversarial Testing. In International Conference on Architectural Support for Programming Langauages and Operating Systems (ASPLOS). 286–301. https://doi.org/10.1145/3445814.3446764
Google Scholar
Digital Library
- Timotej Kapus, Martin Nowack, and Cristian Cadar. 2019. Constraints in Dynamic Symbolic Execution: Bitvectors or Integers? In International Conference on Tests and Proofs (TAP), Porto, Portugal (Lecture Notes in Computer Science, Vol. 11823). Springer-Verlag, 41–54. https://doi.org/10.1007/978-3-030-31157-5_3
Google Scholar
Digital Library
- Adam Kiezun, Vijay Ganesh, Shay Artzi, Philip J. Guo, Pieter Hooimeijer, and Michael D. Ernst. 2012. HAMPI: A Solver for Word Equations over Strings, Regular Expressions, and Context-Free Grammars. ACM Transactions on Software Engineering and Methodology, 21, 4 (2012), Article 25, Nov., https://doi.org/10.1145/2377656.2377662
Google Scholar
Digital Library
- James C. King. 1976. Symbolic Execution and Program Testing. Commun. ACM, 19, 7 (1976), July, 385–394. https://doi.org/10.1145/360248.360252
Google Scholar
Digital Library
- Dexter Kozen. 1985. A Probabilistic PDL. J. Comput. System Sci., 30, 2 (1985), 162–178. https://doi.org/10.1016/0022-0000(85)90012-1
Google Scholar
Cross Ref
- Marta Kwiatkowska, Gethin Norman, and David Parker. 2011. PRISM 4.0: Verification of Probabilistic Real-Time Systems. In International Conference on Computer Aided Verification (CAV), Snowbird, Utah (Lecture Notes in Computer Science, Vol. 6806). Springer-Verlag, 585–591. https://doi.org/10.1007/978-3-642-22110-1_47
Google Scholar
Cross Ref
- Marcel Moosbrugger, Ezio Bartocci, Joost-Pieter Katoen, and Laura Kovács. 2021. Automated Termination Analysis of Polynomial Probabilistic Programs. In European Symposium on Programming (ESOP), Luxembourg City, Luxembourg (Lecture Notes in Computer Science, Vol. 12648). Springer-Verlag, 491–518. https://doi.org/10.1007/978-3-030-72019-3_18
Google Scholar
Digital Library
- Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. ACM Transactions on Programming Languages and Systems, 18, 3 (1996), 325–353. https://doi.org/10.1145/229542.229547
Google Scholar
Digital Library
- Adrian Sampson, Pavel Panchekha, Todd Mytkowicz, Kathryn S. McKinley, Dan Grossman, and Luis Ceze. 2014. Expressing and Verifying Probabilistic Assertions. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Edinburgh, Scotland. 112–122. https://doi.org/10.1145/2594291.2594294
Google Scholar
Digital Library
- Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. 2013. Static Analysis for Probabilistic Programs: Inferring Whole Program Properties from Finitely Many Paths. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Seattle, Washington. 447–458. https://doi.org/10.1145/2499370.2462179
Google Scholar
Digital Library
- Raimondas Sasnauskas, Oscar Soria Dustmann, Benjamin Lucien Kaminski, Klaus Wehrle, Carsten Weise, and Stefan Kowalewski. 2011. Scalable Symbolic Execution of Distributed Systems. In International Conference on Distributed Computing Systems (ICDCS), Minneapolis, Minnesota. 333–342. https://doi.org/10.1109/ICDCS.2011.28
Google Scholar
Digital Library
- Raimondas Sasnauskas, Olaf Landsiedel, Muhammad Hamad Alizai, Carsten Weise, Stefan Kowalewski, and Klaus Wehrle. 2010. KleeNet: Discovering Insidious Interaction Bugs in Wireless Sensor Networks before Deployment. In ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), Stockholm, Sweden. 186–196. https://doi.org/10.1145/1791212.1791235
Google Scholar
Digital Library
- Steve Selvin. 1975. Letters to the Editor. The American Statistician, 29, 1 (1975), 67–71. https://doi.org/10.1080/00031305.1975.10479121 arxiv:https://doi.org/10.1080/00031305.1975.10479121.
Google Scholar
Cross Ref
- Calvin Smith, Justin Hsu, and Aws Albarghouthi. 2019. Trace Abstraction modulo Probability. Proceedings of the ACM on Programming Languages, 3, POPL (2019), Article 39, Jan., https://doi.org/10.1145/3290352 arxiv:1810.12396.
Google Scholar
Digital Library
- Zachary Susag, Sumit Lahiri, Justin Hsu, and Subhajit Roy. 2022. Artifact for Symbolic Execution for Randomized Programs. https://doi.org/10.5281/zenodo.7061819
Google Scholar
Digital Library
- Zachary Susag, Sumit Lahiri, Justin Hsu, and Subhajit Roy. 2022. Symbolic Execution for Randomized Programs. https://doi.org/10.48550/arXiv.2209.08046 arxiv:2209.08046.
Google Scholar
- Jeffrey Scott Vitter. 1985. Random Sampling with a Reservoir. ACM Trans. Math. Software, 11, 1 (1985), March, 37–57. https://doi.org/10.1145/3147.3165
Google Scholar
Digital Library
- Di Wang, Jan Hoffmann, and Thomas W. Reps. 2018. PMAF: An Algebraic Framework for Static Analysis of Probabilistic Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Philadelphia, Pennsylvania. 513–528. https://doi.org/10.1145/3192366.3192408
Google Scholar
Digital Library
- Di Wang, Jan Hoffmann, and Thomas W. Reps. 2021. Central Moment Analysis for Cost Accumulators in Probabilistic Programs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). 559–573. https://doi.org/10.1145/3453483.3454062 arxiv:2001.10150.
Google Scholar
Digital Library
Index Terms
Symbolic execution for randomized programs
Recommendations
Verifying systems rules using rule-directed symbolic execution
ASPLOS '13Systems code must obey many rules, such as "opened files must be closed." One approach to verifying rules is static analysis, but this technique cannot infer precise runtime effects of code, often emitting many false positives. An alternative is ...
Symbolic execution of multithreaded programs from arbitrary program contexts
OOPSLA '14We describe an algorithm to perform symbolic execution of a multithreaded program starting from an arbitrary program context. We argue that this can enable more efficient symbolic exploration of deep code paths in multithreaded programs by allowing the ...
Symbolic execution and program testing
This paper describes the symbolic execution of programs. Instead of supplying the normal inputs to a program (e.g. numbers) one supplies symbols representing arbitrary values. The execution proceeds as in a normal execution except that values may be ...






Comments