Abstract
Relational verification aims to prove properties that relate a pair of programs or two different runs of the same program. While relational properties (e.g., equivalence, non-interference) can be verified by reducing them to standard safety, there are typically many possible reduction strategies, only some of which result in successful automated verification. Motivated by this problem, we propose a novel relational verification algorithm that learns useful reduction strategies using reinforcement learning. Specifically, we show how to formulate relational verification as a Markov Decision Process (MDP) and use reinforcement learning to synthesize an optimal policy for the underlying MDP. The learned policy is then used to guide the search for a successful verification strategy. We have implemented this approach in a tool called Coeus and evaluate it on two benchmark suites. Our evaluation shows that Coeus solves significantly more problems within a given time limit compared to multiple baselines, including two state-of-the-art relational verification tools.
Supplemental Material
- Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. Deepcoder: Learning to write programs. In ICLR.Google Scholar
- Gilles Barthe, Juan Manuel Crespo, and César Kunz. 2011. Relational verification using product programs. In International Symposium on Formal Methods. Springer, 200–214.Google Scholar
Cross Ref
- Gilles Barthe, Juan Manuel Crespo, and César Kunz. 2016. Product programs and relational program logics. Journal of Logical and Algebraic Methods in Programming 85, 5 (2016), 847–859.Google Scholar
Cross Ref
- Gilles Barthe, Pedro R D’Argenio, and Tamara Rezk. 2004. Secure information flow by self-composition. In Computer Security Foundations Workshop, 2004. Proceedings. 17th IEEE. IEEE, 100–114.Google Scholar
Cross Ref
- Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella Béguelin. 2012. Probabilistic relational reasoning for differential privacy. In ACM SIGPLAN Notices, Vol. 47. ACM, 97–110.Google Scholar
Digital Library
- Osbert Bastani, Yewen Pu, and Armando Solar-Lezama. 2018a. Verifiable reinforcement learning via policy extraction. In NIPS.Google Scholar
- Osbert Bastani, Rahul Sharma, Alex Aiken, and Percy Liang. 2017. Synthesizing program input grammars. In PLDI, Vol. 52. ACM, 95–110.Google Scholar
Digital Library
- Osbert Bastani, Rahul Sharma, Alex Aiken, and Percy Liang. 2018b. Active learning of points-to specifications. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 678–692.Google Scholar
Digital Library
- Nels E Beckman and Aditya V Nori. 2011. Probabilistic, modular and scalable inference of typestate specifications. In PLDI, Vol. 46. ACM, 211–221.Google Scholar
Digital Library
- Nick Benton. 2004. Simple relational correctness proofs for static analyses and program transformations. In ACM SIGPLAN Notices, Vol. 39. ACM, 14–25.Google Scholar
Digital Library
- Pavol Bielik, Veselin Raychev, and Martin Vechev. 2016. PHOG: probabilistic model for code. In International Conference on Machine Learning. 2933–2942.Google Scholar
- Pavol Bielik, Veselin Raychev, and Martin Vechev. 2017. Learning a static analyzer from data. In International Conference on Computer Aided Verification. Springer, 233–253.Google Scholar
Cross Ref
- Nikolaj Bjørner, Arie Gurfinkel, Ken McMillan, and Andrey Rybalchenko. 2015. Horn clause solvers for program verification. In Fields of Logic and Computation II. Springer, 24–51.Google Scholar
- Jia Chen, Yu Feng, and Isil Dillig. 2017. Precise Detection of Side-Channel Vulnerabilities using Quantitative Cartesian Hoare Logic. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 875–890.Google Scholar
Digital Library
- Lonnie Chrisman. 1992. Reinforcement learning with perceptual aliasing: The perceptual distinctions approach. In AAAI, Vol. 1992. Citeseer, 183–188.Google Scholar
- Lazaro Clapp, Osbert Bastani, Saswat Anand, and Alex Aiken. 2016. Minimizing GUI event traces. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 422–434.Google Scholar
Digital Library
- Michael R. Clarkson and Fred B. Schneider. 2010. Hyperproperties. Journal of Computer Security 18, 6 (Sept. 2010), 1157–1210.Google Scholar
Digital Library
- Emanuele De Angelis, Fabio Fioravanti, Alberto Pettorossi, and Maurizio Proietti. 2016a. Horn Clause Transformation for Program Verification. Technical Report.Google Scholar
- Emanuele De Angelis, Fabio Fioravanti, Alberto Pettorossi, and Maurizio Proietti. 2016b. Relational verification through Horn clause transformation. In International Static Analysis Symposium. Springer, 147–169.Google Scholar
Cross Ref
- Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 337–340.Google Scholar
Digital Library
- Marco Eilers, Peter Müller, and Samuel Hitz. 2018. Modular Product Programs. In European Symposium on Programming. Springer, 502–529.Google Scholar
- Dennis Felsing, Sarah Grebing, Vladimir Klebanov, Philipp Rümmer, and Mattias Ulbrich. 2014. Automating Regression Verification. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering. 349–360.Google Scholar
Digital Library
- Yu Feng, Ruben Martins, Osbert Bastani, and Isil Dillig. 2018. Program synthesis using conflict-driven learning. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 420–435.Google Scholar
Digital Library
- Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based synthesis of table consolidation and transformation tasks from examples. In PLDI, Vol. 52. ACM, 422–436.Google Scholar
- Cormac Flanagan and K. Rustan M. Leino. 2001. Houdini, an Annotation Assistant for ESC/Java. In Proceedings of the International Symposium of Formal Methods Europe on Formal Methods for Increasing Software Productivity (FME ’01). Springer-Verlag, Berlin, Heidelberg, 500–517.Google Scholar
Digital Library
- Patrice Godefroid, Hila Peleg, and Rishabh Singh. 2017. Learn&fuzz: Machine learning for input fuzzing. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 50–59.Google Scholar
Digital Library
- Joseph A Goguen and José Meseguer. 1982. Security policies and security models. In Security and Privacy, 1982 IEEE Symposium on. IEEE, 11–11.Google Scholar
Cross Ref
- Xiaoxiao Guo, Satinder Singh, Honglak Lee, Richard L Lewis, and Xiaoshi Wang. 2014. Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning. In Advances in neural information processing systems. 3338–3346.Google Scholar
- Arie Gurfinkel, Temesghen Kahsai, and Jorge A. Navas. 2015. SeaHorn: A Framework for Verifying C Programs (Competition Contribution). In Tools and Algorithms for the Construction and Analysis of Systems, Christel Baier and Cesare Tinelli (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 447–450.Google Scholar
- Stefan Heule, Eric Schkufza, Rahul Sharma, and Alex Aiken. 2016. Stratified synthesis: automatically learning the x86-64 instruction set. In PLDI, Vol. 51. ACM, 237–250.Google Scholar
Digital Library
- Geoffrey Irving, Christian Szegedy, Alexander A Alemi, Niklas Eén, François Chollet, and Josef Urban. 2016. Deepmath-deep sequence models for premise selection. In Advances in Neural Information Processing Systems. 2235–2243.Google Scholar
- Ashwin Kalyan, Abhishek Mohta, Oleksandr Polozov, Dhruv Batra, Prateek Jain, and Sumit Gulwani. 2018. Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples. In ICLR.Google Scholar
- Anvesh Komuravelli, Arie Gurfinkel, and Sagar Chaki. 2016. SMT-based Model Checking for Recursive Programs. Formal Methods in System Design 48, 3 (June 2016), 175–205.Google Scholar
Digital Library
- Ted Kremenek, Paul Twohey, Godmar Back, Andrew Ng, and Dawson Engler. 2006. From uncertainty to belief: Inferring the specification within. In Proceedings of the 7th symposium on Operating systems design and implementation. 161–176.Google Scholar
- Shuvendu K Lahiri, Chris Hawblitzel, Ming Kawaguchi, and Henrique Rebêlo. 2012. Symdiff: A language-agnostic semantic diff tool for imperative programs. In International Conference on Computer Aided Verification. Springer, 712–717.Google Scholar
Digital Library
- Shuvendu K Lahiri, Kenneth L McMillan, Rahul Sharma, and Chris Hawblitzel. 2013. Differential assertion checking. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 345–355.Google Scholar
Digital Library
- Woosuk Lee, Kihong Heo, Rajeev Alur, and Mayur Naik. 2018. Accelerating search-based program synthesis using learned probabilistic models. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 436–449.Google Scholar
Digital Library
- Xavier Leroy. 2009. Formal Verification of a Realistic Compiler. Commun. ACM 52, 7 (July 2009), 107–115.Google Scholar
Digital Library
- Percy Liang, Omer Tripp, and Mayur Naik. 2011. Learning minimal abstractions. In POPL, Vol. 46. ACM, 31–42.Google Scholar
Digital Library
- Ben Liblit, Mayur Naik, Alice X Zheng, Alex Aiken, and Michael I Jordan. 2005. Scalable statistical bug isolation. 40, 6 (2005), 15–26.Google Scholar
- Benjamin Livshits, Aditya V Nori, Sriram K Rajamani, and Anindya Banerjee. 2009. Merlin: specification inference for explicit information flow problems, Vol. 44. ACM, 75–86.Google Scholar
- Ravi Mangal, Xin Zhang, Aditya V Nori, and Mayur Naik. 2015. A user-guided approach to program analysis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 462–473.Google Scholar
Digital Library
- R Andrew McCallum. 1993. Overcoming incomplete perception with utile distinction memory. In Proceedings of the Tenth International Conference on Machine Learning. 190–196.Google Scholar
Cross Ref
- William H Montgomery and Sergey Levine. 2016. Guided policy search via approximate mirror descent. In Advances in Neural Information Processing Systems. 4008–4016.Google Scholar
- Dmitry Mordvinov and Grigory Fedyukovich. 2017. Synchronizing constrained Horn clauses. LPAR, EPiC Series in Computing. EasyChair (2017).Google Scholar
- Saswat Padhi, Rahul Sharma, and Todd Millstein. 2016. Data-driven precondition inference with learned features. In PLDI, Vol. 51. ACM, 42–56.Google Scholar
Digital Library
- Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. In NIPS-W.Google Scholar
- Amir Pnueli, Michael Siegel, and Eli Singerman. 1998. Translation Validation. In Proceedings of the 4th International Conference on Tools and Algorithms for Construction and Analysis of Systems (TACAS ’98). Springer-Verlag, Berlin, Heidelberg, 151–166.Google Scholar
Digital Library
- Dan Quinlan and Chunhua Liao. 2011. The ROSE Source-to-Source Compiler Infrastructure. In Cetus Users and Compiler Infrastructure Workshop, in conjunction with PACT 2011.Google Scholar
- Mukund Raghothaman, Sulekha Kulkarni, Kihong Heo, and Mayur Naik. 2018. User-guided program reasoning using Bayesian inference. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 722–735.Google Scholar
Digital Library
- Veselin Raychev, Pavol Bielik, and Martin Vechev. 2016a. Probabilistic model for code with decision trees. In OOPSLA, Vol. 51. ACM, 731–747.Google Scholar
Digital Library
- Veselin Raychev, Pavol Bielik, Martin Vechev, and Andreas Krause. 2016b. Learning programs from noisy data. In POPL, Vol. 51. ACM, 761–774.Google Scholar
Digital Library
- Veselin Raychev, Martin Vechev, and Andreas Krause. 2015. Predicting program properties from big code. In POPL, Vol. 50. ACM, 111–124.Google Scholar
Digital Library
- Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In PLDI, Vol. 49. ACM, 419–428.Google Scholar
Digital Library
- Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic superoptimization. In ASPLOS, Vol. 41. ACM, 305–316.Google Scholar
Digital Library
- Eric Schkufza, Rahul Sharma, and Alex Aiken. 2014. Stochastic optimization of floating-point programs with tunable precision. In PLDI, Vol. 49. ACM, 53–64.Google Scholar
Digital Library
- John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy optimization. In International Conference on Machine Learning. 1889–1897.Google Scholar
- Rahul Sharma and Alex Aiken. 2014. From invariant checking to invariant inference using randomized search. In CAV.Google Scholar
- Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, and Aditya V Nori. 2013. Verification as learning geometric concepts. In International Static Analysis Symposium. Springer, 388–411.Google Scholar
Cross Ref
- Xujie Si, Hanjun Dai, Mukund Raghothaman, Mayur Naik, and Le Song. 2018a. Learning loop invariants for program verification. In Advances in Neural Information Processing Systems. 7762–7773.Google Scholar
- Xujie Si, Yuan Yang, Hanjun Dai, Mayur Naik, and Le Song. 2018b. Learning a Meta-Solver for Syntax-Guided Program Synthesis. In ICLR.Google Scholar
- David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the Game of Go with Deep Neural Networks and Tree Search. Nature 529, 7587 (Jan. 2016), 484–489.Google Scholar
- David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017. Mastering the game of go without human knowledge. Nature 550, 7676 (2017), 354.Google Scholar
- Gagandeep Singh, Markus Püschel, and Martin Vechev. 2018. Fast Numerical Program Analysis with Reinforcement Learning. In International Conference on Computer Aided Verification. Springer, 211–229.Google Scholar
- Marcelo Sousa and Isil Dillig. 2016. Cartesian hoare logic for verifying k-safety properties. In Proc. Conference on Programming Language Design and Implementation. 57–69.Google Scholar
Digital Library
- Marcelo Sousa, Isil Dillig, and Shuvendu Lahiri. 2018. Verifying Semantic Conflict-Freedom in Three-Way Program Merges. arXiv preprint arXiv:1802.06551 (2018).Google Scholar
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.Google Scholar
Digital Library
- Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems. 1057–1063.Google Scholar
- Tachio Terauchi and Alex Aiken. 2005. Secure Information Flow As a Safety Problem. In Proceedings of the 12th International Conference on Static Analysis (SAS’05). 352–367.Google Scholar
Digital Library
- Geoffrey Towell and Jude W. Shavlik. 1992. Interpretation of Artificial Neural Networks: Mapping Knowledge-Based Neural Networks into Rules. In Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann (Eds.). Morgan-Kaufmann, 977–984.Google Scholar
- Mingzhe Wang, Yihe Tang, Jian Wang, and Jia Deng. 2017. Premise selection for theorem proving by deep graph embedding. In Advances in Neural Information Processing Systems. 2786–2796.Google Scholar
- Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. Deeppath: A reinforcement learning method for knowledge graph reasoning. In EMNLP.Google Scholar
- Hongseok Yang. 2007. Relational separation logic. Theoretical Computer Science 375, 1-3 (2007), 308–334.Google Scholar
Digital Library
- Anna Zaks and Amir Pnueli. 2008. Covac: Compiler validation by program analysis of the cross-product. In FM 2008: Formal Methods. Springer, 35–51.Google Scholar
Digital Library
Index Terms
Relational verification using reinforcement learning
Recommendations
Relational Reinforcement Learning
Relational reinforcement learning is presented, a learning technique that combines reinforcement learning with relational learning or inductive logic programming. Due to the use of a more expressive representation language to represent states, actions ...
Learning with whom to communicate using relational reinforcement learning
AAMAS '09: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2Relational reinforcement learning (RRL) has emerged in the machine learning community as a new promising subfield of reinforcement learning (RL) (e.g. [1]). It upgrades RL techniques by using relational representations for states, actions and learned ...
Learning to verify branching time properties
We present a new model checking algorithm for verifying computation tree logic (CTL) properties. Our technique is based on using language inference to learn the fixpoints necessary for checking a CTL formula instead of computing them iteratively as is ...






Comments