skip to main content
research-article
Open Access

Relational verification using reinforcement learning

Published:10 October 2019Publication History
Skip Abstract Section

Abstract

Relational verification aims to prove properties that relate a pair of programs or two different runs of the same program. While relational properties (e.g., equivalence, non-interference) can be verified by reducing them to standard safety, there are typically many possible reduction strategies, only some of which result in successful automated verification. Motivated by this problem, we propose a novel relational verification algorithm that learns useful reduction strategies using reinforcement learning. Specifically, we show how to formulate relational verification as a Markov Decision Process (MDP) and use reinforcement learning to synthesize an optimal policy for the underlying MDP. The learned policy is then used to guide the search for a successful verification strategy. We have implemented this approach in a tool called Coeus and evaluate it on two benchmark suites. Our evaluation shows that Coeus solves significantly more problems within a given time limit compared to multiple baselines, including two state-of-the-art relational verification tools.

Skip Supplemental Material Section

Supplemental Material

a141-chen

Presentation at OOPSLA '19

References

  1. Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. Deepcoder: Learning to write programs. In ICLR.Google ScholarGoogle Scholar
  2. Gilles Barthe, Juan Manuel Crespo, and César Kunz. 2011. Relational verification using product programs. In International Symposium on Formal Methods. Springer, 200–214.Google ScholarGoogle ScholarCross RefCross Ref
  3. Gilles Barthe, Juan Manuel Crespo, and César Kunz. 2016. Product programs and relational program logics. Journal of Logical and Algebraic Methods in Programming 85, 5 (2016), 847–859.Google ScholarGoogle ScholarCross RefCross Ref
  4. Gilles Barthe, Pedro R D’Argenio, and Tamara Rezk. 2004. Secure information flow by self-composition. In Computer Security Foundations Workshop, 2004. Proceedings. 17th IEEE. IEEE, 100–114.Google ScholarGoogle ScholarCross RefCross Ref
  5. Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella Béguelin. 2012. Probabilistic relational reasoning for differential privacy. In ACM SIGPLAN Notices, Vol. 47. ACM, 97–110.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Osbert Bastani, Yewen Pu, and Armando Solar-Lezama. 2018a. Verifiable reinforcement learning via policy extraction. In NIPS.Google ScholarGoogle Scholar
  7. Osbert Bastani, Rahul Sharma, Alex Aiken, and Percy Liang. 2017. Synthesizing program input grammars. In PLDI, Vol. 52. ACM, 95–110.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Osbert Bastani, Rahul Sharma, Alex Aiken, and Percy Liang. 2018b. Active learning of points-to specifications. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 678–692.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Nels E Beckman and Aditya V Nori. 2011. Probabilistic, modular and scalable inference of typestate specifications. In PLDI, Vol. 46. ACM, 211–221.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Nick Benton. 2004. Simple relational correctness proofs for static analyses and program transformations. In ACM SIGPLAN Notices, Vol. 39. ACM, 14–25.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Pavol Bielik, Veselin Raychev, and Martin Vechev. 2016. PHOG: probabilistic model for code. In International Conference on Machine Learning. 2933–2942.Google ScholarGoogle Scholar
  12. Pavol Bielik, Veselin Raychev, and Martin Vechev. 2017. Learning a static analyzer from data. In International Conference on Computer Aided Verification. Springer, 233–253.Google ScholarGoogle ScholarCross RefCross Ref
  13. Nikolaj Bjørner, Arie Gurfinkel, Ken McMillan, and Andrey Rybalchenko. 2015. Horn clause solvers for program verification. In Fields of Logic and Computation II. Springer, 24–51.Google ScholarGoogle Scholar
  14. Jia Chen, Yu Feng, and Isil Dillig. 2017. Precise Detection of Side-Channel Vulnerabilities using Quantitative Cartesian Hoare Logic. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 875–890.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Lonnie Chrisman. 1992. Reinforcement learning with perceptual aliasing: The perceptual distinctions approach. In AAAI, Vol. 1992. Citeseer, 183–188.Google ScholarGoogle Scholar
  16. Lazaro Clapp, Osbert Bastani, Saswat Anand, and Alex Aiken. 2016. Minimizing GUI event traces. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 422–434.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Michael R. Clarkson and Fred B. Schneider. 2010. Hyperproperties. Journal of Computer Security 18, 6 (Sept. 2010), 1157–1210.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Emanuele De Angelis, Fabio Fioravanti, Alberto Pettorossi, and Maurizio Proietti. 2016a. Horn Clause Transformation for Program Verification. Technical Report.Google ScholarGoogle Scholar
  19. Emanuele De Angelis, Fabio Fioravanti, Alberto Pettorossi, and Maurizio Proietti. 2016b. Relational verification through Horn clause transformation. In International Static Analysis Symposium. Springer, 147–169.Google ScholarGoogle ScholarCross RefCross Ref
  20. Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 337–340.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Marco Eilers, Peter Müller, and Samuel Hitz. 2018. Modular Product Programs. In European Symposium on Programming. Springer, 502–529.Google ScholarGoogle Scholar
  22. Dennis Felsing, Sarah Grebing, Vladimir Klebanov, Philipp Rümmer, and Mattias Ulbrich. 2014. Automating Regression Verification. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering. 349–360.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yu Feng, Ruben Martins, Osbert Bastani, and Isil Dillig. 2018. Program synthesis using conflict-driven learning. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 420–435.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based synthesis of table consolidation and transformation tasks from examples. In PLDI, Vol. 52. ACM, 422–436.Google ScholarGoogle Scholar
  25. Cormac Flanagan and K. Rustan M. Leino. 2001. Houdini, an Annotation Assistant for ESC/Java. In Proceedings of the International Symposium of Formal Methods Europe on Formal Methods for Increasing Software Productivity (FME ’01). Springer-Verlag, Berlin, Heidelberg, 500–517.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Patrice Godefroid, Hila Peleg, and Rishabh Singh. 2017. Learn&fuzz: Machine learning for input fuzzing. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 50–59.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Joseph A Goguen and José Meseguer. 1982. Security policies and security models. In Security and Privacy, 1982 IEEE Symposium on. IEEE, 11–11.Google ScholarGoogle ScholarCross RefCross Ref
  28. Xiaoxiao Guo, Satinder Singh, Honglak Lee, Richard L Lewis, and Xiaoshi Wang. 2014. Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning. In Advances in neural information processing systems. 3338–3346.Google ScholarGoogle Scholar
  29. Arie Gurfinkel, Temesghen Kahsai, and Jorge A. Navas. 2015. SeaHorn: A Framework for Verifying C Programs (Competition Contribution). In Tools and Algorithms for the Construction and Analysis of Systems, Christel Baier and Cesare Tinelli (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 447–450.Google ScholarGoogle Scholar
  30. Stefan Heule, Eric Schkufza, Rahul Sharma, and Alex Aiken. 2016. Stratified synthesis: automatically learning the x86-64 instruction set. In PLDI, Vol. 51. ACM, 237–250.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Geoffrey Irving, Christian Szegedy, Alexander A Alemi, Niklas Eén, François Chollet, and Josef Urban. 2016. Deepmath-deep sequence models for premise selection. In Advances in Neural Information Processing Systems. 2235–2243.Google ScholarGoogle Scholar
  32. Ashwin Kalyan, Abhishek Mohta, Oleksandr Polozov, Dhruv Batra, Prateek Jain, and Sumit Gulwani. 2018. Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples. In ICLR.Google ScholarGoogle Scholar
  33. Anvesh Komuravelli, Arie Gurfinkel, and Sagar Chaki. 2016. SMT-based Model Checking for Recursive Programs. Formal Methods in System Design 48, 3 (June 2016), 175–205.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Ted Kremenek, Paul Twohey, Godmar Back, Andrew Ng, and Dawson Engler. 2006. From uncertainty to belief: Inferring the specification within. In Proceedings of the 7th symposium on Operating systems design and implementation. 161–176.Google ScholarGoogle Scholar
  35. Shuvendu K Lahiri, Chris Hawblitzel, Ming Kawaguchi, and Henrique Rebêlo. 2012. Symdiff: A language-agnostic semantic diff tool for imperative programs. In International Conference on Computer Aided Verification. Springer, 712–717.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Shuvendu K Lahiri, Kenneth L McMillan, Rahul Sharma, and Chris Hawblitzel. 2013. Differential assertion checking. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 345–355.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Woosuk Lee, Kihong Heo, Rajeev Alur, and Mayur Naik. 2018. Accelerating search-based program synthesis using learned probabilistic models. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 436–449.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Xavier Leroy. 2009. Formal Verification of a Realistic Compiler. Commun. ACM 52, 7 (July 2009), 107–115.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Percy Liang, Omer Tripp, and Mayur Naik. 2011. Learning minimal abstractions. In POPL, Vol. 46. ACM, 31–42.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Ben Liblit, Mayur Naik, Alice X Zheng, Alex Aiken, and Michael I Jordan. 2005. Scalable statistical bug isolation. 40, 6 (2005), 15–26.Google ScholarGoogle Scholar
  41. Benjamin Livshits, Aditya V Nori, Sriram K Rajamani, and Anindya Banerjee. 2009. Merlin: specification inference for explicit information flow problems, Vol. 44. ACM, 75–86.Google ScholarGoogle Scholar
  42. Ravi Mangal, Xin Zhang, Aditya V Nori, and Mayur Naik. 2015. A user-guided approach to program analysis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 462–473.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. R Andrew McCallum. 1993. Overcoming incomplete perception with utile distinction memory. In Proceedings of the Tenth International Conference on Machine Learning. 190–196.Google ScholarGoogle ScholarCross RefCross Ref
  44. William H Montgomery and Sergey Levine. 2016. Guided policy search via approximate mirror descent. In Advances in Neural Information Processing Systems. 4008–4016.Google ScholarGoogle Scholar
  45. Dmitry Mordvinov and Grigory Fedyukovich. 2017. Synchronizing constrained Horn clauses. LPAR, EPiC Series in Computing. EasyChair (2017).Google ScholarGoogle Scholar
  46. Saswat Padhi, Rahul Sharma, and Todd Millstein. 2016. Data-driven precondition inference with learned features. In PLDI, Vol. 51. ACM, 42–56.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. In NIPS-W.Google ScholarGoogle Scholar
  48. Amir Pnueli, Michael Siegel, and Eli Singerman. 1998. Translation Validation. In Proceedings of the 4th International Conference on Tools and Algorithms for Construction and Analysis of Systems (TACAS ’98). Springer-Verlag, Berlin, Heidelberg, 151–166.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Dan Quinlan and Chunhua Liao. 2011. The ROSE Source-to-Source Compiler Infrastructure. In Cetus Users and Compiler Infrastructure Workshop, in conjunction with PACT 2011.Google ScholarGoogle Scholar
  50. Mukund Raghothaman, Sulekha Kulkarni, Kihong Heo, and Mayur Naik. 2018. User-guided program reasoning using Bayesian inference. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 722–735.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Veselin Raychev, Pavol Bielik, and Martin Vechev. 2016a. Probabilistic model for code with decision trees. In OOPSLA, Vol. 51. ACM, 731–747.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Veselin Raychev, Pavol Bielik, Martin Vechev, and Andreas Krause. 2016b. Learning programs from noisy data. In POPL, Vol. 51. ACM, 761–774.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Veselin Raychev, Martin Vechev, and Andreas Krause. 2015. Predicting program properties from big code. In POPL, Vol. 50. ACM, 111–124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In PLDI, Vol. 49. ACM, 419–428.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Eric Schkufza, Rahul Sharma, and Alex Aiken. 2013. Stochastic superoptimization. In ASPLOS, Vol. 41. ACM, 305–316.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Eric Schkufza, Rahul Sharma, and Alex Aiken. 2014. Stochastic optimization of floating-point programs with tunable precision. In PLDI, Vol. 49. ACM, 53–64.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy optimization. In International Conference on Machine Learning. 1889–1897.Google ScholarGoogle Scholar
  58. Rahul Sharma and Alex Aiken. 2014. From invariant checking to invariant inference using randomized search. In CAV.Google ScholarGoogle Scholar
  59. Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, and Aditya V Nori. 2013. Verification as learning geometric concepts. In International Static Analysis Symposium. Springer, 388–411.Google ScholarGoogle ScholarCross RefCross Ref
  60. Xujie Si, Hanjun Dai, Mukund Raghothaman, Mayur Naik, and Le Song. 2018a. Learning loop invariants for program verification. In Advances in Neural Information Processing Systems. 7762–7773.Google ScholarGoogle Scholar
  61. Xujie Si, Yuan Yang, Hanjun Dai, Mayur Naik, and Le Song. 2018b. Learning a Meta-Solver for Syntax-Guided Program Synthesis. In ICLR.Google ScholarGoogle Scholar
  62. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the Game of Go with Deep Neural Networks and Tree Search. Nature 529, 7587 (Jan. 2016), 484–489.Google ScholarGoogle Scholar
  63. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017. Mastering the game of go without human knowledge. Nature 550, 7676 (2017), 354.Google ScholarGoogle Scholar
  64. Gagandeep Singh, Markus Püschel, and Martin Vechev. 2018. Fast Numerical Program Analysis with Reinforcement Learning. In International Conference on Computer Aided Verification. Springer, 211–229.Google ScholarGoogle Scholar
  65. Marcelo Sousa and Isil Dillig. 2016. Cartesian hoare logic for verifying k-safety properties. In Proc. Conference on Programming Language Design and Implementation. 57–69.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Marcelo Sousa, Isil Dillig, and Shuvendu Lahiri. 2018. Verifying Semantic Conflict-Freedom in Three-Way Program Merges. arXiv preprint arXiv:1802.06551 (2018).Google ScholarGoogle Scholar
  67. Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems. 1057–1063.Google ScholarGoogle Scholar
  69. Tachio Terauchi and Alex Aiken. 2005. Secure Information Flow As a Safety Problem. In Proceedings of the 12th International Conference on Static Analysis (SAS’05). 352–367.Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Geoffrey Towell and Jude W. Shavlik. 1992. Interpretation of Artificial Neural Networks: Mapping Knowledge-Based Neural Networks into Rules. In Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann (Eds.). Morgan-Kaufmann, 977–984.Google ScholarGoogle Scholar
  71. Mingzhe Wang, Yihe Tang, Jian Wang, and Jia Deng. 2017. Premise selection for theorem proving by deep graph embedding. In Advances in Neural Information Processing Systems. 2786–2796.Google ScholarGoogle Scholar
  72. Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. Deeppath: A reinforcement learning method for knowledge graph reasoning. In EMNLP.Google ScholarGoogle Scholar
  73. Hongseok Yang. 2007. Relational separation logic. Theoretical Computer Science 375, 1-3 (2007), 308–334.Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Anna Zaks and Amir Pnueli. 2008. Covac: Compiler validation by program analysis of the cross-product. In FM 2008: Formal Methods. Springer, 35–51.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Relational verification using reinforcement learning

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!