skip to main content
research-article
Open Access

Incremental predicate analysis for regression verification

Published:13 November 2020Publication History
Skip Abstract Section

Abstract

Software products are evolving during their life cycles. Ideally, every revision need be formally verified to ensure software quality. Yet repeated formal verification requires significant computing resources. Verifying each and every revision can be very challenging. It is desirable to ameliorate regression verification for practical purposes. In this paper, we regard predicate analysis as a process of assertion annotation. Assertion annotations can be used as a certificate for the verification results. It is thus a waste of resources to throw them away after each verification. We propose to reuse the previously-yielded assertion annotation in regression verification. A light-weight impact-analysis technique is proposed to analyze the reusability of assertions. A novel assertion strengthening technique is furthermore developed to improve reusability of annotation. With these techniques, we present an incremental predicate analysis technique for regression verification. Correctness of our incremental technique is formally proved. We performed comprehensive experiments on revisions of Linux kernel device drivers. Our technique outperforms the state-of-the-art program verification tool CPAchecker by getting 2.8x speedup in total time and solving additional 393 tasks.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

Software products are evolving during their life cycles. Ideally, every revision need be formally verified to ensure software quality. Yet repeated formal verification requires significant computing resources. Verifying each and every revision can be very challenging. It is desirable to ameliorate regression verification for practical purposes. In this paper, we regard predicate analysis as a process of assertion annotation. Assertion annotations can be used as a certificate for the verification results. It is thus a waste of resources to throw them away after each verification. We propose to reuse the previously-yielded assertion annotation in regression verification. A light-weight impact-analysis technique is proposed to analyze the reusability of assertions. A novel assertion strengthening technique is furthermore developed to improve reusability of annotation. With these techniques, we present an incremental predicate analysis technique for regression verification.

References

  1. June Andronick, Ross Jefery, Gerwin Klein, Rafal Kolanski, Mark Staples, He Zhang, and Liming Zhu. 2012. Large-Scale Formal Verification in Practice: A Process Perspective. In Proceedings of the 34th International Conference on Software Engineering (Zurich, Switzerland) (ICSE '12). IEEE Press, New York, NY, USA, 1002-1011. https://doi.org/10.1109/ICSE. 2012.6227120 Google ScholarGoogle ScholarCross RefCross Ref
  2. Giuliano Antoniol, Gerardo Canfora, Gerardo Casazza, and Andrea De Lucia. 2000. Identifying the Starting Impact Set of a Maintenance Request: A Case Study. In 4th European Conference on Software Maintenance and Reengineering (Zurich, Switzerland) (CSMR 2000 ). IEEE Press, New York, NY, USA, 227-230. https://doi.org/10.1109/CSMR. 2000.827331 Google ScholarGoogle ScholarCross RefCross Ref
  3. Taweesup Apiwattanapong, Alessandro Orso, and Mary Jean Harrold. 2005. Eficient and Precise Dynamic Impact Analysis Using Execute-after Sequences. In Proceedings of the 27th International Conference on Software Engineering (St. Louis, MO, USA) ( ICSE '05). Association for Computing Machinery, New York, NY, USA, 432-441. https://doi.org/10.1145/1062455. 1062534 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Andrea Aquino, Francesco A. Bianchi, Meixian Chen, Giovanni Denaro, and Mauro Pezzè. 2015. Reusing Constraint Proofs in Program Analysis. In Proceedings of the 2015 International Symposium on Software Testing and Analysis (Baltimore, MD, USA) ( ISSTA 2015 ). Association for Computing Machinery, New York, NY, USA, 305-315. https: //doi.org/10.1145/2771783.2771802 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Robert S. Arnold. 1996. Software Change Impact Analysis. IEEE Computer Society Press, Washington, DC, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Thomas Ball, Rupak Majumdar, Todd Millstein, and Sriram K. Rajamani. 2001. Automatic Predicate Abstraction of C Programs. In Proceedings of the ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation (Snowbird, Utah, USA) ( PLDI '01). Association for Computing Machinery, New York, NY, USA, 203-213. https://doi.org/ 10.1145/378795.378846 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Thomas Ball and Sriram K. Rajamani. 2001. The SLAM Toolkit. In Computer Aided Verification. Springer, Berlin, Heidelberg, 260-264. https://doi.org/10.1007/3-540-44585-4_25 Google ScholarGoogle ScholarCross RefCross Ref
  8. Clark Barrett, Aaron Stump, Cesare Tinelli, et al. 2010. The SMT-LIB standard: Version 2.0. In Proceedings of the 8th International Workshop on Satisfiability Modulo Theories (Edinburgh, England), Vol. 13. 14.Google ScholarGoogle Scholar
  9. Dirk Beyer, Thomas A. Henzinger, and Grégory Théoduloz. 2007. Configurable Software Verification: Concretizing the Convergence of Model Checking and Program Analysis. In Computer Aided Verification. Springer, Berlin, Heidelberg, 504-518. https://doi.org/10.1007/978-3-540-73368-3_51 Google ScholarGoogle ScholarCross RefCross Ref
  10. Dirk Beyer and M. Erkan Keremoglu. 2011. CPAchecker: A Tool for Configurable Software Verification. In Computer Aided Verification. Springer, Berlin, Heidelberg, 184-190. https://doi.org/10.1007/978-3-642-22110-1_16 Google ScholarGoogle ScholarCross RefCross Ref
  11. Dirk Beyer, M. Erkan Keremoglu, and Philipp Wendler. 2010. Predicate Abstraction with Adjustable-Block Encoding. In Proceedings of the 2010 Conference on Formal Methods in Computer-Aided Design (Lugano, Switzerland) ( FMCAD '10). FMCAD Inc, Austin, Texas, 189-198. https://doi.org/10.5555/1998496.1998532 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Dirk Beyer, Stefan Löwe, Evgeny Novikov, Andreas Stahlbauer, and Philipp Wendler. 2013. Precision Reuse for Eficient Regression Verification. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (Saint Petersburg, Russia) (ESEC/FSE 2013 ). Association for Computing Machinery, New York, NY, USA, 389-399. https: //doi.org/10.1145/2491411.2491429 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Dirk Beyer, Damien Zuferey, and Rupak Majumdar. 2008. CSIsat: Interpolation for LA+EUF. In Computer Aided Verification. Springer, Berlin, Heidelberg, 304-308. https://doi.org/10.1007/978-3-540-70545-1_29 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ben Breech, Anthony Danalis, Stacey A. Shindo, and Lori L. Pollock. 2004. Online Impact Analysis via Dynamic Compilation Technology. In 20th International Conference on Software Maintenance (Chicago, IL, USA) ( ICSM '04). IEEE Press, New York, NY, USA, 453-457. https://doi.org/10.1109/ICSM. 2004.1357834 Google ScholarGoogle ScholarCross RefCross Ref
  15. Sagar Chaki, Arie Gurfinkel, and Ofer Strichman. 2012. Regression Verification for Multi-threaded Programs. In Verification, Model Checking, and Abstract Interpretation. Springer, Berlin, Heidelberg, 119-135. https://doi.org/10.1007/978-3-642-27940-9_9 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. M.Ajmal Chaumun, Hind Kabaili, Rudolf K. Keller, and François Lustman. 2002. A change impact model for changeability assessment in object-oriented software systems. Science of Computer Programming 45, 2 ( 2002 ), 155-174. https: //doi.org/10.1016/S0167-6423 ( 02 ) 00058-8 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jürgen Christ, Jochen Hoenicke, and Alexander Nutz. 2012. SMTInterpol: An Interpolating SMT Solver. In Model Checking Software. Springer, Berlin, Heidelberg, 248-254. https://doi.org/10.1007/978-3-642-31759-0_19 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Alessandro Cimatti, Alberto Griggio, and Roberto Sebastiani. 2008. Eficient Interpolant Generation in Satisfiability Modulo Theories. In Tools and Algorithms for the Construction and Analysis of Systems. Springer, Berlin, Heidelberg, 397-412. https://doi.org/10.1007/978-3-540-78800-3_30 Google ScholarGoogle ScholarCross RefCross Ref
  19. Edmund Clarke, Orna Grumberg, Somesh Jha, Yuan Lu, and Helmut Veith. 2000. Counterexample-Guided Abstraction Refinement. In Computer Aided Verification. Springer, Berlin, Heidelberg, 154-169. https://doi.org/10.1007/10722167_15 Google ScholarGoogle ScholarCross RefCross Ref
  20. Patrick Cousot and Radhia Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (Los Angeles, California) ( POPL '77). Association for Computing Machinery, New York, NY, USA, 238-252. https://doi.org/10.1145/512950.512973 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Satyaki Das, David L. Dill, and Seungjoon Park. 1999. Experience with Predicate Abstraction. In Computer Aided Verification. Springer, Berlin, Heidelberg, 160-171. https://doi.org/10.1007/3-540-48683-6_16 Google ScholarGoogle ScholarCross RefCross Ref
  22. V. D'Silva, D. Kroening, and G. Weissenbacher. 2008. A Survey of Automated Techniques for Formal Software Verification. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 27, 7 ( 2008 ), 1165-1178. https://doi.org/ 10.1109/TCAD. 2008.923410 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Grigory Fedyukovich, Arie Gurfinkel, and Natasha Sharygina. 2014. Incremental Verification of Compiler Optimizations. In NASA Formal Methods. Springer, Berlin, Heidelberg, 300-306. https://doi.org/10.1007/978-3-319-06200-6_25 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Grigory Fedyukovich, Arie Gurfinkel, and Natasha Sharygina. 2016. Property Directed Equivalence via Abstract Simulation. In Computer Aided Verification. Springer, Berlin, Heidelberg, 433-453. https://doi.org/10.1007/978-3-319-41540-6_24 Google ScholarGoogle ScholarCross RefCross Ref
  25. Grigory Fedyukovich, Ondrej Sery, and Natasha Sharygina. 2013. eVolCheck: Incremental Upgrade Checker for C. In Tools and Algorithms for the Construction and Analysis of Systems. Springer, Berlin, Heidelberg, 292-307. https://doi.org/10.1007/978-3-642-36742-7_21 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Keith Brian Gallagher and James R. Lyle. 1991. Using program slicing in software maintenance. IEEE transactions on software engineering 17, 8 ( 1991 ), 751-761. https://doi.org/10.1.1.39.1532Google ScholarGoogle Scholar
  27. Patrice Godefroid, Shuvendu K. Lahiri, and Cindy Rubio-González. 2011. Statically Validating Must Summaries for Incremental Compositional Dynamic Test Generation. In Static Analysis. Springer, Berlin, Heidelberg, 112-128. https: //doi.org/10.1007/978-3-642-23702-7_12 Google ScholarGoogle ScholarCross RefCross Ref
  28. Benny Godlin and Ofer Strichman. 2009. Regression Verification. In Proceedings of the 46th Annual Design Automation Conference (San Francisco, California) ( DAC '09). Association for Computing Machinery, New York, NY, USA, 466-471. https://doi.org/10.1145/1629911.1630034 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Susanne Graf and Hassen Saidi. 1997. Construction of abstract state graphs with PVS. In Computer Aided Verification. Springer, Berlin, Heidelberg, 72-83. https://doi.org/10.1007/3-540-63166-6_10 Google ScholarGoogle ScholarCross RefCross Ref
  30. Fei He, Shu Mao, and Bow-Yaw Wang. 2016. Learning-Based Assume-Guarantee Regression Verification. In Computer Aided Verification. Springer, Berlin, Heidelberg, 310-328. https://doi.org/10.1007/978-3-319-41528-4_17 Google ScholarGoogle ScholarCross RefCross Ref
  31. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Kenneth L. McMillan. 2004. Abstractions from Proofs. In Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (Venice, Italy) (POPL '04). Association for Computing Machinery, New York, NY, USA, 232-244. https://doi.org/10.1145/964001.964021 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Marco A. A. Sanvido. 2003a. Extreme Model Checking. Springer, Berlin, Heidelberg, 332-358. https://doi.org/10.1007/978-3-540-39910-0_16 Google ScholarGoogle ScholarCross RefCross Ref
  33. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2002. Lazy Abstraction. In Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (Portland, Oregon) (POPL '02). Association for Computing Machinery, New York, NY, USA, 58-70. https://doi.org/10.1145/503272.503279 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2003b. Software Verification with BLAST. In Model Checking Software. Springer, Berlin, Heidelberg, 235-239. https://doi.org/10.1007/3-540-44829-2_17 Google ScholarGoogle ScholarCross RefCross Ref
  35. Xiangyang Jia, Carlo Ghezzi, and Shi Ying. 2015. Enhancing Reuse of Constraint Solutions to Improve Symbolic Execution. In Proceedings of the 2015 International Symposium on Software Testing and Analysis (Baltimore, MD, USA) ( ISSTA 2015 ). Association for Computing Machinery, New York, NY, USA, 177-187. https://doi.org/10.1145/2771783.2771806 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Alexey Khoroshilov, Vadim Mutilin, Alexander Petrenko, and Vladimir Zakharov. 2010. Establishing Linux Driver Verification Process. In Perspectives of Systems Informatics. Springer, Berlin, Heidelberg, 165-176. https://doi.org/10.1007/978-3-642-11486-1_14 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Vladimir Klebanov, Philipp Rümmer, and Mattias Ulbrich. 2018. Automating regression verification of pointer programs by predicate abstraction. Formal methods in system design 52, 3 ( 2018 ), 229-259. https://doi.org/10.1007/s10703-017-0293-8 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Shuvendu K. Lahiri, Kenneth L. McMillan, Rahul Sharma, and Chris Hawblitzel. 2013. Diferential Assertion Checking. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (Saint Petersburg, Russia) (ESEC/FSE 2013 ). Association for Computing Machinery, New York, NY, USA, 345-355. https://doi.org/10.1145/2491411.2491452 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Steven Lauterburg, Ahmed Sobeih, Darko Marinov, and Mahesh Viswanathan. 2008. Incremental State-Space Exploration for Programs with Dynamically Allocated Data. In Proceedings of the 30th International Conference on Software Engineering (Leipzig, Germany) ( ICSE '08). Association for Computing Machinery, New York, NY, USA, 291-300. https://doi.org/10. 1145/1368088.1368128 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. M. M. Lehman and L. A. Belady. 1985. Program Evolution: Processes of Software Change. Academic Press Professional, Inc., USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Manny M Lehman, Dewayne E Perry, and Juan F Ramil. 1998. Implications of evolution metrics on software maintenance. In Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272 ) (Bethesda, Maryland, USA) ( ICSM '98). IEEE Press, New York, NY, USA, 208-217. https://doi.org/10.1109/ICSM. 1998.738510 Google ScholarGoogle ScholarCross RefCross Ref
  42. Meir M Lehman, Juan F Ramil, Paul D Wernick, Dewayne E Perry, and Wladyslaw M Turski. 1997. Metrics and laws of software evolution-the nineties view. In Proceedings Fourth International Software Metrics Symposium (Albuquerque, NM, USA). IEEE Press, New York, NY, USA, 20-32. https://doi.org/10.1109/METRIC. 1997.637156 Google ScholarGoogle ScholarCross RefCross Ref
  43. Mikhail U Mandrykin, Vadim S Mutilin, EM Novikov, Alexey V Khoroshilov, and PE Shved. 2012. Using Linux device drivers for static verification tools benchmarking. Programming and Computer Software 38, 5 ( 2012 ), 245-256. https: //doi.org/10.1134/S0361768812050039 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Kenneth L. McMillan. 2006. Lazy Abstraction with Interpolants. In Computer Aided Verification. Springer, Berlin, Heidelberg, 123-136. https://doi.org/10.1007/11817963_14 Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Sergey Melnik, Hector Garcia-Molina, and Erhard Rahm. 2002. Similarity flooding: A versatile graph matching algorithm and its application to schema matching. In Proceedings 18th International Conference on Data Engineering (San Jose, CA, USA). IEEE Press, New York, NY, USA, 117-128. https://doi.org/10.1109/ICDE. 2002.994702 Google ScholarGoogle ScholarCross RefCross Ref
  46. Flemming Nielson, Hanne R Nielson, and Chris Hankin. 2015. Principles of program analysis. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-03811-6 Google ScholarGoogle ScholarCross RefCross Ref
  47. Alessandro Orso, Taweesup Apiwattanapong, and Mary Jean Harrold. 2003. Leveraging Field Data for Impact Analysis and Regression Testing. In Proceedings of the 9th European Software Engineering Conference Held Jointly with 11th ACM SIGSOFT International Symposium on Foundations of Software Engineering (Helsinki, Finland) (ESEC/FSE-11). Association for Computing Machinery, New York, NY, USA, 128-137. https://doi.org/10.1145/940071.940089 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Bat-Chen Rothenberg, Daniel Dietsch, and Matthias Heizmann. 2018. Incremental Verification Using Trace Abstraction. In Static Analysis. Springer, Berlin, Heidelberg, 364-382. https://doi.org/10.1007/978-3-319-99725-4_22 Google ScholarGoogle ScholarCross RefCross Ref
  49. Barbara G. Ryder and Frank Tip. 2001. Change Impact Analysis for Object-Oriented Programs. In Proceedings of the 2001 ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (Snowbird, Utah, USA) ( PASTE '01). Association for Computing Machinery, New York, NY, USA, 46-53. https://doi.org/10.1145/379605.379661 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Ondrej Sery, Grigory Fedyukovich, and Natasha Sharygina. 2012. Incremental upgrade checking by means of interpolationbased function summaries. In Formal Methods in Computer-Aided Design (FMCAD), 2012 (Cambridge, UK). IEEE Press, New York, NY, USA, 114-121. http://ieeexplore.ieee.org/document/6462563/Google ScholarGoogle Scholar
  51. Wladyslaw M Turski. 1996. Reference Model for Smooth Growth of Software Systems. IEEE Trans. Softw. Eng. 22, 8 (Aug. 1996 ), 1. https://doi.org/10.1109/32.536959 Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Willem Visser, Jaco Geldenhuys, and Matthew B. Dwyer. 2012. Green: Reducing, Reusing and Recycling Constraints in Program Analysis. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (Cary, North Carolina) (FSE '12). Association for Computing Machinery, New York, NY, USA, Article 58, 11 pages. https://doi.org/10.1145/2393596.2393665 Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Daniel Wonisch and Heike Wehrheim. 2012. Predicate Analysis with Block-Abstraction Memoization. In Formal Methods and Software Engineering. Springer, Berlin, Heidelberg, 332-347. https://doi.org/10.1007/978-3-642-34281-3_24 Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Guowei Yang, Matthew B Dwyer, and Gregg Rothermel. 2009. Regression model checking. In Software Maintenance, 2009. ICSM 2009. IEEE International Conference on. IEEE Press, New York, NY, USA, 115-124. https://doi.org/10.1109/ICSM. 2009.5306334 Google ScholarGoogle ScholarCross RefCross Ref
  55. Guowei Yang, Suzette Person, Neha Rungta, and Sarfraz Khurshid. 2014. Directed Incremental Symbolic Execution. ACM Trans. Softw. Eng. Methodol. 24, 1, Article 3 (Oct. 2014 ), 42 pages. https://doi.org/10.1145/2629536 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Incremental predicate analysis for regression verification

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!