skip to main content
research-article

Formal Requirement Debugging for Testing and Verification of Cyber-Physical Systems

Published:12 December 2017Publication History
Skip Abstract Section

Abstract

A framework for the elicitation and debugging of formal specifications for Cyber-Physical Systems is presented. The elicitation of specifications is handled through a graphical interface. Two debugging algorithms are presented. The first checks for erroneous or incomplete temporal logic specifications without considering the system. The second can be utilized for the analysis of reactive requirements with respect to system test traces. The specification debugging framework is applied on a number of formal specifications collected through a user study. The user study establishes that requirement errors are common and that the debugging framework can resolve many insidious specification errors.

References

  1. Houssam Abbas, Georgios Fainekos, Sriram Sankaranarayanan, Franjo Ivančić, and Aarti Gupta. 2013. Probabilistic temporal logic falsification of cyber-physical systems. ACM Transactions on Embedded Computing Systems 12, 2s, Article 95 (May 2013), 30 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Rajeev Alur, Costas Courcoubetis, Nicolas Halbwachs, Thomas A. Henzinger, Pei-Hsin Ho, Xavier Nicollin, Alfredo Olivero, Joseph Sifakis, and Sergio Yovine. 1995. The algorithmic analysis of hybrid systems. Theoretical Computer Science 138, 1 (1995), 3--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Rajeev Alur, Tomás Feder, and Thomas A. Henzinger. 1996. The benefits of relaxing punctuality. Journal of the ACM 43, 1 (1996), 116--146. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Glenn Ammons, David Mandelin, Rastislav Bodík, and James R. Larus. 2003. Debugging temporal specifications with concept analysis. In Proceedings of the ACM SIGPLAN 2003 Conference on Programming Language Design and Implementation. 182--195. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Yashwanth Singh Rahul Annapureddy, Che Liu, Georgios E. Fainekos, and Sriram Sankaranarayanan. 2011. S-TaLiRo: A tool for temporal logic falsification for hybrid systems. In Tools and Algorithms for the Construction and Analysis of Systems (LNCS), Vol. 6605. Springer, 254--257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Marco Autili, Paola Inverardi, and Patrizio Pelliccione. 2007. Graphical scenarios for specifying temporal properties: An automated approach. Automated Software Engineering 14, 3 (2007), 293--340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Thomas Ball and Orna Kupferman. 2008. Vacuity in testing. In Proceedings of Tests and Proofs, Second International Conference (TAP’08). 4--17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Derek L. Beatty and Randal E. Bryant. 1994. Formally verifying a microprocessor using a simulation methodology. In Proceedings of the 31th Design Automation Conference (DAC’94). 596--602. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ilan Beer, Shoham Ben-David, Cindy Eisner, and Yoav Rodeh. 2001. Efficient detection of vacuity in temporal model checking. Formal Methods in System Design 18, 2 (2001), 141--163. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Shoham Ben-David, Fady Copty, Dana Fisman, and Sitvanit Ruah. 2015. Vacuity in practice: Temporal antecedent failure. Formal Methods in System Design 46, 1 (2015), 81--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Marcello M. Bersani, Matteo Rossi, and Pierluigi San Pietro. 2014. A logical characterization of timed (non-)regular languages. In Proceedings of the 39th International Symposium on Mathematical Foundations of Computer Science 2014, Part I (MFCS’14). 75--86.Google ScholarGoogle ScholarCross RefCross Ref
  12. Marcello M. Bersani, Matteo Rossi, and Pierluigi San Pietro. 2016. A tool for deciding the satisfiability of continuous-time metric temporal logic. Acta Informatica 53, 2 (2016), 171--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Marcello Maria Bersani, Matteo Rossi, and Pierluigi San Pietro. 2013. Deciding the satisfiability of MITL specifications. In Proceedings of the 4th International Symposium on Games, Automata, Logics and Formal Verification (EPTCS’13), Vol. 119. Open Publishing Association, 64--78.Google ScholarGoogle ScholarCross RefCross Ref
  14. Hana Chockler and Ofer Strichman. 2009. Before and after vacuity. Formal Methods in System Design 34, 1 (Feb. 2009), 37--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Alessandro Cimatti, Marco Roveri, Viktor Schuppan, and Andrei Tchaltsev. 2008. Diagnostic information for realizability. In Proceedings of the 9th International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI’08). 52--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Edmund M. Clarke, Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press, Cambridge, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Stéphane Demri and Deepak D’Souza. 2007. An automata-theoretic approach to constraint LTL. Information and Computation 205, 3 (2007), 380--415. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Adel Dokhanchi, Bardh Hoxha, and Georgios E. Fainekos. 2015. Metric interval temporal logic specification elicitation and debugging. In ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE’15). 70--79. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Adel Dokhanchi, Shakiba Yaghoubi, Bardh Hoxha, and Georgios Fainekos. 2017. Vacuity aware falsification for MTL request-response specifications. In Proceedings of the 13th IEEE Conference on Automation Science and Engineering (CASE’17).Google ScholarGoogle ScholarCross RefCross Ref
  20. Alexandre Donze. 2010. Breach, A toolbox for verification and parameter synthesis of hybrid systems. In Computer Aided Verification. LNCS, Vol. 6174. Springer, 167--170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Matthew B. Dwyer, George S. Avrunin, and James C. Corbett. 1998. Property specification patterns for finite-state verification. In Proceedings of the 2nd Workshop on Formal Methods in Software Practice (FMSP’98). ACM, 7--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Rüdiger Ehlers and Vasumathi Raman. 2014. Low-effort specification debugging and analysis. In Proceedings 3rd Workshop on Synthesis (SYNT’14).117--133.Google ScholarGoogle ScholarCross RefCross Ref
  23. Georgios Fainekos, Sriram Sankaranarayanan, Koichi Ueda, and Hakan Yazarel. 2012. Verification of automotive control applications using S-TaLiRo. In Proceedings of the American Control Conference.Google ScholarGoogle ScholarCross RefCross Ref
  24. Georgios E. Fainekos. 2011. Revising temporal logic specifications for motion planning. In IEEE International Conference on Robotics and Automation (ICRA’11). 40--45.Google ScholarGoogle ScholarCross RefCross Ref
  25. Dana Fisman, Orna Kupferman, Sarai Sheinvald-Faragy, and Moshe Y. Vardi. 2008. A framework for inherent vacuity. In Proceedings of the 4th International Haifa Verification Conference Hardware and Software: Verification and Testing (HVC’08). 7--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Klaus Havelund and Grigore Rosu. 2004. Efficient monitoring of safety properties. Journal on Software Tools for Technology Transfer 6, 2 (2004), 158--173.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Florian Horn, Wolfgang Thomas, Nico Wallmeier, and Martin Zimmermann. 2015. Optimal strategy synthesis for request-response games. RAIRO - Theoretical Informatics and Applications 49, 3 (2015), 179--203.Google ScholarGoogle ScholarCross RefCross Ref
  28. Bardh Hoxha, Houssam Abbas, and Georgios Fainekos. 2014a. Benchmarks for temporal logic requirements for automotive systems. In Proceedings of Applied Verification for Continuous and Hybrid Systems.Google ScholarGoogle Scholar
  29. Bardh Hoxha, Hoang Bach, Houssam Abbas, Adel Dokhanchi, Yoshihiro Kobayashi, and Georgios Fainekos. 2014b. Towards formal specification visualization for testing and monitoring of cyber-physical systems. In Proceedings of the International Workshop on Design and Implementation of Formal Tools and Systems.Google ScholarGoogle Scholar
  30. Bardh Hoxha, Nikolaos Mavridis, and Georgios Fainekos. 2015. ViSpec: A graphical tool for easy elicitation of MTL requirements. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.Google ScholarGoogle Scholar
  31. Xiaoqing Jin, Alexandre Donze, Jyotirmoy Deshmukh, and Sanjit Seshia. 2013. Mining requirements from closed-loop control models. In Hybrid Systems: Computation and Control. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. James Kapinski, Jyotirmoy V. Deshmukh, Xiaoqing Jin, Hisahiro Ito, and Kenneth R. Butts. 2015. Simulation-guided approaches for verification of automotive powertrain control systems. In Proceedings of the American Control Conference (ACC’15). 4086--4095.Google ScholarGoogle Scholar
  33. Kangjin Kim, Georgios E. Fainekos, and Sriram Sankaranarayanan. 2012. On the revision problem of specification automata. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’12). 5171--5176.Google ScholarGoogle ScholarCross RefCross Ref
  34. Robert Könighofer, Georg Hofferek, and Roderick Bloem. 2013. Debugging formal specifications: A practical approach using model-based diagnosis and counterstrategies. Journal on Software Tools for Technology Transfer 15, 5--6 (2013), 563--583.Google ScholarGoogle Scholar
  35. Sascha Konrad and Betty H. C. Cheng. 2005. Real-time specification patterns. In Proceedings of the 27th International Conference on Software Engineering (ICSE’05). ACM, 372--381. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Ron Koymans. 1990. Specifying real-time properties with metric temporal logic.Real-Time Systems 2, 4 (1990), 255--299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Orna Kupferman and Moshe Y. Vardi. 2003. Vacuity detection in temporal model checking. Journal on Software Tools for Technology Transfer 4, 2 (2003), 224--233.Google ScholarGoogle ScholarCross RefCross Ref
  38. Jianwen Li, Lijun Zhang, Geguang Pu, Moshe Y. Vardi, and Jifeng He. 2013. LTL satisfiability checking revisited. In 2013 20th International Symposium on Temporal Representation and Reasoning. 91--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Oded Maler and Dejan Nickovic. 2004. Monitoring temporal properties of continuous signals. In Proceedings of FORMATS-FTRTFT (LNCS), Vol. 3253. 152--166.Google ScholarGoogle Scholar
  40. Roland Meyer, Johannes Faber, Jochen Hoenicke, and Andrey Rybalchenko. 2008. Model checking duration calculus: A practical approach. Formal Aspects of Computing 20, 4 (2008), 481--505. Google ScholarGoogle ScholarCross RefCross Ref
  41. Amalinda Post, Jochen Hoenicke, and Andreas Podelski. 2011. Vacuous real-time requirements. In 19th IEEE International Requirements Engineering Conference (RE’11). 153--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Vasumathi Raman and Hadas Kress-Gazit. 2011. Analyzing unsynthesizable specifications for high-level robot behavior using LTLMoP. In Proceedings of the 23rd International Conference on Computer Aided Verification (CAV’11). 663--668. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Kristin Y. Rozier and Moshe Y. Vardi. 2010. LTL satisfiability checking. Journal on Software Tools for Technology Transfer 12, 2 (2010), 123--137.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Hengyi Yang, Bardh Hoxha, and Georgios Fainekos. 2012. Querying parametric temporal logic properties on embedded systems. In Testing Software and Systems. Springer, 136--151.Google ScholarGoogle Scholar
  45. Pengcheng Zhang, Bixin Li, and Lars Grunske. 2010. Timed property sequence chart. Journal of Systems and Software 83, 3 (2010), 371--390. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Formal Requirement Debugging for Testing and Verification of Cyber-Physical Systems

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Embedded Computing Systems
        ACM Transactions on Embedded Computing Systems  Volume 17, Issue 2
        Special Issue on MEMCODE 2015 and Regular Papers (Diamonds)
        March 2018
        640 pages
        ISSN:1539-9087
        EISSN:1558-3465
        DOI:10.1145/3160927
        Issue’s Table of Contents

        Copyright © 2017 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 12 December 2017
        • Accepted: 1 September 2017
        • Revised: 1 April 2017
        • Received: 1 January 2016
        Published in tecs Volume 17, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!