Editorial Notes
The authors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected Version of Record was published on March 14, 2023. For reference purposes, the VoR may still be accessed via the Supplemental Material section on this citation page.
Abstract
Formal methods have been widely used to support software testing to guarantee correctness and reliability. For example, model checking technology attempts to ensure that the verification property of a specific formal model is satisfactory for discovering bugs or abnormal behavior from the perspective of temporal logic. However, because automatic approaches are lacking, a software developer/tester must manually specify verification properties. A generative adversarial network (GAN) learns features from input training data and outputs new data with similar or coincident features. GANs have been successfully used in the image processing and text processing fields and achieved interesting and automatic results. Inspired by the power of GANs, in this article, we propose a GAN-based automatic property generation (GAPG) approach to generate verification properties supporting model checking. First, the verification properties in the form of computational tree logic (CTL) are encoded and used as input to the GAN. Second, we introduce regular expressions as grammar rules to check the correctness of the generated properties. These rules work to detect and filter meaningless properties that occur because the GAN learning process is uncontrollable and may generate unsuitable properties in real applications. Third, the learning network is further trained by using labeled information associated with the input properties. These are intended to guide the training process to generate additional new properties, particularly those that map to corresponding formal models. Finally, a series of comprehensive experiments demonstrate that the proposed GAPG method can obtain new verification properties from two aspects: (1) using only CTL formulas and (2) using CTL formulas combined with Kripke structures.
Supplemental Material
Available for Download
Version of Record for “A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective” by Gao et al., ACM Transactions on Multimedia Computing, Communications and Applications, Volume 19, No. 1 (TOMM 19:1).
- [1] . 2014. Extrinsic influence factors in software reliability: A study of 200,000 windows machines. In Proceedings of the 36th International Conference on Software Engineering. 205–214.Google Scholar
- [2] . 2013. Software and Systems Engineering Software Testing Part 1: Concepts and Definitions.
Technical Report . ISO/IEC/IEEE 29119-1.Google Scholar - [3] . 2019. A survey of formal specification application to safety critical systems. In Proceedings of the IEEE 2nd International Conference on Information and Computer Technologies (ICICT). IEEE, 296–302.Google Scholar
Cross Ref
- [4] . 2013. Verification of Communication Protocols in Web Services: Model-checking Service Compositions. John Wiley & Sons.Google Scholar
- [5] . 2019. Design of real-time control in poloidal field power supply based on finite-state machine. IEEE Trans. Plasma Sci. 47, 4 (2019), 1878–1883.Google Scholar
Cross Ref
- [6] . 2019. IEEE standard for the functional verification language e. IEEE Std 1647-2019 (Revision of IEEE Std 1647-2016) (2019), 1–622.
DOI: Google ScholarCross Ref
- [7] . 2018. Integrating GAN with CNN for face sketch synthesis. In Proceedings of the IEEE 4th International Conference on Computer and Communications (ICCC). IEEE, 1483–1487.Google Scholar
Cross Ref
- [8] . 2020. RPD-GAN: Learning to draw realistic paintings with generative adversarial network. IEEE Trans. Image Process. 29 (2020), 8706–8720.Google Scholar
Digital Library
- [9] . 2020. FGGAN: Feature-guiding generative adversarial networks for text generation. IEEE Access 8 (2020), 105217–105225.Google Scholar
Cross Ref
- [10] . 2017. Probabilistic model checking-based service selection method for business process modeling. Int. J. Softw. Eng. Knowl. Eng. 27, 06 (2017), 897–923.Google Scholar
Cross Ref
- [11] . 2018. Model checking techniques applied to satellite operational mode management. IEEE Syst. J. 13, 1 (2018), 1018–1029.Google Scholar
Cross Ref
- [12] . 2018. Automated quantitative verification for service-based system design: A visualization transform tool perspective. Int. J. Softw. Eng. Knowl. Eng. 28, 10 (2018), 1369–1397.Google Scholar
Cross Ref
- [13] . 2019. Transformation-based approach to security verification for cyber-physical systems. IEEE Syst. J. 13, 4 (2019), 3989–4000.Google Scholar
Cross Ref
- [14] . 2013. Predictive web service monitoring using probabilistic model checking. Appl. Math. Inf. Sci. 7, 1L (2013), 139–148.Google Scholar
Cross Ref
- [15] . 2018. A novel approach to modeling and verifying real-time systems for high reliability. IEEE Trans. Reliab. 67, 2 (2018), 481–493.Google Scholar
Cross Ref
- [16] . 2017. A formal specification and verification framework for timed security protocols. IEEE Trans. Softw. Eng. 44, 8 (2017), 725–746.Google Scholar
Digital Library
- [17] . 2019. A specification-based semi-formal functional verification method by a stage transition graph model. IEEE Access 7 (2019), 14947–14958.Google Scholar
Cross Ref
- [18] . 2017. Computational conversion via translation rules for transforming C++ code into UPPAAL’s automata. IEEE Access 5 (2017), 14455–14467.Google Scholar
Cross Ref
- [19] . 2021. Advances in verification of ReLU neural networks. J. Global Optim. 81, 1 (2021), 109–152.Google Scholar
Digital Library
- [20] . 2020. FANNet: Formal analysis of noise tolerance, training bias and input sensitivity in neural networks. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 666–669.Google Scholar
Cross Ref
- [21] . 2018. Model checking an artificial neural networks system in medical diagnosis. In Proceedings of the 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE, 852–856.Google Scholar
Cross Ref
- [22] . 2020. Verification of neural network behaviour: Formal guarantees for power system applications. IEEE Trans. Smart Grid 12, 1 (2020), 383–397.Google Scholar
Cross Ref
- [23] . 2020. Formal synthesis of Lyapunov neural networks. IEEE Contr. Syst. Lett. 5, 3 (2020), 773–778.Google Scholar
Cross Ref
- [24] . 2015. All-solution satisfiability modulo theories: Applications, algorithms and benchmarks. In Proceedings of the 10th International Conference on Availability, Reliability and Security. IEEE, 100–109.Google Scholar
Digital Library
- [25] . 2014. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014).Google Scholar
- [26] . 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. (2014).Google Scholar
- [27] . 2018. Semi-supervised text regression with conditional generative adversarial networks. In Proceedings of the IEEE International Conference on Big Data (Big Data). IEEE, 5375–5377.Google Scholar
Cross Ref
- [28] . 2018. DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems. In Proceedings of the 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 132–142.Google Scholar
Digital Library
- [29] . 2016. A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42, 9 (2016), 805–824.Google Scholar
Cross Ref
- [30] . 2015. How (not) to train your generative model: Scheduled sampling, likelihood, adversary?arXiv preprint arXiv:1511.05101 (2015).Google Scholar
- [31] . 2017. SeqGAN: Sequence generative adversarial nets with policy gradient. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2852–2858.Google Scholar
Cross Ref
- [32] . 2019. Improving conditional sequence generative adversarial networks by stepwise evaluation. IEEE/ACM Trans. Audio, Speech Lang. Process. 27, 4 (2019), 788–798.Google Scholar
Digital Library
- [33] . 2017. Four-valued modal logic: Kripke semantics and duality. J. Logic Computat. 27, 1 (2017), 155–199.Google Scholar
Cross Ref
- [34] . 2008. Principles of Model Checking. The MIT Press.Google Scholar
Digital Library
- [35] . 2019. Survey on reinforcement learning applications in communication networks. J. Commun. Inf. Netw. 4, 2 (2019), 30–39.Google Scholar
Cross Ref
- [36] . 2012. Monte Carlo beam search. IEEE Trans. Computat. Intell. AI Games 4, 1 (2012), 68–72.Google Scholar
Cross Ref
- [37] . 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489.Google Scholar
Cross Ref
- [38] . 2018. IEEE standard for fall protection for electric utility transmission and distribution on poles and structures. IEEE Std 1307-2018 (Revision of IEEE Std 1307-2004) (2018), 1–46.
DOI: Google ScholarCross Ref
- [39] . 2019. Economic LSTM approach for recurrent neural networks. IEEE Trans. Circ. Syst. II: Express Briefs 66, 11 (2019), 1885–1889.Google Scholar
Cross Ref
- [40] . 2016. Deep Learning. The MIT Press.Google Scholar
Digital Library
- [41] . 2019. FB-CNN: Feature fusion-based bilinear CNN for classification of fruit fly image. IEEE Access 8 (2019), 3987–3995.Google Scholar
Cross Ref
- [42] . 2019. A dilated CNN model for image classification. IEEE Access 7 (2019), 124087–124095.Google Scholar
Index Terms
A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective
Recommendations
Formal verification of ASMs using MDGs
We present a framework for the formal verification of abstract state machine (ASM) designs using the multiway decision graphs (MDG) tool. ASM is a state based language for describing transition systems. MDG provides symbolic representation of transition ...
Coverage metrics for formal verification
In formal verification, we verify that a system is correct with respect to a specification. Even when the system is proven to be correct, there is still a question of how complete the specification is and whether it really covers all the behaviors of ...
Fifteen Years of Formal Property Verification in Intel
25 Years of Model CheckingModel checking technologies have been applied to hardware verification in the last 15 years. Pioneering work has been conducted in Intel since 1990 using model checking technologies to build industrial hardware verification systems. This paper reviews ...






Comments