Abstract
We present a Monte-Carlo optimization technique for finding system behaviors that falsify a metric temporal logic (MTL) property. Our approach performs a random walk over the space of system inputs guided by a robustness metric defined by the MTL property. Robustness is guiding the search for a falsifying behavior by exploring trajectories with smaller robustness values. The resulting testing framework can be applied to a wide class of cyber-physical systems (CPS). We show through experiments on complex system models that using our framework can help automatically falsify properties with more consistency as compared to other means, such as uniform sampling.
- Abbas, H. and Fainekos, G. 2011a. Linear hybrid system falsification through descent. Tech. rep. arXiv:1105.1733.Google Scholar
- Abbas, H. and Fainekos, G. 2011b. Linear hybrid system falsification through local search. In Automated Technology for Verification and Analysis. Lecture Notes in Computer Science, vol. 6996. Springer, 503--510. Google Scholar
Digital Library
- Alur, R., Dang, T., and Ivančić, F. 2003. Progress on reachability analysis of hybrid systems using predicate abstraction. In Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 2623. Springer, 4--19. Google Scholar
Digital Library
- Alur, R., Henzinger, T. A., Lafferriere, G., and Pappas, G. J. 2000. Discrete abstractions of hybrid systems. Proc. IEEE 88, 2, 971--984.Google Scholar
- Alur, R., Courcoubetis, C., Halbwachs, N., Henzinger, T. A., Ho, P.-H., Nicollin, X., Olivero, A., Sifakis, J., and Yovine, S. 1995. The algorithmic analysis of hybrid systems. Theor. Comput. Sci. 138, 1, 3--34. Google Scholar
Digital Library
- Alur, R., Dang, T., Esposito, J., Hur, Y., Ivancic, F., Kumar, V., Lee, I., Mishra, P., Pappas, G. J., and Sokolsky, O. 2003. Hierarchical modeling and analysis of embedded systems. Proc. IEEE 91, 1, 11--28.Google Scholar
Cross Ref
- Andrieu, C., Freitas, N. D., Doucet, A., and Jordan, M. I. 2003. An introduction to MCMC for machine learning. Machine Learn. 50, 5--43.Google Scholar
Cross Ref
- Annapureddy, Y. S. R., Liu, C., Fainekos, G. E., and Sankaranarayanan, S. 2011. S-taliro: A tool for temporal logic falsification for hybrid systems. In Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, vol. 6605. Springer, 254--257. Google Scholar
Digital Library
- Bandemer, H. and Gottwald, S. 1995. Fuzzy Sets, Fuzzy Logic, Fuzzy Methods, with Applications. Wiley, New York, NY.Google Scholar
- Bhatia, A. and Frazzoli, E. 2004. Incremental search methods for reachability analysis of continuous and hybrid systems. In Proceedings of HSCC. Lecture Notes in Computer Science, vol. 2993. Springer, 142--156.Google Scholar
- Boyd, S. and Vandenberghe, S. 2004. Convex Optimization. Cambridge University Press. http://www.stanford.edu/ boyd/cvxbook.html. Google Scholar
Digital Library
- Branicky, M., Curtiss, M., Levine, J., and Morgan, S. 2006. Sampling-based planning, control and verification of hybrid systems. IEE Control Theory Appl. 153, 5, 575--590.Google Scholar
Cross Ref
- Chib, S. and Greenberg, E. 1995. Understanding the Metropolis-Hastings algorithm. Amer. Statistician 49, 4, 327--335.Google Scholar
- Clarke, E., Donze, A., and Legay, A. 2009. Statistical model checking of analog mixed-signal circuits with an application to a third order Δ − σ modulator. In Hardware and Software: Verification and Testing. Lecture Notes in Computer Science, vol. 5394/2009. 149--163. Google Scholar
Digital Library
- Cormen, T. H., Leiserson, C. E., Rivest, R. L., and Stein, C. 2001. Introduction to Algorithms 2nd Ed. MIT Press/McGraw-Hill. Google Scholar
Digital Library
- Dang, T., Donzé, A., and Maler, O. 2004. Verification of analog and mixed-signal circuits using hybrid system techniques. In Proceedings of the 5th International Conference on Formal Methods in Computer-Aided Design. Lecture Notes in Computer Science, vol. 3312. Springer, 21--36.Google Scholar
- Dang, T., Donze, A., Maler, O., and Shalev, N. 2008. Sensitive state-space exploration. In Proceedings of the 47th IEEE CDC. 4049--4054.Google Scholar
- de Alfaro, L., Faella, M., and Stoelinga, M. 2004. Linear and branching metrics for quantitative transition systems. In Proceedings of the 31st ICALP. Lecture Notes in Computer Science, vol. 3142. Springer, 97--109.Google Scholar
- Donzé, A. and Maler, O. 2007. Systematic simulation using sensitivity analysis. In Proceeding of HSCC. Lecture Notes in Computer Science, vol. 4416. Springer, 174--189. Google Scholar
Digital Library
- Egerstedt, M. and Martin, C. 2009. Control Theoretic Splines: Optimal Control, Statistics, and Path Planning. Princeton University Press, Princeton, NJ.Google Scholar
- Esposito, J. M. and Kumar, V. 2004. An asynchronous integration and event detection algorithm for simulating multi-agent hybrid systems. ACM Trans. Model. Comput. Simul. 14, 4, 363--388. Google Scholar
Digital Library
- Esposito, J. M., Kim, J., and Kumar, V. 2004. Adaptive RRTs for validating hybrid robotic control systems. In Proceedings of the International Workshop on the Algorithmic Foundations of Robotics.Google Scholar
- Esterel Technologies. 2011. Scade success stories. http://www.esterel-technologies.com/technology/success-stories/.Google Scholar
- Fainekos, G. E. and Pappas, G. J. 2006. Robustness of temporal logic specifications for finite state sequences in metric spaces. Tech. rep. MS-CIS-06-05, Dept. of CIS, Univ. of Pennsylvania.Google Scholar
- Fainekos, G. E. and Pappas, G. J. 2009. Robustness of temporal logic specifications for continuous-time signals. Theor. Comput. Sci. 410, 42, 4262--4291. Google Scholar
Digital Library
- Fainekos, G. E., Girard, A., and Pappas, G. J. 2006. Temporal logic verification using simulation. In Proceedings of FORMATS. Lecture Notes in Computer Science, vol. 4202. Springer, 171--186. Google Scholar
Digital Library
- Fainekos, G. E., Sankaranarayanan, S., Ivančić, F., and Gupta, A. 2009. Robustness of model-based simulations. In Proceedings of the IEEE Real-Time Systems Symposium. 345--354. Google Scholar
Digital Library
- Fehnker, A. and Ivančić, F. 2004. Benchmarks for hybrid systems verification. In Proceedings of HSCC. Lecture Notes in Computer Science, vol. 2993. Springer, 326--341.Google Scholar
- Frehse, G., Guernic, C. L., Donz, A., Cotton, S., Ray, R., Lebeltel, O., Ripado, R., Girard, A., Dang, T., and Maler, O. 2011. Spaceex: Scalable verification of hybrid systems. In Proceedings of the 23rd CAV. Google Scholar
Digital Library
- Frenkel, D. and Smit, B. 1996. Understanding Molecular Simulation: From Algorithms to Applications. Academic Press, Walthan, MA. Google Scholar
Digital Library
- Girard, A. and Pappas, G. J. 2006. Verification using simulation. In Proceedings of HSCC. Lecture Notes in Computer Science, vol. 3927. Springer, 272--286. Google Scholar
Digital Library
- Grosu, R. and Smolka, S. 2005. Monte carlo model checking. In Proceedings of TACAS. Lecture Notes in Computer Science, vol. 3440. Springer, 271--286. Google Scholar
Digital Library
- Henzinger, T. A. 1996. The theory of hybrid automata. In Proceedings of LICS. 278--292. Google Scholar
Digital Library
- Henzinger, T. A., Kopke, P. W., Puri, A., and Varaiya, P. 1998. What’s decidable about hybrid automata? J. Comput. Syst. Sci. 57, 1, 94--124. Google Scholar
Digital Library
- Julius, A. A., Fainekos, G. E., Anand, M., Lee, I., and Pappas, G. J. 2007. Robust test generation and coverage for hybrid systems. In Proceedings of HSCC. Lecture Notes in Computer Science, vol. 4416. Springer, 329--342. Google Scholar
Digital Library
- Kapinski, J., Krogh, B. H., Maler, O., and Stursberg, O. 2003. On systematic simulation of open continuous systems. In Proceedings of HSCC. Lecture Notes in Computer Science, vol. 2623. Springer, 283--297. Google Scholar
Digital Library
- Kopperman, R. 1988. All topologies come from generalized metrics. Amer. Math. Month. 95, 89--97. Google Scholar
Digital Library
- Koymans, R. 1990. Specifying real-time properties with metric temporal logic. Real-Time Syst. 2, 4, 255--299. Google Scholar
Digital Library
- Lamine, K. B. and Kabanza, F. 2000. Using fuzzy temporal logic for monitoring behavior-based mobile robots. In Proceedings of the IASTED International Conference Robotics and Applications, M. Hamza Ed., 116--122.Google Scholar
- Lee, E. A. and Varaiya, P. 2003. Structure and Interpretation of Signals and Systems. Addison Wesley, Reading, MA.Google Scholar
- Lerda, F., Kapinski, J., Clarke, E. M., and Krogh, B. H. 2008. Verification of supervisory control software using state proximity and merging. In Proceedings of HSCC. Lecture Notes in Computer Science, vol. 4981. Springer, 344--357. Google Scholar
Digital Library
- Lovasz, L. and Vempala, S. 2003. Hit-and-run is fast and fun. Tech rep. MSR-TR-2003.05. http://www-math.mit.edu/ vempala/papers/logcon-hitrun.ps.Google Scholar
- Lovasz, L. and Vempala, S. 2006. Hit-and-run from a corner. SIAM J. Comput. 35, 4, 985--1005. Google Scholar
Digital Library
- Lygeros, J., Johansson, K. H., Simic, S. N., Zhang, J., and Sastry, S. 2003. Dynamical properties of hybrid automata. IEEE Trans. Autom. Control 48, 2--17.Google Scholar
- Mathworks. 2011. Simulink user stories. http://www.mathworks.com/products/simulink/userstories.html.Google Scholar
- Nahhal, T. and Dang, T. 2007. Test coverage for continuous and hybrid systems. In Proceedings of CAV. Lecture Notes in Computer Science, vol. 4590. Springer, 449--462. Google Scholar
Digital Library
- Nghiem, T., Sankaranarayanan, S., Fainekos, G. E., Ivancic, F., Gupta, A., and Pappas, G. J. 2010. Monte-carlo techniques for falsification of temporal properties of non-linear hybrid systems. In Proceedings of the 13th ACM International Conference on Hybrid Systems: Computation and Control. 211--220. Google Scholar
Digital Library
- Plaku, E., Kavraki, L. E., and Vardi, M. Y. 2007. Hybrid systems: From verification to falsification. In Proceedings of CAV. Lecture Notes in Computer Science, vol. 4590. Springer, 463--476. Google Scholar
Digital Library
- Plaku, E., Kavraki, L. E., and Vardi, M. Y. 2009. Falsification of ltl safety properties in hybrid systems. In Proceedings of TACAS. Lecture Notes in Computer Science, vol. 5505. Springer, 368--382. Google Scholar
Digital Library
- Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T. 1992. Numerical Recipes: The Art of Scientific Computing 2nd Ed. Cambridge University Press.Google Scholar
Digital Library
- Randall, D. 2006. Rapidly mixing markov chains with applications in computer science and physics. Comput. Sci. Eng. 8, 2. Google Scholar
Digital Library
- Rizk, A., Batt, G., Fages, F., and Soliman, S. 2008. On a continuous degree of satisfaction of temporal logic formulae with applications to systems biology. In Proceeding of the 6th International Conference on Computational Methods in Systems Biology. Lecture Notes in Computer Science, vol. 5307. Springer, 251--268. Google Scholar
Digital Library
- Romeijn, H. and Smith, R. 1994. Simulated annealing for constrained global optimization. J. Global Optim. 5, 101--126.Google Scholar
Cross Ref
- Rubinstein, R. Y. and Kroese, D. P. 2008. Simulation and the Monte Carlo Method. Wiley Series in Probability and Mathematical Statistics. Google Scholar
Digital Library
- Sanfelice, R. G. and Teel, A. R. 2010. Dynamical properties of hybrid systems simulators. Automatica 46, 2, 239--248. Google Scholar
Digital Library
- Sankaranarayanan, S., Chang, R. M., Jiang, G., and Ivancic, F. 2007. State space exploration using feedback constraint generation and monte-carlo sampling. In Proceedings of ESEC/SIGSOFT FSE. 321--330. Google Scholar
Digital Library
- Seda, A. K. and Hitzler, P. 2008. Generalized distance functions in the theory of computation. Comput. J. 53, 4. Google Scholar
Digital Library
- Smith, R. 1984. Monte Carlo procedures for generating points uniformly distributed over bounded regions. Oper. Res. 38, 3, 1296--1308.Google Scholar
Digital Library
- Smith, R. L. 1996. The hit-and-run sampler: A globally reaching markov chain sampler for generating arbitrary multivariate distributions. In Proceedings of the 28th Conference on Winter Simulation. 260--264. Google Scholar
Digital Library
- Sontag, E. D. 1998. Mathematical Control Theory: Deterministic Finite Dimensional Systems 2nd Ed. Springer. Google Scholar
Digital Library
- Tripakis, S. and Dang, T. 2009. Model-Based Design for Embedded Systems. CRC Press, 383--436. Google Scholar
Digital Library
- Younes, H. L. S. and Simmons, R. G. 2006. Statistical probabilitistic model checking with a focus on time-bounded properties. Inform. Comput. 204, 9, 1368--1409. Google Scholar
Digital Library
- Zabinsky, A., Smith, R., MacDonald, J., Romeijn, H., and Kaufman, D. 1993. Improving hit-and-run for global optimization. J. Global Optim. 3, 171--192.Google Scholar
Cross Ref
- Zhao, Q., Krogh, B. H., and Hubbard, P. 2003. Generating test inputs for embedded control systems. IEEE Control Syst. Mag. Aug., 49--57.Google Scholar
Index Terms
Probabilistic Temporal Logic Falsification of Cyber-Physical Systems
Recommendations
Falsification of temporal properties of hybrid systems using the cross-entropy method
HSCC '12: Proceedings of the 15th ACM international conference on Hybrid Systems: Computation and ControlRandomized testing is a popular approach for checking properties of large embedded system designs. It is well known that a uniform random choice of test inputs is often sub-optimal. Ideally, the choice of inputs has to be guided by choosing the right ...
Monte-carlo techniques for falsification of temporal properties of non-linear hybrid systems
HSCC '10: Proceedings of the 13th ACM international conference on Hybrid systems: computation and controlWe present a Monte-Carlo optimization technique for finding inputs to a system that falsify a given Metric Temporal Logic (MTL) property. Our approach performs a random walk over the space of inputs guided by a robustness metric defined by the MTL ...
Relaxed Decidability and the Robust Semantics of Metric Temporal Logic
HSCC '17: Proceedings of the 20th International Conference on Hybrid Systems: Computation and ControlRelaxed notions of decidability widen the scope of automatic verification of hybrid systems. In quasi-decidability and delta-decidability, the fundamental compromise is that if we are willing to accept a slight error in the algorithm's answer, or a ...






Comments