Abstract
Many problems in the design and analysis of cyber-physical systems (CPS) reduce to the following optimization problem: given a CPS which transforms continuous-time input traces in Rm to continuous-time output traces in Rn and a cost function over output traces, find an input trace which minimizes the cost. Cyber-physical systems are typically so complex that solving the optimization problem analytically by examining the system dynamics is not feasible. We consider a black-box approach, where the optimization is performed by testing the input-output behaviour of the CPS.
We provide a unified, tool-supported methodology for CPS testing and optimization. Our tool is the first CPS testing tool that supports Bayesian optimization. It is also the first to employ fully automated dimensionality reduction techniques. We demonstrate the potential of our tool by running experiments on multiple industrial case studies. We compare the effectiveness of Bayesian optimization to state-of-the-art testing techniques based on CMA-ES and Simulated Annealing.
- H. Abbas and G. Fainekos. 2012. Convergence proofs for Simulated Annealing falsification of safety properties. In Communication, Control, and Computing (Allerton), 2012 50th Annual Allerton Conference on. IEEE, 1594--1601.Google Scholar
- T. Akazaki. 2016. Falsification of Conditional Safety Properties for Cyber-Physical Systems with Gaussian Process Regression. 439--446.Google Scholar
- R. Alur. 2015. Principles of Cyber-Physical Systems. The MIT Press. Google Scholar
Digital Library
- R. Alur, T. Feder, and T. A. Henzinger. 1996. The benefits of relaxing punctuality. J. ACM 43, 1 (1996), 116--146. Google Scholar
Digital Library
- Y. Annpureddy, C. Liu, G. E. Fainekos, and S. Sankaranarayanan. 2011. S-TaLiRo: A tool for temporal logic falsification for hybrid systems. In TACAS 11 (Lecture Notes in Computer Science), Vol. 6605. Springer, 254--257. Google Scholar
Digital Library
- S. Bansal, R. Calandra, T. Xiao, S. Levine, and C. Tomlin. 2017. Goal-driven dynamics learning via Bayesian optimization. CoRR abs/1703.09260 (2017).Google Scholar
- M. Branicky. 1995. Studies in hybrid systems: modeling, analysis, and control. Ph.D. thesis, Massachusetts Institute of Technology. Google Scholar
Digital Library
- E. Brochu, V. M. Cora, and N. de Freitas. 2010. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. CoRR abs/1012.2599 (2010).Google Scholar
- A. D. Bull. 2011. Convergence rates of efficient global optimization algorithms. J. Mach. Learn. Res. 12 (Nov. 2011), 2879--2904. Google Scholar
Digital Library
- J. Deshmukh, X. Jin, J. Kapinski, and O. Maler. 2015. Stochastic local search for falsification of hybrid systems. In ATVA. Springer, 500--517.Google Scholar
- A. Donzé. 2010. Breach, A Toolbox for Verification and Parameter Synthesis of Hybrid Systems. Springer, 167--170. Google Scholar
Digital Library
- A. Donzé and O. Maler. 2010. Robust Satisfaction of Temporal Logic over Real-Valued Signals. Springer, 92--106. Google Scholar
Digital Library
- T. Dreossi, T. Dang, A. Donzé, J. Kapinski, X. Jin, and J. V. Deshmukh. 2015. Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems. Springer International Publishing, 127--142.Google Scholar
- B. Fabien. 1998. Some tools for the direct solution of optimal control problems. Advances in Engineering Software 29, 1 (1998), 45--61. Google Scholar
Digital Library
- G. Fainekos. 2015. Automotive control design bug-finding with the S-TaLiRo tool. In ACC 2015. 4096.Google Scholar
Cross Ref
- S. Grünewälder, J.-Y. Audibert, M. Opper, and J. Shawe-Taylor. 2010. Regret Bounds for Gaussian Process Bandit Problems. In AISTATS 2010. 273--280.Google Scholar
- N. Hansen. 2016. The CMA Evolution Strategy: A tutorial. CoRR abs/1604.00772 (2016).Google Scholar
- M. Huang, K. Zaseck, K. Butts, and I. Kolmanovsky. 2016. Rate-based model predictive controller for diesel engine air path: Design and experimental evaluation. IEEE Trans. on Control Systems Technology 99 (2016), 1--14.Google Scholar
- X. Jin, J. V. Deshmukh, J. Kapinski, K. Ueda, and K. Butts. 2014. Powertrain control verification benchmark. In HSCC’14. ACM, 253--262. Google Scholar
Digital Library
- W. B. Johnson and J. Lindenstrauss. 1984. Extensions of Lipschitz mappings into a Hilbert space. Contemporary Mathematics 26 (1984), 189--206.Google Scholar
Cross Ref
- D. R. Jones, C. D. Perttunen, and B. E. Stuckman. 1993. Lipschitzian optimization without the Lipschitz constant. Journal of Optimization Theory and Applications 79, 1 (1993), 157--181.Google Scholar
Digital Library
- S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 1983. Optimization by simulated annealing. Science 220, 4598 (1983), 671--680.Google Scholar
- D. Lizotte, T. Wang, M. Bowling, and D. Schuurmans. 2007. Automatic gait optimization with Gaussian process regression. In IJCAI 07. 944--949. Google Scholar
Digital Library
- N. Mahendran, Z. Wang, F. Hamze, and N. D. Freitas. 2012. Adaptive MCMC with Bayesian optimization. In AISTATS-12), N. D. Lawrence and M. A. Girolami (Eds.), Vol. 22. 751--760.Google Scholar
- O. Maler and D. Nickovic. 2013. Monitoring properties of analog and mixed-signal circuits. STTT 15, 3 (2013), 247--268.Google Scholar
Digital Library
- A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe. 2016. Automatic LQR tuning based on Gaussian process global optimization. In ICRA 16. 270--277.Google Scholar
- Mathworks. 2017. Simulink—Simulation and model-based design. https://www.mathworks.com/products/simulink.html.Google Scholar
- W. Messner and D. Tilbury. 2017. Control tutorials for MATLAB and Simulink. http://ctms.engin.umich.edu/CTMS/index.php?aux=Basics_Simulink% #27.Google Scholar
- M. J. D. Powell. 2009. The BOBYQA algorithm for bound constrained optimization without derivatives. Technical Report DAMTP2009/NA06. University of Cambridge.Google Scholar
- C. E. Rasmussen and C. K. I. Williams. 2006. Gaussian processes for machine learning. MIT Press. Google Scholar
Digital Library
- S. Sankaranarayanan and G. Fainekos. 2012. Falsification of temporal properties of hybrid systems using the cross-entropy method. In HSCC 12. ACM, 125--134. Google Scholar
Digital Library
- P. Tabuada. 2009. Verification and Control of Hybrid Systems - A Symbolic Approach. Springer. Google Scholar
Digital Library
- Z. Wang, F. Hutter, M. Zoghi, D. Matheson, and N. De Freitas. 2016. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research 55, 1 (2016), 361--387. Google Scholar
Cross Ref
- Y. Xue and P. Bogdan. 2017. Constructing compact causal mathematical models for complex dynamics. In ICCPS’17. ACM, 97--107. Google Scholar
Digital Library
Index Terms
Testing Cyber-Physical Systems through Bayesian Optimization
Recommendations
A Survey on Testing for Cyber Physical System
ICTSS 2015: Proceedings of the 27th IFIP WG 6.1 International Conference on Testing Software and Systems - Volume 9447Cyber Physical Systems CPS bridge the cyber-world of computing and communications with the physical world and require development of secure and reliable software. It asserts a big challenge not only on testing and verifying the correctness of all ...
TREGO: a trust-region framework for efficient global optimization
AbstractEfficient global optimization (EGO) is the canonical form of Bayesian optimization that has been successfully applied to solve global optimization of expensive-to-evaluate black-box problems. However, EGO struggles to scale with dimension, and ...
Black-box optimization benchmarking of IPOP-saACM-ES and BIPOP-saACM-ES on the BBOB-2012 noiseless testbed
GECCO '12: Proceedings of the 14th annual conference companion on Genetic and evolutionary computationIn this paper, we study the performance of IPOP-saACM-ES and BIPOP-saACM-ES, recently proposed self-adaptive surrogate-assisted Covariance Matrix Adaptation Evolution Strategies. Both algorithms were tested using restarts till a total number of function ...






Comments