skip to main content
research-article

Dealing with Uncertainty in pWCET Estimations

Published:26 September 2020Publication History
Skip Abstract Section

Abstract

The problem of estimating a tight and safe Worst-Case Execution Time (WCET), needed for certification in safety-critical environment, is a challenging problem for modern embedded systems. A possible solution proposed in past years is to exploit statistical tools to obtain a probability distribution of the WCET. These probabilistic real-time analyses for WCET are, however, subject to errors, even when all the applicability hypotheses are satisfied and verified. This is caused by the uncertainties of the probabilistic-WCET distribution estimator. This article aims at improving the measurement-based probabilistic timing analysis approach providing some techniques to analyze and deal with such uncertainties. The so-called region of acceptance model based on state-of-the-art statistical test procedures is defined over the distribution space parameters. From this model, a set of strategies is derived and discussed to provide the methodology to deal with the trade-off safety/tightness of the WCET estimation. These techniques are then tested over real datasets, including industrial safety-critical applications, to show the increased value of using the proposed approach in probabilistic WCET analyses.

References

  1. J. Abella, C. Hernandez, E. Quiñones, F. J. Cazorla, P. R. Conmy, M. Azkarate-askasua, J. Perez, E. Mezzetti, and T. Vardanega. 2015. WCET analysis methods: Pitfalls and challenges on their trustworthiness. In Proceedings of the 10th IEEE International Symposium on Industrial Embedded Systems (SIES’15). IEEE, 1--10. DOI:https://doi.org/10.1109/SIES.2015.7185039Google ScholarGoogle ScholarCross RefCross Ref
  2. J. Abella, E. Quiñones, F. Wartel, T. Vardanega, and F. J. Cazorla. 2014. Heart of gold: Making the improbable happen to increase confidence in MBPTA. In Proceedings of the 26th Euromicro Conference on Real-time Systems. IEEE, 255--265. DOI:https://doi.org/10.1109/ECRTS.2014.33Google ScholarGoogle Scholar
  3. Kostiantyn Berezovskyi, Fabrice Guet, Luca Santinelli, Konstantinos Bletsas, and Eduardo Tovar. 2016. Measurement-based probabilistic timing analysis for graphics processor units. In Proceedings of the Architecture of Computing Systems (ARCS’16). Springer International Publishing, 223--236.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. G. Bernat, A. Colin, and S. M. Petters. 2002. WCET analysis of probabilistic hard real-time systems. In Proceedings of the 23rd IEEE Real-time Systems Symposium (RTSS’02). IEEE, 279--288. DOI:https://doi.org/10.1109/REAL.2002.1181582Google ScholarGoogle Scholar
  5. Francisco J. Cazorla, Leonidas Kosmidis, Enrico Mezzetti, Carles Hernandez, Jaume Abella, and Tullio Vardanega. 2019. Probabilistic worst-case timing analysis: Taxonomy and comprehensive survey. ACM Comput. Surv. 52, 1 (Feb. 2019). DOI:https://doi.org/10.1145/3301283Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Francisco J. Cazorla, Eduardo Quiñones, Tullio Vardanega, Liliana Cucu, Benoit Triquet, Guillem Bernat, Emery Berger, Jaume Abella, Franck Wartel, Michael Houston, Luca Santinelli, Leonidas Kosmidis, Code Lo, and Dorin Maxim. 2013. PROARTIS: Probabilistically analyzable real-time systems. ACM Trans. Embed. Comput. Syst. 12, 2s (May 2013). DOI:https://doi.org/10.1145/2465787.2465796Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Xavier Civit, Joan del Castillo, and Jaume Abella. 2018. A reliable statistical analysis of the best-fit distribution for high execution times. In Proceedings of the 21st Euromicro Conference on Digital System Design (DSD’18). IEEE, 727--734. DOI:https://doi.org/10.1109/DSD.2018.00012Google ScholarGoogle ScholarCross RefCross Ref
  8. Harald Cramér. 1928. On the composition of elementary errors. Scand. Actuar. J. 1928, 1 (1928), 13--74. DOI:https://doi.org/10.1080/03461238.1928.10416862Google ScholarGoogle ScholarCross RefCross Ref
  9. Sandor Csorgo and Julian J. Faraway. 1996. The exact and asymptotic distributions of Cramer-von Mises statistics. J. Roy. Stat. Soc. Series B (Methodol.) 58 (01 1996), 221--234. DOI:https://doi.org/10.2307/2346175Google ScholarGoogle Scholar
  10. L. Cucu-Grosjean, L. Santinelli, M. Houston, C. Lo, T. Vardanega, L. Kosmidis, J. Abella, E. Mezzetti, E. Quiñones, and F. J. Cazorla. 2012. Measurement-based probabilistic timing analysis for multi-path programs. In Proceedings of the 24th Euromicro Conference on Real-time Systems. IEEE, 91--101. DOI:https://doi.org/10.1109/ECRTS.2012.31Google ScholarGoogle Scholar
  11. Corentin Damman, Gregory Edison, Fabrice Guet, Eric Noulard, Luca Santinelli, and Jerome Hugues. 2016. Architectural performance analysis of FPGA synthesized LEON processors. In Proceedings of the 27th International Symposium on Rapid System Prototyping: Shortening the Path from Specification to Prototype. ACM, New York, NY, 33--40. DOI:https://doi.org/10.1145/2990299.2990306Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. D. Dasari, B. Akesson, V. Nélis, M. A. Awan, and S. M. Petters. 2013. Identifying the sources of unpredictability in COTS-based multicore. In Proceedings of the International Symposium on Industrial Embedded Systems. IEEE, 39--48. DOI:https://doi.org/10.1109/SIES.2013.6601469Google ScholarGoogle Scholar
  13. Robert Davis and Liliana Cucu-Grosjean. 2019. A survey of probabilistic schedulability analysis techniques for real-time systems. Leibniz Trans. Embed. Syst. 6, 1 (2019), 04–1–04:53. DOI:https://doi.org/10.4230/LITES-v006-i001-a004Google ScholarGoogle Scholar
  14. Robert I. Davis, Alan Burns, and David Griffin. 2017. On the Meaning of pWCET Distributions and Their Use in Schedulability Analysis. Retrieved from https://www-users.cs.york.ac.uk/ robdavis/papers/RTSOPS2017pWCET.pdf.Google ScholarGoogle Scholar
  15. Robert I. Davis, Luca Santinelli, Sebastian Altmeyer, Claire Maiza, and Liliana Cucu-Grosjean. 2013. Analysis of probabilistic cache related pre-emption delays. In Proceedings of the 25th Euromicro Conference on Real-time Systems (ECRTS’13). IEEE, 168--179. DOI:https://doi.org/10.1109/ECRTS.2013.27Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Darinka Dentcheva and Andrzej Ruszczyński. 2003. Portfolio Optimization with Stochastic Dominance Constraints. Elsevier. DOI:https://doi.org/10.18452/8306Google ScholarGoogle Scholar
  17. Julien Durand, Youcef Bouchebaba, and Luca Santinelli. 2019. Statistical analysis for shared resources effects with multi-core real-time systems. In Proceedings of the 13th IEEE International Symposium on Embedded Multicore/Many-core Systems-on-chip (MCSoC’19). IEEE, 362--371. DOI:https://doi.org/10.1109/MCSoC.2019.00058Google ScholarGoogle ScholarCross RefCross Ref
  18. S. Edgar and A. Burns. 2001. Statistical analysis of WCET for scheduling. In Proceedings of the 22nd IEEE Real-time Systems Symposium (RTSS’01). IEEE, 215--224. DOI:https://doi.org/10.1109/REAL.2001.990614Google ScholarGoogle Scholar
  19. M. Fernandez, D. Morales, L. Kosmidis, A. Bardizbanyan, I. Broster, C. Hernandez, E. Quinones, J. Abella, F. Cazorla, P. Machado, and L. Fossati. 2017. Probabilistic timing analysis on time-randomized platforms for the space domain. In Proceedings of the Conference on Design, Automation 8 Test in Europe. ACM and IEEE, 738--739.Google ScholarGoogle Scholar
  20. R. A. Fisher and L. H. C. Tippett. 1928. Limiting forms of the frequency distribution of the largest or smallest member of a sample. Math. Proc. Cambr. Philos. Soc. 24, 2 (1928), 180--190. DOI:https://doi.org/10.1017/S0305004100015681Google ScholarGoogle ScholarCross RefCross Ref
  21. William Fornaciari, Giovanni Agosta, David Atienza, Carlo Brandolese, Leila Cammoun, Luca Cremona, Alessandro Cilardo, Albert Farres, José Flich, Carles Hernandez, Michal Kulchewski, Simone Libutti, José Maria Martínez, Giuseppe Massari, Ariel Oleksiak, Anna Pupykina, Federico Reghenzani, Rafael Tornero, Michele Zanella, Marina Zapater, and Davide Zoni. 2018. Reliable power and time-constraints-aware predictive management of heterogeneous exascale systems. In Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS’18). Association for Computing Machinery, New York, NY, 187--194. DOI:https://doi.org/10.1145/3229631.3239368Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S. Jiménez Gil, I. Bate, G. Lima, L. Santinelli, A. Gogonel, and L. Cucu-Grosjean. 2017. Open challenges for probabilistic measurement-based worst-case execution time. IEEE Embed. Syst. Lett. 9, 3 (Sept. 2017), 69--72. DOI:https://doi.org/10.1109/LES.2017.2712858Google ScholarGoogle Scholar
  23. B. Gnedenko. 1943. Sur La Distribution Limite Du Terme Maximum D’Une Serie Aleatoire. Ann. Math. 44, 3 (1943), 423--453. Retrieved from http://www.jstor.org/stable/1968974.Google ScholarGoogle ScholarCross RefCross Ref
  24. Fabrice Guet, Luca Santinelli, and Jerome Morio. 2017. On the representativity of execution time measurements: Studying dependence and multi-mode tasks. In Proceedings of the 17th International Workshop on Worst-case Execution Time Analysis (WCET’17). (OpenAccess Series in Informatics (OASIcs)), Jan Reineke (Ed.), Vol. 57. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 3:1–3:13. DOI:https://doi.org/10.4230/OASIcs.WCET.2017.3Google ScholarGoogle Scholar
  25. Jeffery Hansen, Scott Hissam, and Gabriel A. Moreno. 2009. Statistical-based WCET estimation and validation. In Proceedings of the 9th International Workshop on Worst-case Execution Time Analysis (WCET’09). (OpenAccess Series in Informatics (OASIcs)), Niklas Holsti (Ed.), Vol. 10. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 1--11. DOI:https://doi.org/10.4230/OASIcs.WCET.2009.2291Google ScholarGoogle Scholar
  26. Jun-Haeng Heo, Hongjoon Shin, Woosung Nam, Juseong Om, and Changsam Jeong. 2013. Approximation of modified Anderson–Darling test statistics for extreme value distributions with unknown shape parameter. J. Hydrol. 499 (2013), 41--49.Google ScholarGoogle ScholarCross RefCross Ref
  27. R. Kirner and P. Puschner. 2008. Obstacles in worst-case execution time analysis. In Proceedings of the 11th IEEE International Symposium on Object and Component-oriented Real-time Distributed Computing (ISORC’08). IEEE, 333--339. DOI:https://doi.org/10.1109/ISORC.2008.65Google ScholarGoogle Scholar
  28. A. Kolmogorov. 1933. Sulla determinazione empírica di uma legge di distribuzione. Inst. Ital. Attuari, Giorn.4 (1933), 83--91.Google ScholarGoogle Scholar
  29. L. Kosmidis, E. Quińones, J. Abella, T. Vardanega, I. Broster, and F. J. Cazorla. 2014. Measurement-based probabilistic timing analysis and its impact on processor architecture. In Proceedings of the 17th Euromicro Conference on Digital System Design. IEEE, 401--410. DOI:https://doi.org/10.1109/DSD.2014.50Google ScholarGoogle Scholar
  30. G. Lima, D. Dias, and E. Barros. 2016. Extreme value theory for estimating task execution time bounds: A careful look. In Proceedings of the Euromicro Conference on Real-time Systems (ECRTS’16). IEEE, 200--211. DOI:https://doi.org/10.1109/ECRTS.2016.20Google ScholarGoogle Scholar
  31. Frank J. Massey Jr. 1951. The Kolmogorov-Smirnov test for goodness of fit. J. Amer. Stat. Assoc. 46, 253 (1951), 68--78.Google ScholarGoogle ScholarCross RefCross Ref
  32. Daniel McFadden. 1978. Modeling the choice of residential location. Transport. Res. Rec. 1, 673 (1978), 72--77.Google ScholarGoogle Scholar
  33. Thomas Nolte, Meng Liu, and Bjorn Lisper. 2014. Challenges with probabilities in response-time analysis of real-time systems. In Proceedings of the 5th Real-time Scheduling Open Problems Seminar (RTSOPS’14). 3--4.Google ScholarGoogle Scholar
  34. Edgar Elias Osuna. 2013. Tail-restricted stochastic dominance. IMA J. Manag. Math. 24, 1 (2013), 21--44. DOI:https://doi.org/10.1093/imaman/dpr023Google ScholarGoogle ScholarCross RefCross Ref
  35. Stefan M. Petters. 2003. Comparison of trace generation methods for measurement based WCET analysis. In Proceedings of the 3rd International Workshop on Worst-case Execution Time Analysis.Google ScholarGoogle Scholar
  36. James P. Quirk and Rubin Saposnik. 1962. Admissibility and measurable utility functions. Rev. Econ. Studies 29, 2 (1962), 140--146. DOI:https://doi.org/10.2307/2295819Google ScholarGoogle ScholarCross RefCross Ref
  37. F. Reghenzani, G. Massari, and W. Fornaciari. 2018. chronovise: Measurement-based probabilistic timing analysis framework. J. Open Source Softw. 3, 28 (2018), 711. DOI:https://doi.org/10.21105/joss.00711Google ScholarGoogle ScholarCross RefCross Ref
  38. F. Reghenzani, G. Massari, and W. Fornaciari. 2018. The misconception of exponential tail upper-bounding in probabilistic real-time. IEEE Embed. Syst. Lett. 11, 3 (2018), 77--80. DOI:https://doi.org/10.1109/LES.2018.2889114Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. F. Reghenzani, G. Massari, and W. Fornaciari. 2019. A probabilistic approach to energy-constrained mixed-criticality systems. In Proceedings of the IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED’19). IEEE/ACM, 1--6. DOI:https://doi.org/10.1109/ISLPED.2019.8824991Google ScholarGoogle Scholar
  40. F. Reghenzani, G. Massari, W. Fornaciari, and A. Galimberti. 2019. Probabilistic-WCET reliability: On the experimental validation of EVT hypotheses. In Proceedings of the International Conference on Omni-layer Intelligent Systems (COINS’19). ACM, New York, NY, 229--234. DOI:https://doi.org/10.1145/3312614.3312660Google ScholarGoogle Scholar
  41. F. Reghenzani, G. Massari, L. Santinelli, and W. Fornaciari. 2019. Statistical power estimation dataset for external validation GoF tests on EVT distribution. Data in Brief 25 (June 2019), 104071. DOI:https://doi.org/10.1016/j.dib.2019.104071Google ScholarGoogle Scholar
  42. Federico Reghenzani, Luca Santinelli, and William Fornaciari. 2019. Why statistical power matters for probabilistic real-time: Work-in-progress. In Proceedings of the International Conference on Embedded Software Companion (EMSOFT’19). Association for Computing Machinery, New York, NY. DOI:https://doi.org/10.1145/3349568.3351555Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Michael Thomas Rolf-Dieter Reiss. 2007. Statistical Analysis of Extreme Values with Applications to Insurance, Finance, Hydrology and Other Fields. Birkhäuser Basel. DOI:https://doi.org/10.1007/978-3-7643-7399-3Google ScholarGoogle Scholar
  44. L. Santinelli, F. Guet, and J. Morio. 2017. Revising measurement-based probabilistic timing analysis. In Proceedings of the IEEE Real-time and Embedded Technology and Applications Symposium (RTAS’17). IEEE, 199--208. DOI:https://doi.org/10.1109/RTAS.2017.16Google ScholarGoogle Scholar
  45. Luca Santinelli and Zhishan Guo. 2017. On the criticality of probabilistic worst-case execution time models. In Dependable Software Engineering. Theories, Tools, and Applications, Kim Guldstrand Larsen, Oleg Sokolsky, and Ji Wang (Eds.). Springer, 59--74.Google ScholarGoogle Scholar
  46. K. P. Silva, L. F. Arcaro, and R. S. d. Oliveira. 2017. On using GEV or Gumbel models when applying EVT for probabilistic WCET estimation. In Proceedings of the IEEE Real-time Systems Symposium (RTSS’17). IEEE, 220--230. DOI:https://doi.org/10.1109/RTSS.2017.00028Google ScholarGoogle ScholarCross RefCross Ref
  47. Ganghuai Wang, J. H. Lambert, and Y. Y. Haimes. 1999. Stochastic ordering of extreme value distributions. IEEE Trans. Syst. Man Cyber. Part A: Syst. Hum. 29, 6 (Nov. 1999), 696--701. DOI:https://doi.org/10.1109/3468.798077Google ScholarGoogle Scholar
  48. F. Wartel, L. Kosmidis, C. Lo, B. Triquet, E. Quiñones, J. Abella, A. Gogonel, A. Baldovin, E. Mezzetti, L. Cucu, T. Vardanega, and F. J. Cazorla. 2013. Measurement-based probabilistic timing analysis: Lessons from an integrated-modular avionics case study. In Proceedings of the 8th IEEE International Symposium on Industrial Embedded Systems. IEEE, 241--248. DOI:https://doi.org/10.1109/SIES.2013.6601497Google ScholarGoogle Scholar

Index Terms

  1. Dealing with Uncertainty in pWCET Estimations

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Embedded Computing Systems
            ACM Transactions on Embedded Computing Systems  Volume 19, Issue 5
            Special Issue on LCETES, Part 1, Real-Time, Critical Systems, and Approximation
            September 2020
            229 pages
            ISSN:1539-9087
            EISSN:1558-3465
            DOI:10.1145/3426818
            Issue’s Table of Contents

            Copyright © 2020 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 26 September 2020
            • Online AM: 7 May 2020
            • Revised: 1 April 2020
            • Accepted: 1 April 2020
            • Received: 1 October 2019
            Published in tecs Volume 19, Issue 5

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!