skip to main content
research-article

PISCOT: A Pipelined Split-Transaction COTS-Coherent Bus for Multi-Core Real-Time Systems

Published:29 October 2022Publication History
Skip Abstract Section

Abstract

Tasks in modern embedded systems such as automotive and avionics communicate among each other using shared data towards achieving the desired functionality of the whole system. In commodity platforms, cores communicate data through the shared memory hierarchy and correctness is maintained by a cache coherence protocol. Recent works investigated the deployment of coherence protocols in real-time systems and showed significant performance improvements. Nonetheless, we find these works to require modifications to commodity coherence protocols, assume simple in-order pipelines, and most importantly suffer from significant latency delays due to coherence interference along with average performance degradation. In this work, we propose PISCOT: a predictable and coherent bus architecture that (i) provides a considerably tighter bound compared to the state-of-the-art predictable coherent solutions (4× tighter bounds in a quad-core system). (ii) It does so with a negligible performance loss compared to conventional high-performance architecture coherence delays (less than 4% for SPLASH-3 benchmarks). This improves average performance by up to 5× (2.8× on average) compared to its predictable coherence counterpart. Finally, (iii) it achieves that without requiring any modifications to conventional coherence protocols. We show this by integrating PISCOT on top of two protocols with a detailed implementation with complete transient states: MSI and MESI.

REFERENCES

  1. [1] Jr. W. L. Bain and Ahuja S. R.. 1981. Performance analysis of high-speed digital buses for multiprocessing systems. In Proceedings of the 8th Annual Symposium on Computer Architecture. 107133.Google ScholarGoogle Scholar
  2. [2] Bansal Ayoosh, Singh Jayati, Hao Yifan, Wen Jen-Yang, Mancuso Renato, and Caccamo Marco. 2019. Cache where you want! Reconciling predictability and coherent caching. arXiv preprint arXiv:1909.05349 (2019).Google ScholarGoogle Scholar
  3. [3] Becker Matthias, Dasari Dakshina, Nicolic Borislav, Akesson Benny, Nélis Vincent, and Nolte Thomas. 2016. Contention-free execution of automotive applications on a clustered many-core platform. In IEEE Euromicro Conference on Real-Time Systems (ECRTS).Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Chisholm M., Kim N., Ward B. C., Otterness N., Anderson J. H., and Smith F. D.. 2016. Reconciling the tension between hardware isolation and data sharing in mixed-criticality, multicore systems. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  5. [5] Fischer Michael A.. 1988. Fair Arbitration Technique for a Split Transaction Bus in a Multiprocessor Computer System. US Patent 4,785,394.Google ScholarGoogle Scholar
  6. [6] semicondutor Freescale. 2016. QorIQ T2080 Reference Manual. Also supports T2081. Document Number: T2080RM. Rev. 3, 11/2016.Google ScholarGoogle Scholar
  7. [7] Gracioli Giovani, Alhammad Ahmed, Mancuso Renato, Fröhlich Antônio Augusto, and Pellizzoni Rodolfo. 2015. A survey on cache management mechanisms for real-time embedded systems. ACM Comput. Surv. (2015).Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Gracioli Giovani and Fröhlich Antônio Augusto. 2015. On the design and evaluation of a real-time operating system for cache-coherent multicore architectures. ACM SIGOPS Oper. Syst. Rev. (2015).Google ScholarGoogle Scholar
  9. [9] Guo Danlu, Hassan Mohamed, Pellizzoni Rodolfo, and Patel Hiren. 2018. A comparative study of predictable dram controllers. ACM Transactions on Embedded Computing Systems (TECS) (2018).Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Hamann Arne, Dasari Dakshina, Kramer Simon, Pressler Michael, and Wurst Falk. 2017. Communication centric design in complex automotive embedded systems. In 29th Euromicro Conference on Real-Time Systems (ECRTS 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google ScholarGoogle Scholar
  11. [11] Hardy D., Piquet T., and Puaut I.. 2009. Using bypass to tighten WCET estimates for multi-core processors with shared instruction caches. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  12. [12] Hassan Mohamed. 2018. On the off-chip memory latency of real-time systems: Is DDR DRAM really the best option?. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  13. [13] Hassan Mohamed. 2020. Discriminative coherence: Balancing performance and latency bounds in data-sharing multi-core real-time systems. In Euromicro Conference on Real-Time Systems (ECRTS). 122.Google ScholarGoogle Scholar
  14. [14] Hassan M., Kaushik A. M., and Patel H.. 2017. Predictable cache coherence for multi-core real-time systems. In IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).Google ScholarGoogle Scholar
  15. [15] Hassan M. and Patel H.. 2016. Criticality- and requirement-aware bus arbitration for multi-core mixed criticality systems. In IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).Google ScholarGoogle Scholar
  16. [16] Hassan Mohamed and Pellizzoni Rodolfo. 2018. Bounding DRAM interference in COTS heterogeneous MPSoCs for mixed criticality systems. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2018).Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Hebbache Farouk, Jan Mathieu, Brandner Florian, and Pautet Laurent. 2018. Shedding the shackles of time-division multiplexing. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  18. [18] Hennessy John L. and Patterson David A.. 2011. Computer Architecture: A Quantitative Approach. Elsevier.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Hessien Salah and Hassan Mohamed. 2020. The best of all worlds: Improving predictability at the performance of conventional coherence with no protocol modifications. In 2020 IEEE Real-Time Systems Symposium (RTSS). IEEE, 218230.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Kaushik Anirudh Mohan, Hassan Mohamed, and Patel Hiren. 2020. Designing predictable cache coherence protocols for multi-core real-time systems. IEEE Trans. Comput. (2020).Google ScholarGoogle Scholar
  21. [21] Kaushik Anirudh M. and Patel Hiren. 2021. A systematic approach to achieving tight worst-case latency and high-performance under predictable cache coherence. In Proceedings of the IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). IEEE, 112.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Kaushik Anirudh M., Tegegn Paulos, Wu Zhuanhao, and Patel Hiren. 2019. CARP: A data communication mechanism for multi-core mixed-criticality systems. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  23. [23] Kelter Timon, Falk Heiko, Marwedel Peter, Chattopadhyay Sudipta, and Roychoudhury Abhik. 2011. Bus-aware multicore WCET analysis through TDMA offset bounds. In Euromicro Conference on Real-Time Systems (ECRTS).Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Khaira Manpreet S.. 1996. Fast First-come First Served Arbitration Method. US Patent 5,574,867.Google ScholarGoogle Scholar
  25. [25] Kim Namhoon, Chisholm Micaiah, Otterness Nathan, Anderson James H., and Smith F. Donelson. 2017. Allowing shared libraries while supporting hardware isolation in multicore real-time systems. In IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).Google ScholarGoogle Scholar
  26. [26] Lesage Benjamin, Hardy Damien, and Puaut Isabelle. 2010. Shared data caches conflicts reduction for WCET computation in multi-core architectures. In International Conference on Real-Time and Network Systems.Google ScholarGoogle Scholar
  27. [27] Mancuso Renato, Dudko Roman, Betti Emiliano, Cesati Marco, Caccamo Marco, and Pellizzoni Rodolfo. 2013. Real-time cache management framework for multi-core architectures. In IEEE 19th Real-Time and Embedded Technology and Applications Symposium (RTAS).Google ScholarGoogle Scholar
  28. [28] Martin Milo M. K., Hill Mark D., and S Daniel J.orin. 2012. Why on-chip cache coherence is here to stay. Communications of ACM (2012).Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Paolieri Marco, Quiñones Eduardo, Cazorla Francisco J., Bernat Guillem, and Valero Mateo. 2009. Hardware support for WCET analysis of hard real-time multicore systems. ACM SIGARCH Computer Architecture News (2009).Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Pellizzoni Rodolfo, Bui Bach D., Caccamo Marco, and Sha Lui. 2008. Coscheduling of CPU and I/O transactions in COTS-based embedded systems. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  31. [31] Poletti Francesco, Bertozzi Davide, Benini Luca, and Bogliolo Alessandro. 2003. Performance analysis of arbitration policies for SoC communication architectures. Design Automation for Embedded Systems (2003).Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Pong Fong and Dubois Michel. 1995. A new approach for the verification of cache coherence protocols. IEEE Transactions on Parallel and Distributed Systems (1995).Google ScholarGoogle Scholar
  33. [33] Pujol Roger, Tabani Hamid, Abella Jaume, Hassan Mohamed, and Cazorla Francisco J.. 2020. Empirical evidence for MPSoCs in critical systems: The case of NXP’s T2080 cache coherence. In IEEE Design Automation and Test in Europe (DATE). 14.Google ScholarGoogle Scholar
  34. [34] Collins) D. Radack et al. (Rockwell. 2018. Civil Certification of Multi-core Processing Systems in Commercial Avionics.Google ScholarGoogle Scholar
  35. [35] Schoeberl Martin, Puffitsch Wolfgang, and Huber Benedikt. 2009. Towards time-predictable data caches for chip-multiprocessors. In Springer International Workshop on Software Technologies for Embedded and Ubiquitous Systems (IFIP).Google ScholarGoogle Scholar
  36. [36] Sensfelder Nathanaël, Brunel Julien, and Pagetti Claire. 2019. Modeling cache coherence to expose interference. In 31st Euromicro Conference on Real-Time Systems (ECRTS 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google ScholarGoogle Scholar
  37. [37] Sensfelder Nathanaël, Brunel Julien, and Pagetti Claire. 2020. On how to identify cache coherence: Case of the NXP QorIQ T4240. In 32nd Euromicro Conference on Real-Time Systems (ECRTS 2020). Schloss Dagstuhl-Leibniz-Zentrum für Informatik.Google ScholarGoogle Scholar
  38. [38] Singhal Ashok, Liencres Bjorn, Price Jeff, Cerauskis Frederick M., Broniarczyk David, Cheung Gerald, Hagersten Erik, and Agarwal Nalini. 1999. Implementing Snooping on a Split-transaction Computer System Bus. US Patent 5,978,874.Google ScholarGoogle Scholar
  39. [39] Sorin Daniel J., Hill Mark D., and Wood David A.. 2011. A primer on memory consistency and cache coherence. Synthesis Lectures on Computer Architecture (2011).Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Sritharan N., Kaushik A. M., Hassan M., and Patel H.. 2017. Hourglass: Predictable time-based cache coherence protocol for dual-critical multi-core systems. (2017).Google ScholarGoogle Scholar
  41. [41] Sritharan Nivedita, Kaushik Anirudh Mohan, Hassan Mohamed, and Patel Hiren. 2019. Enabling predictable, simultaneous and coherent data sharing in mixed criticality systems. (2019).Google ScholarGoogle Scholar
  42. [42] Yoon Man-Ki, Kim Jung-Eun, and Sha Lui. 2011. Optimizing tunable WCET with shared resource allocation and arbitration in hard real-time multicore systems. In IEEE Real-Time Systems Symposium (RTSS).Google ScholarGoogle Scholar
  43. [43] Younis Mohamed and Aboutabl Mohamed. 2002. Communication Handling in Integrated Modular Avionics. US Patent App. 09/821,601.Google ScholarGoogle Scholar
  44. [44] Yun Heechul, Pellizzoni Rodolfo, and Valsan Prathap Kumar. 2015. Parallelism-aware memory interference delay analysis for COTS multicore systems. In Euromicro Conference on Real-Time Systems (ECRTS).Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Ziakas Dimitrios, Baum Allen, Maddox Robert A., and Safranek Robert J.. 2010. Intel® quickpath interconnect architectural features supporting scalable system architectures. In IEEE Symposium on High Performance Interconnects.Google ScholarGoogle Scholar

Index Terms

  1. PISCOT: A Pipelined Split-Transaction COTS-Coherent Bus for Multi-Core Real-Time Systems

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Embedded Computing Systems
            ACM Transactions on Embedded Computing Systems  Volume 22, Issue 1
            January 2023
            512 pages
            ISSN:1539-9087
            EISSN:1558-3465
            DOI:10.1145/3567467
            • Editor:
            • Tulika Mitra
            Issue’s Table of Contents

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 29 October 2022
            • Online AM: 22 August 2022
            • Accepted: 23 July 2022
            • Revised: 9 June 2022
            • Received: 24 December 2021
            Published in tecs Volume 22, Issue 1

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Refereed
          • Article Metrics

            • Downloads (Last 12 months)100
            • Downloads (Last 6 weeks)8

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Full Text

          View this article in Full Text.

          View Full Text

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!