skip to main content
research-article

Lightweight Robust Size Aware Cache Management

Published:24 August 2022Publication History
Skip Abstract Section

Abstract

Modern key-value stores, object stores, Internet proxy caches, and Content Delivery Networks (CDN) often manage objects of diverse sizes, e.g., blobs, video files of different lengths, images with varying resolutions, and small documents. In such workloads, size-aware cache policies outperform size-oblivious algorithms. Unfortunately, existing size-aware algorithms tend to be overly complicated and computationally expensive.

Our work follows a more approachable pattern; we extend the prevalent (size-oblivious) TinyLFU cache admission policy to handle variable-sized items. Implementing our approach inside two popular caching libraries only requires minor changes. We show that our algorithms yield competitive or better hit-ratios and byte hit-ratios compared to the state-of-the-art size-aware algorithms such as AdaptSize, LHD, LRB, and GDSF. Further, a runtime comparison indicates that our implementation is faster by up to 3× compared to the best alternative, i.e., it imposes a much lower CPU overhead.

REFERENCES

  1. [1] Abrams Marc, Standridge Charles R., Abdulla Ghaleb, Williams Stephen, and Fox Edward A.. 1995. Caching Proxies: Limitations and Potentials. Technical Report. USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Adas D., Einziger G., and Friedman R.. 2017. TinyCache - an effective cache admission filter. In GLOBECOM 2017-2017 IEEE Global Communications Conference. 16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Akhtar Zahaib, Li Yaguang, Govindan Ramesh, Halepovic Emir, Hao Shuai, Liu Yan, and Sen Subhabrata. 2019. AViC: A cache for adaptive bitrate video. In Proc. of the 15th International ACM Conference on Emerging Networking Experiments And Technologies (CoNEXT). 305317.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Arlitt Martin, Friedrich Rich, and Jin Tai. 2000. Performance evaluation of web proxy cache replacement policies. Perform. Eval. 39, 1–4 (Feb. 2000), 149164.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Bansal Sorav and Modha Dharmendra S.. 2004. CAR: Clock with adaptive replacement. In Proc. of the 3rd USENIX Conf. on File and Storage Technologies (FAST). 187200.Google ScholarGoogle Scholar
  6. [6] Beckmann Nathan, Chen Haoxian, and Cidon Asaf. 2018. LHD: Improving cache hit rate by maximizing hit density. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 389403.Google ScholarGoogle Scholar
  7. [7] Beckmann Nathan, Chen Haoxian, and Cidon Asaf. 2018. Simulation Code for the LHD Cache Replacement Policy. (2018). https://github.com/CMU-CORGI/LHD.Google ScholarGoogle Scholar
  8. [8] Belady L. A.. 1966. A study of replacement algorithms for a virtual-storage computer. IBM Systems Journal 5, 2 (1966), 78101.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Berg Benjamin, Berger Daniel S., McAllister Sara, Grosof Isaac, Gunasekar Sathya, Lu Jimmy, Uhlar Michael, Carrig Jim, Beckmann Nathan, Harchol-Balter Mor, and Ganger Gregory R.. 2020. The CacheLib caching engine: Design and experiences at scale. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI). 753768.Google ScholarGoogle Scholar
  10. [10] Berger Daniel S., Beckmann Nathan, and Harchol-Balter Mor. 2018. Practical bounds on optimal caching with variable object sizes. Proceedings of the ACM on Measurement and Analysis of Computing Systems 2, 2 (2018), 138.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Berger Daniel S., Sitaraman Ramesh K., and Harchol-Balter Mor. 2017. AdaptSize: Orchestrating the hot object memory cache in a content delivery network. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 483498.Google ScholarGoogle Scholar
  12. [12] Berger Daniel S., Sitaraman Ramesh K., and Harchol-Balter Mor. 2017. Webcachesim, a C++ Cache Simulator. (2017). https://github.com/dasebe/webcachesim.Google ScholarGoogle Scholar
  13. [13] Blankstein Aaron, Sen Siddhartha, and Freedman Michael J.. 2017. Hyperbolic caching: Flexible caching for web applications. In 2017 USENIX Annual Technical Conference (USENIX ATC). 499511.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Cherkasova Ludmila. 1998. Improving WWW Proxies Performance with Greedy-Dual-Size-Frequency Caching Policy. Technical Report. In HP Tech. Report.Google ScholarGoogle Scholar
  15. [15] Chockler Gregory, Laden Guy, and Vigfusson Ymir. 2010. Data caching as a cloud service. In Proceedings of the 4th ACM International Workshop on Large Scale Distributed Systems and Middleware (LADIS’10). ACM, 1821.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Chockler Gregory, Laden Guy, and Vigfusson Ymir. 2011. Design and implementation of caching services in the cloud. IBM Journal of Research and Development 55, 6 (2011), 9:1–9:11.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Cohen Saar and Matias Yossi. 2003. Spectral bloom filters. In Proc. of the ACM SIGMOD Int. Conf. on Management of Data. 241252.Google ScholarGoogle Scholar
  18. [18] Cormode Graham and Muthukrishnan S.. 2005. An improved data stream summary: The count-min sketch and its applications. J. Algorithms 55, 1 (April 2005), 5875.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Dgraph. 2020. Ristretto: A High Performance Memory-Bound Go Cache. (2020). https://github.com/dgraph-io/ristretto.Google ScholarGoogle Scholar
  20. [20] Einziger Gil, Eytan Ohad, Friedman Roy, and Manes Ben. 2018. Adaptive software cache management. In Proceedings of the 19th International Middleware Conference. 94106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Einziger G., Friedman R., and Manes B.. 2017. TinyLFU: A highly efficient cache admission policy. ACM Transactions on Storage (TOS’17).Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Fitzpatrick Brad. 2004. Distributed caching with Memcached. Linux J. 2004, 124 (Aug. 2004).Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Gold Haddas. 2020. Netflix and YouTube are slowing down in Europe to keep the internet from breaking. CNN Business (2020). https://www.cnn.com/2020/03/19/tech/netflix-internet-overload-eu/index.htmlGoogle ScholarGoogle Scholar
  24. [24] Hasan S., Gorinsky S., Dovrolis C., and Sitaraman R. K.. 2014. Trade-offs in optimizing the cache deployments of CDNs. In IEEE Conference on Computer Communications (INFOCOM). 460468.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Hennessy John L. and Patterson David A.. 2012. Computer Architecture - A Quantitative Approach (5. ed.). Morgan Kaufmann.Google ScholarGoogle Scholar
  26. [26] Jiang Song, Chen Feng, and Zhang Xiaodong. 2005. CLOCK-Pro: An effective improvement of the CLOCK replacement. In Proc. of the USENIX Annual Technical Conference (ATC).Google ScholarGoogle Scholar
  27. [27] Jiang Song and Zhang Xiaodong. 2002. LIRS: An efficient low inter-reference recency set replacement policy to improve buffer cache performance. In Proc. of the International Conference on Measurements and Modeling of Computer Systems SIGMETRICS. 3142.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Johnson Theodore and Shasha Dennis. 1994. 2Q: A low overhead high performance buffer management replacement algorithm. In Proc. of the 20th Int. Conf. on Very Large Data Bases (VLDB). 439450.Google ScholarGoogle Scholar
  29. [29] Karakostas G. and Serpanos D. N.. 2002. Exploitation of different types of locality for web caches. In Proc. of the 7th Int. Symposium on Computers and Communications (ISCC).Google ScholarGoogle Scholar
  30. [30] Shah Anirban Ketan and Matani Mitra Dhruv. 2010. An O(1) Algorithm for Implementing the LFU Cache Eviction Scheme. Technical Report. http://dhruvbird.com/lfu.pdf.Google ScholarGoogle Scholar
  31. [31] Lee Chunghan, Kumano Tatsuo, Matsuki Tatsuma, Endo Hiroshi, Fukumoto Naoto, and Sugawara Mariko. 2017. Understanding storage traffic characteristics on enterprise virtual desktop infrastructure. In Proceedings of the 10th ACM International Systems and Storage Conference. 111.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Lee Donghee, Choi Jongmoo, Kim Jong-Hun, Noh Sam H., Min Sang Lyul, Cho Yookun, and Kim Chong-Sang. 2001. LRFU: A spectrum of policies that subsumes the least recently used and least frequently used policies. IEEE Trans. Computers 50, 12 (2001), 13521361.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Li Cong. 2018. DLIRS: Improving low inter-reference recency set cache replacement policy with dynamics. In Proc. of the 11th ACM International Systems and Storage Conference (SYSTOR).Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Manes Ben. 2016. Caffeine: A high performance caching library for Java 8. https://github.com/ben-manes/caffeine.Google ScholarGoogle Scholar
  35. [35] Megiddo Nimrod and Modha Dharmendra S.. 2003. ARC: A self-tuning, low overhead replacement cache. In Proc. of the 2nd USENIX Conf. on File and Storage Technologies (FAST). 115130.Google ScholarGoogle Scholar
  36. [36] Mertz Jhonny and Nunes Ingrid. 2017. Understanding application-level caching in web applications: A comprehensive introduction and survey of state-of-the-art approaches. ACM Comput. Surv. 50, 6 (Nov. 2017).Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Narayanan Dushyanth, Donnelly Austin, and Rowstron Antony. 2008. Write off-loading: Practical power management for enterprise storage. ACM Transactions on Storage (TOS) 4, 3 (2008), 123.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Neglia Giovanni, Carra Damiano, Feng Mingdong, Janardhan Vaishnav, Michiardi Pietro, and Tsigkari Dimitra. 2017. Access-time-aware cache algorithms. ACM Trans. Model. Perform. Eval. Comput. Syst. 2, 4 (Nov. 2017).Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] O’Neil Elizabeth J., O’Neil Patrick E., and Weikum Gerhard. 1993. The LRU-K page replacement algorithm for database disk buffering. ACM SIGMOD Rec. 22, 2 (June 1993), 297306.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Park Sejin and Park Chanik. 2017. FRD: A filtering based buffer cache algorithm that considers both frequency and reuse distance. In Proc. of the 33rd IEEE International Conference on Massive Storage Systems and Technology (MSST).Google ScholarGoogle Scholar
  41. [41] Redis-Labs. 2020. Using Redis as an LRU Cache. (2020). https://redis.io/topics/lru-cache.Google ScholarGoogle Scholar
  42. [42] Rodriguez Liana V., Yusuf Farzana, Lyons Steven, Paz Eysler, Rangaswami Raju, Liu Jason, Zhao Ming, and Narasimhan Giri. 2021. Learning cache replacement with CACHEUS. In 19th USENIX Conference on File and Storage Technologies (FAST).Google ScholarGoogle Scholar
  43. [43] Song Zhenyu. 2020. A Simulator for CDN Caching and Web Caching Policies. (2020). https://github.com/sunnyszy/lrb.Google ScholarGoogle Scholar
  44. [44] Song Zhenyu, Berger Daniel S., Li Kai, and Lloyd Wyatt. 2020. Learning relaxed Belady for content distribution network caching. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 529544.Google ScholarGoogle Scholar
  45. [45] Starobinski David and Tse David. 2001. Probabilistic methods for web caching. Performance Evaluation 46, 2 (2001), 125137.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Subramanian Ranjith, Smaragdakis Yannis, and Loh Gabriel H.. 2006. Adaptive caches: Effective shaping of cache behavior to workloads. In Proc. of the 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 385396.Google ScholarGoogle Scholar
  47. [47] Tanenbaum Andrew S.. 2001. Modern Operating Systems (2nd ed.). Upper Saddle River, NJ, USA: Prentice-Hall.Google ScholarGoogle Scholar
  48. [48] Vietri Giuseppe, Rodriguez Liana V., Martinez Wendy A., Lyons Steven, Liu Jason, Rangaswami Raju, Zhao Ming, and Narasimhan Giri. 2018. Driving cache replacement with ML-based LeCaR. In 10th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage).Google ScholarGoogle Scholar
  49. [49] Waldspurger Carl, Saemundsson Trausti, Ahmad Irfan, and Park Nohhyun. 2017. Cache modeling and optimization using miniature simulations. In USENIX Annual Technical Conference (ATC). 487498.Google ScholarGoogle Scholar
  50. [50] Young Neal. 1991. On-line caching as cache size varies. In Proc. of the Second Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 241250.Google ScholarGoogle Scholar
  51. [51] Zhong Chen, Zhao Xingsheng, and Jiang Song. 2021. LIRS2: An improved LIRS replacement algorithm. In Proc. of the 14th ACM International Systems and Storage Conference (SYSTOR).Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Zhou Ke, Sun Si, Wang Hua, Huang Ping, He Xubin, Lan Rui, Li Wenyan, Liu Wenjie, and Yang Tianming. 2018. Demystifying cache policies for photo stores at scale: A Tencent case study. In Proceedings of the 2018 International Conference on Supercomputing. 284294.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Lightweight Robust Size Aware Cache Management

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Storage
        ACM Transactions on Storage  Volume 18, Issue 3
        August 2022
        244 pages
        ISSN:1553-3077
        EISSN:1553-3093
        DOI:10.1145/3555792
        • Editor:
        • Sam H. Noh
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 24 August 2022
        • Online AM: 2 June 2022
        • Revised: 1 December 2021
        • Accepted: 1 December 2021
        • Received: 1 June 2021
        Published in tos Volume 18, Issue 3

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed
      • Article Metrics

        • Downloads (Last 12 months)298
        • Downloads (Last 6 weeks)28

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!