Abstract
Modern key-value stores, object stores, Internet proxy caches, and Content Delivery Networks (CDN) often manage objects of diverse sizes, e.g., blobs, video files of different lengths, images with varying resolutions, and small documents. In such workloads, size-aware cache policies outperform size-oblivious algorithms. Unfortunately, existing size-aware algorithms tend to be overly complicated and computationally expensive.
Our work follows a more approachable pattern; we extend the prevalent (size-oblivious) TinyLFU cache admission policy to handle variable-sized items. Implementing our approach inside two popular caching libraries only requires minor changes. We show that our algorithms yield competitive or better hit-ratios and byte hit-ratios compared to the state-of-the-art size-aware algorithms such as AdaptSize, LHD, LRB, and GDSF. Further, a runtime comparison indicates that our implementation is faster by up to 3× compared to the best alternative, i.e., it imposes a much lower CPU overhead.
- [1] . 1995. Caching Proxies: Limitations and Potentials.
Technical Report . USA.Google ScholarDigital Library
- [2] . 2017. TinyCache - an effective cache admission filter. In GLOBECOM 2017-2017 IEEE Global Communications Conference. 1–6.Google Scholar
Digital Library
- [3] . 2019. AViC: A cache for adaptive bitrate video. In Proc. of the 15th International ACM Conference on Emerging Networking Experiments And Technologies (CoNEXT). 305–317.Google Scholar
Digital Library
- [4] . 2000. Performance evaluation of web proxy cache replacement policies. Perform. Eval. 39, 1–4 (
Feb. 2000), 149–164.Google ScholarDigital Library
- [5] . 2004. CAR: Clock with adaptive replacement. In Proc. of the 3rd USENIX Conf. on File and Storage Technologies (FAST). 187–200.Google Scholar
- [6] . 2018. LHD: Improving cache hit rate by maximizing hit density. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 389–403.Google Scholar
- [7] . 2018. Simulation Code for the LHD Cache Replacement Policy. (2018). https://github.com/CMU-CORGI/LHD.Google Scholar
- [8] . 1966. A study of replacement algorithms for a virtual-storage computer. IBM Systems Journal 5, 2 (1966), 78–101.Google Scholar
Digital Library
- [9] . 2020. The CacheLib caching engine: Design and experiences at scale. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI). 753–768.Google Scholar
- [10] . 2018. Practical bounds on optimal caching with variable object sizes. Proceedings of the ACM on Measurement and Analysis of Computing Systems 2, 2 (2018), 1–38.Google Scholar
Digital Library
- [11] . 2017. AdaptSize: Orchestrating the hot object memory cache in a content delivery network. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 483–498.Google Scholar
- [12] . 2017. Webcachesim, a C++ Cache Simulator. (2017). https://github.com/dasebe/webcachesim.Google Scholar
- [13] . 2017. Hyperbolic caching: Flexible caching for web applications. In 2017 USENIX Annual Technical Conference (USENIX ATC). 499–511.Google Scholar
Digital Library
- [14] . 1998. Improving WWW Proxies Performance with Greedy-Dual-Size-Frequency Caching Policy.
Technical Report . In HP Tech. Report.Google Scholar - [15] . 2010. Data caching as a cloud service. In Proceedings of the 4th ACM International Workshop on Large Scale Distributed Systems and Middleware (LADIS’10). ACM, 18–21.Google Scholar
Digital Library
- [16] . 2011. Design and implementation of caching services in the cloud. IBM Journal of Research and Development 55, 6 (2011), 9:1–9:11.Google Scholar
Digital Library
- [17] . 2003. Spectral bloom filters. In Proc. of the ACM SIGMOD Int. Conf. on Management of Data. 241–252.Google Scholar
- [18] . 2005. An improved data stream summary: The count-min sketch and its applications. J. Algorithms 55, 1 (
April 2005), 58–75.Google ScholarDigital Library
- [19] . 2020. Ristretto: A High Performance Memory-Bound Go Cache. (2020). https://github.com/dgraph-io/ristretto.Google Scholar
- [20] . 2018. Adaptive software cache management. In Proceedings of the 19th International Middleware Conference. 94–106.Google Scholar
Digital Library
- [21] . 2017. TinyLFU: A highly efficient cache admission policy. ACM Transactions on Storage (TOS’17).Google Scholar
Digital Library
- [22] . 2004. Distributed caching with Memcached. Linux J. 2004, 124 (
Aug. 2004).Google ScholarDigital Library
- [23] . 2020. Netflix and YouTube are slowing down in Europe to keep the internet from breaking. CNN Business (2020). https://www.cnn.com/2020/03/19/tech/netflix-internet-overload-eu/index.htmlGoogle Scholar
- [24] . 2014. Trade-offs in optimizing the cache deployments of CDNs. In IEEE Conference on Computer Communications (INFOCOM). 460–468.Google Scholar
Cross Ref
- [25] . 2012. Computer Architecture - A Quantitative Approach (5. ed.). Morgan Kaufmann.Google Scholar
- [26] . 2005. CLOCK-Pro: An effective improvement of the CLOCK replacement. In Proc. of the USENIX Annual Technical Conference (ATC).Google Scholar
- [27] . 2002. LIRS: An efficient low inter-reference recency set replacement policy to improve buffer cache performance. In Proc. of the International Conference on Measurements and Modeling of Computer Systems SIGMETRICS. 31–42.Google Scholar
Digital Library
- [28] . 1994. 2Q: A low overhead high performance buffer management replacement algorithm. In Proc. of the 20th Int. Conf. on Very Large Data Bases (VLDB). 439–450.Google Scholar
- [29] . 2002. Exploitation of different types of locality for web caches. In Proc. of the 7th Int. Symposium on Computers and Communications (ISCC).Google Scholar
- [30] . 2010. An O(1) Algorithm for Implementing the LFU Cache Eviction Scheme.
Technical Report . http://dhruvbird.com/lfu.pdf.Google Scholar - [31] . 2017. Understanding storage traffic characteristics on enterprise virtual desktop infrastructure. In Proceedings of the 10th ACM International Systems and Storage Conference. 1–11.Google Scholar
Digital Library
- [32] . 2001. LRFU: A spectrum of policies that subsumes the least recently used and least frequently used policies. IEEE Trans. Computers 50, 12 (2001), 1352–1361.Google Scholar
Digital Library
- [33] . 2018. DLIRS: Improving low inter-reference recency set cache replacement policy with dynamics. In Proc. of the 11th ACM International Systems and Storage Conference (SYSTOR).Google Scholar
Digital Library
- [34] . 2016. Caffeine: A high performance caching library for Java 8. https://github.com/ben-manes/caffeine.Google Scholar
- [35] . 2003. ARC: A self-tuning, low overhead replacement cache. In Proc. of the 2nd USENIX Conf. on File and Storage Technologies (FAST). 115–130.Google Scholar
- [36] . 2017. Understanding application-level caching in web applications: A comprehensive introduction and survey of state-of-the-art approaches. ACM Comput. Surv. 50, 6 (
Nov. 2017).Google ScholarDigital Library
- [37] . 2008. Write off-loading: Practical power management for enterprise storage. ACM Transactions on Storage (TOS) 4, 3 (2008), 1–23.Google Scholar
Digital Library
- [38] . 2017. Access-time-aware cache algorithms. ACM Trans. Model. Perform. Eval. Comput. Syst. 2, 4 (
Nov. 2017).Google ScholarDigital Library
- [39] . 1993. The LRU-K page replacement algorithm for database disk buffering. ACM SIGMOD Rec. 22, 2 (
June 1993), 297–306.Google ScholarDigital Library
- [40] . 2017. FRD: A filtering based buffer cache algorithm that considers both frequency and reuse distance. In Proc. of the 33rd IEEE International Conference on Massive Storage Systems and Technology (MSST).Google Scholar
- [41] . 2020. Using Redis as an LRU Cache. (2020). https://redis.io/topics/lru-cache.Google Scholar
- [42] . 2021. Learning cache replacement with CACHEUS. In 19th USENIX Conference on File and Storage Technologies (FAST).Google Scholar
- [43] . 2020. A Simulator for CDN Caching and Web Caching Policies. (2020). https://github.com/sunnyszy/lrb.Google Scholar
- [44] . 2020. Learning relaxed Belady for content distribution network caching. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 529–544.Google Scholar
- [45] . 2001. Probabilistic methods for web caching. Performance Evaluation 46, 2 (2001), 125–137.Google Scholar
Digital Library
- [46] . 2006. Adaptive caches: Effective shaping of cache behavior to workloads. In Proc. of the 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 385–396.Google Scholar
- [47] . 2001. Modern Operating Systems (2nd ed.). Upper Saddle River, NJ, USA: Prentice-Hall.Google Scholar
- [48] . 2018. Driving cache replacement with ML-based LeCaR. In 10th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage).Google Scholar
- [49] . 2017. Cache modeling and optimization using miniature simulations. In USENIX Annual Technical Conference (ATC). 487–498.Google Scholar
- [50] . 1991. On-line caching as cache size varies. In Proc. of the Second Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 241–250.Google Scholar
- [51] . 2021. LIRS2: An improved LIRS replacement algorithm. In Proc. of the 14th ACM International Systems and Storage Conference (SYSTOR).Google Scholar
Digital Library
- [52] . 2018. Demystifying cache policies for photo stores at scale: A Tencent case study. In Proceedings of the 2018 International Conference on Supercomputing. 284–294.Google Scholar
Digital Library
Index Terms
Lightweight Robust Size Aware Cache Management
Recommendations
Size-Aware Cache Management for Compressed Cache Architectures
A practical way to increase the effective capacity of a microprocessor's cache, without physically increasing the cache size, is to employ data compression. Last-Level Caches (LLC) are particularly amenable to such compression schemes, since the primary ...
Heterogeneous way-size cache
ICS '06: Proceedings of the 20th annual international conference on SupercomputingSet-associative cache architectures are commonly used. These caches consist of a number of ways, each of the same size. We have observed that the different ways have very different utilization, which motivates the design of caches with heterogeneous way ...
An Efficient Lightweight Shared Cache Design for Chip Multiprocessors
APPT '09: Proceedings of the 8th International Symposium on Advanced Parallel Processing TechnologiesThe large working sets of commercial and scientific workloads favor a shared L2 cache design that maximizes the aggregate cache capacity and minimizes off-chip memory requests in Chip Multiprocessors (CMP). The exponential increase in the number of ...






Comments