Abstract
Web applications employ key-value stores to cache the data that is most commonly accessed. The cache improves an web application's performance by serving its requests from memory, avoiding fetching them from the backend database. Since the memory space is limited, maximizing the memory utilization is a key to delivering the best performance possible. This has lead to the use of multi-tenant systems, allowing applications to share cache space. In addition, application data access patterns change over time, so the system should be adaptive in its memory allocation.
In this work, we address both multi-tenancy (where a single cache is used for multiple applications) and dynamic workloads (changing access patterns) using a model that relates the cache size to the application miss ratio, known as a miss ratio curve. Intuitively, the larger the cache, the less likely the system will need to fetch the data from the database. Our efficient, online construction of the miss ratio curve allows us to determine a near optimal memory allocation given the available system memory, while adapting to changing data access patterns. We show that our model outperforms an existing state-of-the-art sharing model, Memshare, in terms of overall cache hit ratio and does so at a lower time cost. We show that for a typical system, overall hit ratio is consistently 1 percentage point greater and 99.9th percentile latency is reduced by as much as 2.9% under standard web application workloads containing millions of requests.
- Cristina L. Abad, Andres G. Abad, and Luis E. Lucio. 2017. Dynamic Memory Partitioning for Cloud Caches with Heterogeneous Backends. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (ICPE ’17). ACM, New York, NY, USA, 87–90. Google Scholar
Digital Library
- Amazon. 2018. Amazon Web Services. (May 2018). Retrieved May 10, 2018 from https://aws.amazon.comGoogle Scholar
- Berk Atikoglu, Yuehai Xu, Eitan Frachtenberg, Song Jiang, and Mike Paleczny. 2012. Workload Analysis of a Large-scale Key-value Store. In Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS ’12). ACM, New York, NY, USA, 53–64. Google Scholar
Digital Library
- E. Berg and E. Hagersten. 2004. StatCache: a probabilistic approach to efficient and accurate data locality analysis. In 2004 IEEE International Symposium on - ISPASS Performance Analysis of Systems and Software (ISPASS ’04). 20–27. Google Scholar
Digital Library
- Aaron Blankstein, Siddhartha Sen, and Michael J. Freedman. 2017. Hyperbolic Caching: Flexible Caching for Web Applications. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 499–511. Google Scholar
Digital Library
- L. Breslau, Pei Cao, Li Fan, G. Phillips, and S. Shenker. 1999. Web caching and Zipf-like distributions: evidence and implications. In Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings (INFOCOM ’99), Vol. 1. 126–134 vol.1.Google Scholar
- Damiano Carra. 2016. Benchmarks for testing memcached memory management. (December 2016). Retrieved December 10, 2016 from http://profs.sci.univr.it/~carra/mctools/Google Scholar
- D. Carra and P. Michiardi. 2014. Memory partitioning in Memcached: An experimental performance analysis. In 2014 IEEE International Conference on Communications (ICC ’14). 1154–1159.Google Scholar
- Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti. 2015. Dynacache: Dynamic Cloud Caching. In 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15). USENIX Association, Santa Clara, CA. Google Scholar
Digital Library
- Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti. 2016. Cliffhanger: Scaling Performance Cliffs in Web Memory Caches. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). USENIX Association, Santa Clara, CA, 379–392. Google Scholar
Digital Library
- Asaf Cidon, Daniel Rushton, Stephen M. Rumble, and Ryan Stutsman. 2017. Memshare: a Dynamic Multi-tenant Key-value Cache. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 321–334. Google Scholar
Digital Library
- Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (SoCC ’10). ACM, New York, NY, USA, 143–154. Commun. ACM 11, 5 (May 1968), 323–333. Google Scholar
Digital Library
- Chen Ding and Xiaoya Xiang. 2012. A Higher Order Theory of Locality. In Proceedings of the 2012 ACM SIGPLAN Workshop on Memory Systems Performance and Correctness (MSPC ’12). ACM, New York, NY, USA, 68–69. Google Scholar
Digital Library
- D. Eklov and E. Hagersten. 2010. StatStack: Efficient modeling of LRU caches. In 2010 IEEE International Symposium on Performance Analysis of Systems Software (ISPASS ’10). 55–65.Google Scholar
- Changpeng Fang, S. Can, S. Onder, and Zhenlin Wang. 2005. Instruction based memory distance analysis and its application to optimization. In 14th International Conference on Parallel Architectures and Compilation Techniques (PACT’05). 27–37. Google Scholar
Digital Library
- Xiameng Hu, Xiaolin Wang, Yechen Li, Lan Zhou, Yingwei Luo, Chen Ding, Song Jiang, and Zhenlin Wang. 2015. LAMA: Optimized Localityaware Memory Allocation for Key-value Cache. In 2015 USENIX Annual Technical Conference (USENIX ATC 15). USENIX Association, Santa Clara, CA, 57–69. Google Scholar
Digital Library
- X. Hu, X. Wang, L. Zhou, Y. Luo, C. Ding, S. Jiang, and Z. Wang. 2017. Optimizing Locality-Aware Memory Management of Key-Value Caches. IEEE Transactions on Computers (TC ’17) 66, 5 (May 2017), 862–875. Google Scholar
Digital Library
- Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Chen Ding, and Zhenlin Wang. 2016. Kinetic Modeling of Data Eviction in Cache. In 2016 USENIX Annual Technical Conference (USENIX ATC 16). USENIX Association, Denver, CO, 351–364. Google Scholar
Digital Library
- Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Zhenlin Wang, Chen Ding, and Chencheng Ye. 2018. Fast Miss Ratio Curve Modeling for Storage Cache. ACM Trans. Storage (TOS’18) 14, 2, Article 12 (April 2018), 34 pages. Google Scholar
Digital Library
- N. P. Jouppi. 1990. Improving direct-mapped cache performance by the addition of a small fully-associative cache and prefetch buffers. In The 17th Annual International Symposium on Computer Architecture (ISCA ’90). 364–373. Google Scholar
Digital Library
- Redis Labs. 2018. redis. (May 2018). Retrieved December 10, 2016 from https://redis.ioGoogle Scholar
- Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C. Li, Ryan McElroy, Mike Paleczny, Daniel Peek, Paul Saab, David Stafford, Tony Tung, and Venkateshwaran Venkataramani. 2013. Scaling Memcache at Facebook. In Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13). USENIX, Lombard, IL, 385–398. Google Scholar
Digital Library
- Qifan Pu, Haoyuan Li, Matei Zaharia, Ali Ghodsi, and Ion Stoica. 2016. FairRide: Near-Optimal, Fair Cache Sharing. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). USENIX Association, Santa Clara, CA, 393–406. Google Scholar
Digital Library
- Trausti Saemundsson, Hjortur Bjornsson, Gregory Chockler, and Ymir Vigfusson. 2014. Dynamic Performance Profiling of Cloud Caches. In Proceedings of the ACM Symposium on Cloud Computing (SOCC ’14). ACM, New York, NY, USA, Article 28, 14 pages. Google Scholar
Digital Library
- Carl A. Waldspurger, Nohhyun Park, Alexander Garthwaite, and Irfan Ahmad. 2015. Efficient MRC Construction with SHARDS. In Proceedings of the 13th USENIX Conference on File and Storage Technologies (FAST’15). USENIX Association, Berkeley, CA, USA, 95–110. Google Scholar
Digital Library
- Zhigang Wang, Xiaolin Wang, Fang Hou, Yingwei Luo, and Zhenlin Wang. 2016. Dynamic Memory Balancing for Virtualization. ACM Trans. Archit. Code Optim. (TACO ’16) 13, 1, Article 2 (March 2016), 25 pages. Google Scholar
Digital Library
- Jake Wires, Stephen Ingram, Zachary Drudi, Nicholas J. A. Harvey, and Andrew Warfield. 2014. Characterizing Storage Workloads with Counter Stacks. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, Google Scholar
Digital Library
- Ting Yang, Emery D. Berger, Scott F. Kaplan, and J. Eliot B. Moss. 2006. CRAMM: Virtual Memory Support for Garbage-collected Applications. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI ’06). USENIX Association, Berkeley, CA, USA, 103–116. Google Scholar
Digital Library
- Weiming Zhao and Zhenlin Wang. 2009. Dynamic Memory Balancing for Virtual Machines. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’09). ACM, New York, NY, USA, 21–30. Google Scholar
Digital Library
- Yutao Zhong, Xipeng Shen, and Chen Ding. 2009. Program Locality Analysis Using Reuse Distance. ACM Trans. Program. Lang. Syst. (TOPLAS ’09) 31, 6, Article 20 (Aug. 2009), 39 pages. Google Scholar
Digital Library
Index Terms
mPart: miss-ratio curve guided partitioning in key-value stores
Recommendations
Fast Miss Ratio Curve Modeling for Storage Cache
The reuse distance (least recently used (LRU) stack distance) is an essential metric for performance prediction and optimization of storage cache. Over the past four decades, there have been steady improvements in the algorithmic efficiency of reuse ...
mPart: miss-ratio curve guided partitioning in key-value stores
ISMM 2018: Proceedings of the 2018 ACM SIGPLAN International Symposium on Memory ManagementWeb applications employ key-value stores to cache the data that is most commonly accessed. The cache improves an web application's performance by serving its requests from memory, avoiding fetching them from the backend database. Since the memory space ...
Replacement Policy Adaptable Miss Curve Estimation for Efficient Cache Partitioning
Cache replacement policies and cache partitioning are well-known cache management techniques which aim to eliminate inter- and intra-application contention caused by co-running applications, respectively. Since replacement policies can change ...







Comments