skip to main content
article
Public Access

mPart: miss-ratio curve guided partitioning in key-value stores

Published:18 June 2018Publication History
Skip Abstract Section

Abstract

Web applications employ key-value stores to cache the data that is most commonly accessed. The cache improves an web application's performance by serving its requests from memory, avoiding fetching them from the backend database. Since the memory space is limited, maximizing the memory utilization is a key to delivering the best performance possible. This has lead to the use of multi-tenant systems, allowing applications to share cache space. In addition, application data access patterns change over time, so the system should be adaptive in its memory allocation.

In this work, we address both multi-tenancy (where a single cache is used for multiple applications) and dynamic workloads (changing access patterns) using a model that relates the cache size to the application miss ratio, known as a miss ratio curve. Intuitively, the larger the cache, the less likely the system will need to fetch the data from the database. Our efficient, online construction of the miss ratio curve allows us to determine a near optimal memory allocation given the available system memory, while adapting to changing data access patterns. We show that our model outperforms an existing state-of-the-art sharing model, Memshare, in terms of overall cache hit ratio and does so at a lower time cost. We show that for a typical system, overall hit ratio is consistently 1 percentage point greater and 99.9th percentile latency is reduced by as much as 2.9% under standard web application workloads containing millions of requests.

References

  1. Cristina L. Abad, Andres G. Abad, and Luis E. Lucio. 2017. Dynamic Memory Partitioning for Cloud Caches with Heterogeneous Backends. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (ICPE ’17). ACM, New York, NY, USA, 87–90. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Amazon. 2018. Amazon Web Services. (May 2018). Retrieved May 10, 2018 from https://aws.amazon.comGoogle ScholarGoogle Scholar
  3. Berk Atikoglu, Yuehai Xu, Eitan Frachtenberg, Song Jiang, and Mike Paleczny. 2012. Workload Analysis of a Large-scale Key-value Store. In Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS ’12). ACM, New York, NY, USA, 53–64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. E. Berg and E. Hagersten. 2004. StatCache: a probabilistic approach to efficient and accurate data locality analysis. In 2004 IEEE International Symposium on - ISPASS Performance Analysis of Systems and Software (ISPASS ’04). 20–27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Aaron Blankstein, Siddhartha Sen, and Michael J. Freedman. 2017. Hyperbolic Caching: Flexible Caching for Web Applications. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 499–511. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. L. Breslau, Pei Cao, Li Fan, G. Phillips, and S. Shenker. 1999. Web caching and Zipf-like distributions: evidence and implications. In Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings (INFOCOM ’99), Vol. 1. 126–134 vol.1.Google ScholarGoogle Scholar
  7. Damiano Carra. 2016. Benchmarks for testing memcached memory management. (December 2016). Retrieved December 10, 2016 from http://profs.sci.univr.it/~carra/mctools/Google ScholarGoogle Scholar
  8. D. Carra and P. Michiardi. 2014. Memory partitioning in Memcached: An experimental performance analysis. In 2014 IEEE International Conference on Communications (ICC ’14). 1154–1159.Google ScholarGoogle Scholar
  9. Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti. 2015. Dynacache: Dynamic Cloud Caching. In 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15). USENIX Association, Santa Clara, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti. 2016. Cliffhanger: Scaling Performance Cliffs in Web Memory Caches. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). USENIX Association, Santa Clara, CA, 379–392. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Asaf Cidon, Daniel Rushton, Stephen M. Rumble, and Ryan Stutsman. 2017. Memshare: a Dynamic Multi-tenant Key-value Cache. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 321–334. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (SoCC ’10). ACM, New York, NY, USA, 143–154. Commun. ACM 11, 5 (May 1968), 323–333. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Chen Ding and Xiaoya Xiang. 2012. A Higher Order Theory of Locality. In Proceedings of the 2012 ACM SIGPLAN Workshop on Memory Systems Performance and Correctness (MSPC ’12). ACM, New York, NY, USA, 68–69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. D. Eklov and E. Hagersten. 2010. StatStack: Efficient modeling of LRU caches. In 2010 IEEE International Symposium on Performance Analysis of Systems Software (ISPASS ’10). 55–65.Google ScholarGoogle Scholar
  15. Changpeng Fang, S. Can, S. Onder, and Zhenlin Wang. 2005. Instruction based memory distance analysis and its application to optimization. In 14th International Conference on Parallel Architectures and Compilation Techniques (PACT’05). 27–37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Xiameng Hu, Xiaolin Wang, Yechen Li, Lan Zhou, Yingwei Luo, Chen Ding, Song Jiang, and Zhenlin Wang. 2015. LAMA: Optimized Localityaware Memory Allocation for Key-value Cache. In 2015 USENIX Annual Technical Conference (USENIX ATC 15). USENIX Association, Santa Clara, CA, 57–69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. X. Hu, X. Wang, L. Zhou, Y. Luo, C. Ding, S. Jiang, and Z. Wang. 2017. Optimizing Locality-Aware Memory Management of Key-Value Caches. IEEE Transactions on Computers (TC ’17) 66, 5 (May 2017), 862–875. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Chen Ding, and Zhenlin Wang. 2016. Kinetic Modeling of Data Eviction in Cache. In 2016 USENIX Annual Technical Conference (USENIX ATC 16). USENIX Association, Denver, CO, 351–364. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Zhenlin Wang, Chen Ding, and Chencheng Ye. 2018. Fast Miss Ratio Curve Modeling for Storage Cache. ACM Trans. Storage (TOS’18) 14, 2, Article 12 (April 2018), 34 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. N. P. Jouppi. 1990. Improving direct-mapped cache performance by the addition of a small fully-associative cache and prefetch buffers. In The 17th Annual International Symposium on Computer Architecture (ISCA ’90). 364–373. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Redis Labs. 2018. redis. (May 2018). Retrieved December 10, 2016 from https://redis.ioGoogle ScholarGoogle Scholar
  22. Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C. Li, Ryan McElroy, Mike Paleczny, Daniel Peek, Paul Saab, David Stafford, Tony Tung, and Venkateshwaran Venkataramani. 2013. Scaling Memcache at Facebook. In Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13). USENIX, Lombard, IL, 385–398. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Qifan Pu, Haoyuan Li, Matei Zaharia, Ali Ghodsi, and Ion Stoica. 2016. FairRide: Near-Optimal, Fair Cache Sharing. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). USENIX Association, Santa Clara, CA, 393–406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Trausti Saemundsson, Hjortur Bjornsson, Gregory Chockler, and Ymir Vigfusson. 2014. Dynamic Performance Profiling of Cloud Caches. In Proceedings of the ACM Symposium on Cloud Computing (SOCC ’14). ACM, New York, NY, USA, Article 28, 14 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Carl A. Waldspurger, Nohhyun Park, Alexander Garthwaite, and Irfan Ahmad. 2015. Efficient MRC Construction with SHARDS. In Proceedings of the 13th USENIX Conference on File and Storage Technologies (FAST’15). USENIX Association, Berkeley, CA, USA, 95–110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Zhigang Wang, Xiaolin Wang, Fang Hou, Yingwei Luo, and Zhenlin Wang. 2016. Dynamic Memory Balancing for Virtualization. ACM Trans. Archit. Code Optim. (TACO ’16) 13, 1, Article 2 (March 2016), 25 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Jake Wires, Stephen Ingram, Zachary Drudi, Nicholas J. A. Harvey, and Andrew Warfield. 2014. Characterizing Storage Workloads with Counter Stacks. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ting Yang, Emery D. Berger, Scott F. Kaplan, and J. Eliot B. Moss. 2006. CRAMM: Virtual Memory Support for Garbage-collected Applications. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI ’06). USENIX Association, Berkeley, CA, USA, 103–116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Weiming Zhao and Zhenlin Wang. 2009. Dynamic Memory Balancing for Virtual Machines. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’09). ACM, New York, NY, USA, 21–30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Yutao Zhong, Xipeng Shen, and Chen Ding. 2009. Program Locality Analysis Using Reuse Distance. ACM Trans. Program. Lang. Syst. (TOPLAS ’09) 31, 6, Article 20 (Aug. 2009), 39 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. mPart: miss-ratio curve guided partitioning in key-value stores

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 53, Issue 5
          ISMM '18
          May 2018
          119 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/3299706
          Issue’s Table of Contents
          • cover image ACM Conferences
            ISMM 2018: Proceedings of the 2018 ACM SIGPLAN International Symposium on Memory Management
            June 2018
            119 pages
            ISBN:9781450358019
            DOI:10.1145/3210563

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 18 June 2018

          Check for updates

          Qualifiers

          • article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!