skip to main content
research-article

Management of Multilevel, Multiclient Cache Hierarchies with Application Hints

Published:01 May 2011Publication History
Skip Abstract Section

Abstract

Multilevel caching, common in many storage configurations, introduces new challenges to traditional cache management: data must be kept in the appropriate cache and replication avoided across the various cache levels. Additional challenges are introduced when the lower levels of the hierarchy are shared by multiple clients. Sharing can have both positive and negative effects. While data fetched by one client can be used by another client without incurring additional delays, clients competing for cache buffers can evict each other’s blocks and interfere with exclusive caching schemes.

We present a global noncentralized, dynamic and informed management policy for multiple levels of cache, accessed by multiple clients. Our algorithm, MC2, combines local, per client management with a global, system-wide scheme, to emphasize the positive effects of sharing and reduce the negative ones. Our local management scheme, Karma, uses readily available information about the client’s future access profile to save the most valuable blocks, and to choose the best replacement policy for them. The global scheme uses the same information to divide the shared cache space between clients, and to manage this space. Exclusive caching is maintained for nonshared data and is disabled when sharing is identified.

Previous studies have partially addressed these challenges through minor changes to the storage interface. We show that all these challenges can in fact be addressed by combining minor interface changes with smart allocation and replacement policies. We show the superiority of our approach through comparison to existing solutions, including LRU, ARC, MultiQ, LRU-SP, and Demote, as well as a lower bound on optimal I/O response times. Our simulation results demonstrate better cache performance than all other solutions and up to 87% better performance than LRU on representative workloads.

References

  1. Aggarwal, A., Alpern, B., Chandra, A., and Snir, M. 1987. A model for hierarchical memory. In Proceedings of the ACM Conference on Theory of Computing. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ari, I. 2004. Design and management of globally distributed network caches. Ph.D. thesis, University of California Santa Cruz. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Arpaci-Dusseau, A. C. and Arpaci-Dusseau, R. H. 2001. Information and control in gray-box systems. In Proceedings of the Symposium on Operating Systems Principles (SOSP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Azagury, A., Dreizin, V., Factor, M., Henis, E., Naor, D., Rinetzky, N., Rodeh, O., Satran, J., Tavory, A., and Yerushalmi, L. 2003. Towards an object store. In Proceedings of the NASA Goddard Conference on Mass Storage Systems and Technologies (MSST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Baek, S. H. and Park, K. H. 2008. Prefetching with adaptive cache culling for striped disk arrays. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Bansal, S. and Modha, D. S. 2004. CAR: Clock with adaptive replacement. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Barzilai, T. and Golan, G. 2002. Accessing application identification information in the storage tier. Disclosure IL8-2002-0055, IBM Haifa Labs.Google ScholarGoogle Scholar
  8. Batsakis, A., Burns, R., Kanevsky, A., Lentini, J., and Talpey, T. 2008. AWOL: an adaptive write optimizations layer. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Belady, L. 1966. A study of replacement algorithms for a virtual-storage computer. IBM Syst. J. 5, 2, 78--101. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Bhadkamkar, M., Guerra, J., Useche, L., Burnett, S., Liptak, J., Rangaswami, R., and Hristidis, V. 2009. BORG: block-reORGanization for self-optimizing storage systems. In Proceedings of the 7th Conference on File and Stroage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Cao, P., Felten, E. W., Karlin, A. R., and Li, K. 1996. Implementation and performance of integrated application-controlled file caching, prefetching, and disk scheduling. ACM Trans. Comput. Syst. 14, 4, 311--343. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chen, Z., Zhou, Y., and Li, K. 2003. Eviction-based placement for storage caches. In Proceedings of the USENIX Annual Technical Conference.Google ScholarGoogle Scholar
  13. Chen, Z., Zhang, Y., Zhou, Y., Scott, H., and Schiefer, B. 2005. Empirical evaluation of multi-level buffer cache collaboration for storage systems. In Proceedings of the ACM SIGMETRICS Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Choi, J., Noh, S. H., Min, S. L., and Cho, Y. 1999. An implementation study of a detection-based adaptive block replacement scheme. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Choi, J., Noh, S. H., Min, S. L., and Cho, Y. 2000. Towards application/file-level characterization of block references: a case for fine-grained buffer management. In Proceedings of the ACM SIGMETRICS Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Chou, H.-T. and DeWitt, D. J. 1985. An evaluation of buffer management strategies for relational database systems. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Chrobak, M. and Noga, J. 2000. Competitive algorithms for relaxed list update and multilevel caching. J. Algor. 34, 2, 282--308. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Dahlin, M. D., Wang, R. Y., Anderson, T. E., and Patterson, D. A. 1994. Cooperative caching: using remote client memory to improve file system performance. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Ding, X., Jiang, S., Chen, F., Davis, K., and Zhang, X. 2007. DiskSeen: Exploiting disk layout and access history to enhance i/o prefetch. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Gabor, R., Weiss, S., and Mendelson, A. 2007. Fairness enforcement in switch on event multithreading. ACM Trans. Architec. Code Optim. 4, 3, 15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Ganger, G. R., Ellard, D., and Seltzer, M. I. 2004. File classification in self-∗ storage systems. In Proceedings of the International Conference on Autonomic Computing (ICAC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Gill, B. 2008. On multi-level exclusive caching: Offline optimality and why promotions are better than demotions. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Gill, B. S. and Modha, D. S. 2005a. SARC: Sequential prefetching in adaptive replacement cache. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Gill, B. S. and Modha, D. S. 2005b. WOW: Wise ordering for writes-combining spatial and temporal locality in non-volatile caches. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gill, B. S. and Bathen, L. A. D. 2007. AMP: adaptive multi-stream prefetching in a shared cache. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Gniady, C., Butt, A. R., and Hu., Y. C. 2004. Program counter based pattern classification in buffer caching. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Gramacy, R. B., Warmuth, M. K., Brandt, S. A., and Ari, I. 2002. Adaptive caching by refetching. In Neural Information Processing Systems (NIPS).Google ScholarGoogle Scholar
  28. Harizopoulos, S. and Ailamaki, A. 2003. A case for staged database systems. In Proceedings of the Conference on Innovative Data Systems Research (CIDR).Google ScholarGoogle Scholar
  29. Jauhari, R., Carey, M. J., and Livny, M. 1990. Priority-hints: An algorithm for priority-based buffer management. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Jiang, S. and Zhang, X. 2002. LIRS: An efficient low inter-reference recency set replacement policy to improve buffer cache performance. In Proceedings of the ACM SIGMETRICS Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jiang, S. and Zhang, X. 2004. ULC: A file block placement and replacement protocol to effectively exploit hierarchical locality in multi-level buffer caches. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Jiang, S., Ding, X., Chen, F., Tan, E., and Zhang, X. 2005. DULO: An effective buffer cache management scheme to exploit both temporal and spatial localities. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Jiang, S., Petrini, F., Ding, X., and Zhang, X. 2006. A locality-aware cooperative cache management protocol to improve network file system performance. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Johnson, T. and Shasha, D. 1994. 2Q: a low overhead high performance buffer management replacement algorithm. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Karpovich, J. F., Grimshaw, A. S., and French, J. C. 1994. Extensible file system (ELFS): an object-oriented approach to high performance file I/O. In Proceedings of the Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Kim, J. M., Choi, J., Kim, J., Noh, S. H., Min, S. L., Cho, Y., and Kim, C. S. 2000. A low-overhead high-performance unified buffer management scheme that exploits sequential and looping references. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Lee, D., Choi, J., Kim, J.-H., Noh, S. H., Min, S. L., Cho, Y., and Kim, C.-S. 1999. On the existence of a spectrum of policies that subsumes the least recently used (LRU) and least frequently used (LFU) policies. In Proceedings of the ACM SIGMETRICS Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Li, X., Aboulnaga, A., Salem, K., Sachedina, A., and Gao, S. 2005. Second-tier cache management using write hints. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Li, Z., Chen, Z., Srinivasan, S. M., and Zhou, Y. 2004. C-miner: Mining block correlations in storage systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Liang, S., Jiang, S., and Zhang, X. 2007. STEP: Sequentiality and thrashing detection based prefetching to improve performance of networked storage servers. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Liu, X., Aboulnaga, A., Salem, K., and Li, X. 2009. CLIC: client-informed caching for storage servers. In Proceedings of the 7th Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Llanos, D. R. and Palop, B. 2006. An open-source TPC-C implementation for parallel and distributed systems. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Lumb, C. R., Schindler, J., and Ganger, G. R. 2002. Freeblock scheduling outside of disk firmware. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Megiddo, N. and Modha, D. S. 2003. ARC: A self-tuning, low overhead replacement cache. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Momjian, B. 2000. PostgreSQL: Introduction and Concepts. Addison-Wesley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Muntz, D. and Honeyman, P. 1992. Multi-level caching in distributed file systems - or - your cache ain’t nuthin’ but trash. In Proceedings of the USENIX Winter Conference.Google ScholarGoogle Scholar
  47. Ng, R., Faloutsos, C., and Sellis, T. 1991. Flexible buffer allocation based on marginal gains. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. O’Neil, E. J., O’Neil, P. E., and Weikum, G. 1993. The LRU-K page replacement algorithm for database disk buffering. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Ou, L., He, X., Kosa, M. J., and Scott, S. L. 2005. A unified multiple-level cache for high performance storage systems. In Proceedings of the IEEE International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Patterson, R. H., Gibson, G. A., Ginting, E., Stodolsky, D., and Zelenka, J. 2001. Informed prefetching and caching. In Proceedings of the High Performance Mass Storage and Parallel I/O: Technologies and Applications, H. Jin, T. Cortes, and R. Buyya Eds., IEEE Computer Society Press and Wiley, NY, 224--244.Google ScholarGoogle Scholar
  51. Robinson, J. T. and Devarakonda, M. V. 1990. Data cache management using frequency-based replacement. In Proceedings of the ACM SIGMETRICS Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Sacco, G. M. and Schkolnick, M. 1982. A mechanism for managing the buffer pool in a relational database system using the hot set model. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Sacco, G. M. and Schkolnick, M. 1986. Buffer management in relational database systems. ACM Trans. Datab. Syst. 11, 4, 473--498. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Sarkar, P. and Hartman, J. 1996. Efficient cooperative caching using hints. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Schindler, J., Griffin, J. L., Lumb, C. R., and Ganger, G. R. 2002. Track-aligned extents: Matching access patterns to disk drive characteristics. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Schindler, J., Ailamaki, A., and Ganger., G. R. 2003. Lachesis: Robust database storage management based on device-specific performance characteristics. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Selinger, P. G., Astrahan, M. M., Chamberlin, D. D., Lorie, R. A., and Price, T. G. 1979. Access path selection in a relational database management system. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Sivathanu, M., Prabhakaran, V., Popovici, F. I., Denehy, T. E., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2003. Semantically-smart disk systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Sivathanu, M., Bairavasundaram, L. N., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005. Database-aware semantically-smart storage. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Soundararajan, G., Mihailescu, M., and Amza, C. 2008. Context-aware prefetching at the storage server. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Soundararajan, G., Lupei, D., Ghanbari, S., Popescu, A. D., Chen, J., and Amza, C. 2009. Dynamic resource allocation for database servers running on virtual storage. In Proceedings of the 7th Conference on File and Stroage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Stonebraker, M. 1981. Operating system support for database management. Comm. ACM 24, 7, 412--418. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Storm, A. J., Garcia-Arellano, C., Lightstone, S. S., Diao, Y., and Surendra, M. 2006. Adaptive self-tuning memory in DB2. In Proceedings of the 32nd International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Takagi, M. and Hiraki, K. 2004. Inter-reference gap distribution replacement: an improved replacement algorithm for set-associative caches. In Proceedings of the International Conference on Supercomputing (ICS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Voelker, G. M., Anderson, E. J., Kimbrel, T., Feeley, M. J., Chase, J. S., Karlin, A. R., and Levy, H. M. 1998. Implementing cooperative prefetching and caching in a globally-managed memory system. In Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Wachs, M., Abd-El-Malek, M., Thereska, E., and Ganger, G. R. 2007. Argon: Performance insulation for shared storage servers. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Wong, T. M. and Wilkes, J. 2002. My cache or yours? Making storage more exclusive. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Yadgar, G., Factor, M., and Schuster, A. 2007. Karma: Know-It-All Replacement for a Multilevel Cache. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Yadgar, G., Factor, M., Li, K., and Schuster, A. 2008. MC<sup>2</sup>: Multiple Clients on A Multilevel Cache. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Zhang, Z., Lee, K., Ma, X., and Zhou, Y. 2008. PFC: Transparent optimization of existing prefetching strategies for multi-level storage systems. In Proceedings of the 28th International Conference on Distributed Computing Systems (ICDCS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Zhang, Z., Kulkarni, A., Ma, X., and Zhou, Y. 2009. Memory resource allocation for file system prefetching: From a supply chain management perspective. In Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys). Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Zhou, P., Pandey, V., Sundaresan, J., Raghuraman, A., Zhou, Y., and Kumar, S. 2004a. Dynamic tracking of page miss ratio curve for memory management. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Zhou, Y., Philbin, J., and Li, K. 2001. The multi-queue replacement algorithm for second level buffer caches. In Proceedings of the USENIX Annual Technical Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Zhou, Y., Chen, Z., and Li, K. 2004b. Second-level buffer cache management. IEEE Trans. Paral. Distrib. Syst. 15, 6, 505--519. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Management of Multilevel, Multiclient Cache Hierarchies with Application Hints

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Computer Systems
      ACM Transactions on Computer Systems  Volume 29, Issue 2
      May 2011
      132 pages
      ISSN:0734-2071
      EISSN:1557-7333
      DOI:10.1145/1963559
      Issue’s Table of Contents

      Copyright © 2011 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 May 2011
      • Accepted: 1 December 2010
      • Revised: 1 September 2010
      • Received: 1 June 2009
      Published in tocs Volume 29, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!