Abstract
Multilevel caching, common in many storage configurations, introduces new challenges to traditional cache management: data must be kept in the appropriate cache and replication avoided across the various cache levels. Additional challenges are introduced when the lower levels of the hierarchy are shared by multiple clients. Sharing can have both positive and negative effects. While data fetched by one client can be used by another client without incurring additional delays, clients competing for cache buffers can evict each other’s blocks and interfere with exclusive caching schemes.
We present a global noncentralized, dynamic and informed management policy for multiple levels of cache, accessed by multiple clients. Our algorithm, MC2, combines local, per client management with a global, system-wide scheme, to emphasize the positive effects of sharing and reduce the negative ones. Our local management scheme, Karma, uses readily available information about the client’s future access profile to save the most valuable blocks, and to choose the best replacement policy for them. The global scheme uses the same information to divide the shared cache space between clients, and to manage this space. Exclusive caching is maintained for nonshared data and is disabled when sharing is identified.
Previous studies have partially addressed these challenges through minor changes to the storage interface. We show that all these challenges can in fact be addressed by combining minor interface changes with smart allocation and replacement policies. We show the superiority of our approach through comparison to existing solutions, including LRU, ARC, MultiQ, LRU-SP, and Demote, as well as a lower bound on optimal I/O response times. Our simulation results demonstrate better cache performance than all other solutions and up to 87% better performance than LRU on representative workloads.
- Aggarwal, A., Alpern, B., Chandra, A., and Snir, M. 1987. A model for hierarchical memory. In Proceedings of the ACM Conference on Theory of Computing. Google Scholar
Digital Library
- Ari, I. 2004. Design and management of globally distributed network caches. Ph.D. thesis, University of California Santa Cruz. Google Scholar
Digital Library
- Arpaci-Dusseau, A. C. and Arpaci-Dusseau, R. H. 2001. Information and control in gray-box systems. In Proceedings of the Symposium on Operating Systems Principles (SOSP). Google Scholar
Digital Library
- Azagury, A., Dreizin, V., Factor, M., Henis, E., Naor, D., Rinetzky, N., Rodeh, O., Satran, J., Tavory, A., and Yerushalmi, L. 2003. Towards an object store. In Proceedings of the NASA Goddard Conference on Mass Storage Systems and Technologies (MSST). Google Scholar
Digital Library
- Baek, S. H. and Park, K. H. 2008. Prefetching with adaptive cache culling for striped disk arrays. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Bansal, S. and Modha, D. S. 2004. CAR: Clock with adaptive replacement. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Barzilai, T. and Golan, G. 2002. Accessing application identification information in the storage tier. Disclosure IL8-2002-0055, IBM Haifa Labs.Google Scholar
- Batsakis, A., Burns, R., Kanevsky, A., Lentini, J., and Talpey, T. 2008. AWOL: an adaptive write optimizations layer. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Belady, L. 1966. A study of replacement algorithms for a virtual-storage computer. IBM Syst. J. 5, 2, 78--101. Google Scholar
Digital Library
- Bhadkamkar, M., Guerra, J., Useche, L., Burnett, S., Liptak, J., Rangaswami, R., and Hristidis, V. 2009. BORG: block-reORGanization for self-optimizing storage systems. In Proceedings of the 7th Conference on File and Stroage Technologies (FAST). Google Scholar
Digital Library
- Cao, P., Felten, E. W., Karlin, A. R., and Li, K. 1996. Implementation and performance of integrated application-controlled file caching, prefetching, and disk scheduling. ACM Trans. Comput. Syst. 14, 4, 311--343. Google Scholar
Digital Library
- Chen, Z., Zhou, Y., and Li, K. 2003. Eviction-based placement for storage caches. In Proceedings of the USENIX Annual Technical Conference.Google Scholar
- Chen, Z., Zhang, Y., Zhou, Y., Scott, H., and Schiefer, B. 2005. Empirical evaluation of multi-level buffer cache collaboration for storage systems. In Proceedings of the ACM SIGMETRICS Conference. Google Scholar
Digital Library
- Choi, J., Noh, S. H., Min, S. L., and Cho, Y. 1999. An implementation study of a detection-based adaptive block replacement scheme. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Choi, J., Noh, S. H., Min, S. L., and Cho, Y. 2000. Towards application/file-level characterization of block references: a case for fine-grained buffer management. In Proceedings of the ACM SIGMETRICS Conference. Google Scholar
Digital Library
- Chou, H.-T. and DeWitt, D. J. 1985. An evaluation of buffer management strategies for relational database systems. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- Chrobak, M. and Noga, J. 2000. Competitive algorithms for relaxed list update and multilevel caching. J. Algor. 34, 2, 282--308. Google Scholar
Digital Library
- Dahlin, M. D., Wang, R. Y., Anderson, T. E., and Patterson, D. A. 1994. Cooperative caching: using remote client memory to improve file system performance. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Ding, X., Jiang, S., Chen, F., Davis, K., and Zhang, X. 2007. DiskSeen: Exploiting disk layout and access history to enhance i/o prefetch. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Gabor, R., Weiss, S., and Mendelson, A. 2007. Fairness enforcement in switch on event multithreading. ACM Trans. Architec. Code Optim. 4, 3, 15. Google Scholar
Digital Library
- Ganger, G. R., Ellard, D., and Seltzer, M. I. 2004. File classification in self-∗ storage systems. In Proceedings of the International Conference on Autonomic Computing (ICAC). Google Scholar
Digital Library
- Gill, B. 2008. On multi-level exclusive caching: Offline optimality and why promotions are better than demotions. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Gill, B. S. and Modha, D. S. 2005a. SARC: Sequential prefetching in adaptive replacement cache. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Gill, B. S. and Modha, D. S. 2005b. WOW: Wise ordering for writes-combining spatial and temporal locality in non-volatile caches. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Gill, B. S. and Bathen, L. A. D. 2007. AMP: adaptive multi-stream prefetching in a shared cache. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Gniady, C., Butt, A. R., and Hu., Y. C. 2004. Program counter based pattern classification in buffer caching. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Gramacy, R. B., Warmuth, M. K., Brandt, S. A., and Ari, I. 2002. Adaptive caching by refetching. In Neural Information Processing Systems (NIPS).Google Scholar
- Harizopoulos, S. and Ailamaki, A. 2003. A case for staged database systems. In Proceedings of the Conference on Innovative Data Systems Research (CIDR).Google Scholar
- Jauhari, R., Carey, M. J., and Livny, M. 1990. Priority-hints: An algorithm for priority-based buffer management. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- Jiang, S. and Zhang, X. 2002. LIRS: An efficient low inter-reference recency set replacement policy to improve buffer cache performance. In Proceedings of the ACM SIGMETRICS Conference. Google Scholar
Digital Library
- Jiang, S. and Zhang, X. 2004. ULC: A file block placement and replacement protocol to effectively exploit hierarchical locality in multi-level buffer caches. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Jiang, S., Ding, X., Chen, F., Tan, E., and Zhang, X. 2005. DULO: An effective buffer cache management scheme to exploit both temporal and spatial localities. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Jiang, S., Petrini, F., Ding, X., and Zhang, X. 2006. A locality-aware cooperative cache management protocol to improve network file system performance. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Johnson, T. and Shasha, D. 1994. 2Q: a low overhead high performance buffer management replacement algorithm. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- Karpovich, J. F., Grimshaw, A. S., and French, J. C. 1994. Extensible file system (ELFS): an object-oriented approach to high performance file I/O. In Proceedings of the Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA). Google Scholar
Digital Library
- Kim, J. M., Choi, J., Kim, J., Noh, S. H., Min, S. L., Cho, Y., and Kim, C. S. 2000. A low-overhead high-performance unified buffer management scheme that exploits sequential and looping references. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Lee, D., Choi, J., Kim, J.-H., Noh, S. H., Min, S. L., Cho, Y., and Kim, C.-S. 1999. On the existence of a spectrum of policies that subsumes the least recently used (LRU) and least frequently used (LFU) policies. In Proceedings of the ACM SIGMETRICS Conference. Google Scholar
Digital Library
- Li, X., Aboulnaga, A., Salem, K., Sachedina, A., and Gao, S. 2005. Second-tier cache management using write hints. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Li, Z., Chen, Z., Srinivasan, S. M., and Zhou, Y. 2004. C-miner: Mining block correlations in storage systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Liang, S., Jiang, S., and Zhang, X. 2007. STEP: Sequentiality and thrashing detection based prefetching to improve performance of networked storage servers. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Liu, X., Aboulnaga, A., Salem, K., and Li, X. 2009. CLIC: client-informed caching for storage servers. In Proceedings of the 7th Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Llanos, D. R. and Palop, B. 2006. An open-source TPC-C implementation for parallel and distributed systems. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS). Google Scholar
Digital Library
- Lumb, C. R., Schindler, J., and Ganger, G. R. 2002. Freeblock scheduling outside of disk firmware. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Megiddo, N. and Modha, D. S. 2003. ARC: A self-tuning, low overhead replacement cache. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Momjian, B. 2000. PostgreSQL: Introduction and Concepts. Addison-Wesley. Google Scholar
Digital Library
- Muntz, D. and Honeyman, P. 1992. Multi-level caching in distributed file systems - or - your cache ain’t nuthin’ but trash. In Proceedings of the USENIX Winter Conference.Google Scholar
- Ng, R., Faloutsos, C., and Sellis, T. 1991. Flexible buffer allocation based on marginal gains. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google Scholar
Digital Library
- O’Neil, E. J., O’Neil, P. E., and Weikum, G. 1993. The LRU-K page replacement algorithm for database disk buffering. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google Scholar
Digital Library
- Ou, L., He, X., Kosa, M. J., and Scott, S. L. 2005. A unified multiple-level cache for high performance storage systems. In Proceedings of the IEEE International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS). Google Scholar
Digital Library
- Patterson, R. H., Gibson, G. A., Ginting, E., Stodolsky, D., and Zelenka, J. 2001. Informed prefetching and caching. In Proceedings of the High Performance Mass Storage and Parallel I/O: Technologies and Applications, H. Jin, T. Cortes, and R. Buyya Eds., IEEE Computer Society Press and Wiley, NY, 224--244.Google Scholar
- Robinson, J. T. and Devarakonda, M. V. 1990. Data cache management using frequency-based replacement. In Proceedings of the ACM SIGMETRICS Conference. Google Scholar
Digital Library
- Sacco, G. M. and Schkolnick, M. 1982. A mechanism for managing the buffer pool in a relational database system using the hot set model. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- Sacco, G. M. and Schkolnick, M. 1986. Buffer management in relational database systems. ACM Trans. Datab. Syst. 11, 4, 473--498. Google Scholar
Digital Library
- Sarkar, P. and Hartman, J. 1996. Efficient cooperative caching using hints. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Schindler, J., Griffin, J. L., Lumb, C. R., and Ganger, G. R. 2002. Track-aligned extents: Matching access patterns to disk drive characteristics. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Schindler, J., Ailamaki, A., and Ganger., G. R. 2003. Lachesis: Robust database storage management based on device-specific performance characteristics. In Proceedings of the International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- Selinger, P. G., Astrahan, M. M., Chamberlin, D. D., Lorie, R. A., and Price, T. G. 1979. Access path selection in a relational database management system. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google Scholar
Digital Library
- Sivathanu, M., Prabhakaran, V., Popovici, F. I., Denehy, T. E., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2003. Semantically-smart disk systems. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Sivathanu, M., Bairavasundaram, L. N., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005. Database-aware semantically-smart storage. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Soundararajan, G., Mihailescu, M., and Amza, C. 2008. Context-aware prefetching at the storage server. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Soundararajan, G., Lupei, D., Ghanbari, S., Popescu, A. D., Chen, J., and Amza, C. 2009. Dynamic resource allocation for database servers running on virtual storage. In Proceedings of the 7th Conference on File and Stroage Technologies (FAST). Google Scholar
Digital Library
- Stonebraker, M. 1981. Operating system support for database management. Comm. ACM 24, 7, 412--418. Google Scholar
Digital Library
- Storm, A. J., Garcia-Arellano, C., Lightstone, S. S., Diao, Y., and Surendra, M. 2006. Adaptive self-tuning memory in DB2. In Proceedings of the 32nd International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- Takagi, M. and Hiraki, K. 2004. Inter-reference gap distribution replacement: an improved replacement algorithm for set-associative caches. In Proceedings of the International Conference on Supercomputing (ICS). Google Scholar
Digital Library
- Voelker, G. M., Anderson, E. J., Kimbrel, T., Feeley, M. J., Chase, J. S., Karlin, A. R., and Levy, H. M. 1998. Implementing cooperative prefetching and caching in a globally-managed memory system. In Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems. Google Scholar
Digital Library
- Wachs, M., Abd-El-Malek, M., Thereska, E., and Ganger, G. R. 2007. Argon: Performance insulation for shared storage servers. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Wong, T. M. and Wilkes, J. 2002. My cache or yours? Making storage more exclusive. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Yadgar, G., Factor, M., and Schuster, A. 2007. Karma: Know-It-All Replacement for a Multilevel Cache. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST). Google Scholar
Digital Library
- Yadgar, G., Factor, M., Li, K., and Schuster, A. 2008. MC<sup>2</sup>: Multiple Clients on A Multilevel Cache. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Zhang, Z., Lee, K., Ma, X., and Zhou, Y. 2008. PFC: Transparent optimization of existing prefetching strategies for multi-level storage systems. In Proceedings of the 28th International Conference on Distributed Computing Systems (ICDCS). Google Scholar
Digital Library
- Zhang, Z., Kulkarni, A., Ma, X., and Zhou, Y. 2009. Memory resource allocation for file system prefetching: From a supply chain management perspective. In Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys). Google Scholar
Digital Library
- Zhou, P., Pandey, V., Sundaresan, J., Raghuraman, A., Zhou, Y., and Kumar, S. 2004a. Dynamic tracking of page miss ratio curve for memory management. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). Google Scholar
Digital Library
- Zhou, Y., Philbin, J., and Li, K. 2001. The multi-queue replacement algorithm for second level buffer caches. In Proceedings of the USENIX Annual Technical Conference. Google Scholar
Digital Library
- Zhou, Y., Chen, Z., and Li, K. 2004b. Second-level buffer cache management. IEEE Trans. Paral. Distrib. Syst. 15, 6, 505--519. Google Scholar
Digital Library
Index Terms
Management of Multilevel, Multiclient Cache Hierarchies with Application Hints
Recommendations
Hint-K: An Efficient Multi-level Cache Using K-Step Hints
ICPP '10: Proceedings of the 2010 39th International Conference on Parallel ProcessingI/O performance has been critical for large scale distributed systems. Many approaches, including hint-based multi-level cache, have been proposed to smooth the gap between different levels. These solutions demote or promote cache blocks based on the ...
Decoupled Cache Segmentation: Mutable Policy with Automated Bypass
PACT '11: Proceedings of the 2011 International Conference on Parallel Architectures and Compilation TechniquesThe least recently used (LRU) replacement policy performs poorly in the last-level cache (LLC) because temporal locality of memory accesses is filtered by first and second level caches. We propose a cache segmentation technique that adapts to cache ...






Comments