Abstract
Memory disaggregation is a promising technique in datacenters with the benefit of improving resource utilization, failure isolation, and elasticity. Hashing indexes have been widely used to provide fast lookup services in distributed memory systems. However, traditional hashing indexes become inefficient for disaggregated memory, since the computing power in the memory pool is too weak to execute complex index requests. To provide efficient indexing services in disaggregated memory scenarios, this article proposes RACE hashing, a one-sided RDMA-Conscious Extendible hashing index with lock-free remote concurrency control and efficient remote resizing. RACE hashing enables all index operations to be efficiently executed by using only one-sided RDMA verbs without involving any compute resource in the memory pool. To support remote concurrent access with high performance, RACE hashing leverages a lock-free remote concurrency control scheme to enable different clients to concurrently operate the same hashing index in the memory pool in a lock-free manner. To resize the hash table with low overheads, RACE hashing leverages an extendible remote resizing scheme to reduce extra RDMA accesses caused by extendible resizing and allow concurrent request execution during resizing. Extensive experimental results demonstrate that RACE hashing outperforms state-of-the-art distributed in-memory hashing indexes by 1.4–13.7× in YCSB hybrid workloads.
- [1] 2022. Gen-Z Technology. Retrieved from https://genzconsortium.org/.Google Scholar
- [2] 2022. Memcached—A Distributed Memory Object Caching System. Retrieved from https://memcached.org/.Google Scholar
- [3] 2022. Redis. Retrieved from https://redis.io/.Google Scholar
- [4] . 2018. Remote regions: A simple abstraction for remote memory. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’18). 775–787.Google Scholar
- [5] . 2019. Designing far memory data structures: Think outside the box. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS’19). 120–126.Google Scholar
Digital Library
- [6] . 2014. FireBox: A hardware building block for 2020 warehouse-scale computers. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST’14).Google Scholar
- [7] . 2012. Workload analysis of a large-scale key-value store. In Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS’12). 53–64.Google Scholar
Digital Library
- [8] . 2015. Intel Omni-path architecture: Enabling scalable, high performance fabrics. In Proceedings of the IEEE 23rd Annual Symposium on High-Performance Interconnects (HOTI’15). IEEE, 1–9.Google Scholar
Digital Library
- [9] . 2017. Tolerating faults in disaggregated datacenters. In Proceedings of the 16th ACM Workshop on Hot Topics in Networks (HotNets’17). 164–170.Google Scholar
Digital Library
- [10] . 2020. Lock-free concurrent level hashing for persistent memory. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’20). 799–812.Google Scholar
- [11] . 2010. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (SoCC’10). 143–154.Google Scholar
Digital Library
- [12] . 2015. R2C2: A network stack for rack-scale computers. In Proceedings of the ACM Conference on Special Interest Group on Data Communication (SIGCOMM’15). 551–564.Google Scholar
Digital Library
- [13] Ana Lúcia De Moura and Roberto Ierusalimschy. 2009. Revisiting coroutines. ACM Trans. Program. Lang. Syst. 31, 2, Article 6 (February 2009), 31 pages.Google Scholar
- [14] . 2014. FaRM: Fast remote memory. In Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI’14). 401–414.Google Scholar
- [15] . 1979. Extendible hashing-a fast access method for dynamic files. ACM Trans. Database Syst. 4, 3 (1979), 315–344.Google Scholar
Digital Library
- [16] . 2013. Memc3: Compact and concurrent memcache with dumber caching and smarter hashing. In Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI’13). 371–384.Google Scholar
- [17] . 2015. Beyond processor-centric operating systems. In Proceedings of the 15th Workshop on Hot Topics in Operating Systems (HotOS’15).Google Scholar
- [18] . 1997. Transactional client-server cache consistency: Alternatives and performance. ACM Trans. Database Syst. 22, 3 (1997), 315–363.Google Scholar
Digital Library
- [19] . 2016. Network requirements for resource disaggregation. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’16). 249–264.Google Scholar
- [20] . 2020. CoroBase: Coroutine-oriented main-memory database engine. arXiv: 2010.15981. Retrieved from https://arxiv.org/abs/2010.15981.Google Scholar
- [21] . 2008. Hopscotch hashing. In Proceedings of the International Symposium on Distributed Computing (DISC’08). Springer, 350–364.Google Scholar
Digital Library
- [22] . 2015. The Machine: A New Kind of Computer. Retrieved from https://www.hpl.hp.com/research/systems-research/themachine/.Google Scholar
- [23] . 2022. Intel Rack Scale Design Architecture. Retrieved from https://www.intel.com/content/www/us/en/architecture-and-technology/rack-scale-design-overview.html.Google Scholar
- [24] . 2016. Design guidelines for high performance RDMA systems. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’16). 437–450.Google Scholar
- [25] . 2013. Meet the walkers: Accelerating index traversals for in-memory databases. In Proceedings of the 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’15). 468–479.Google Scholar
Digital Library
- [26] . 2017. KV-Direct: High-performance in-memory key-value store with programmable NIC. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP’17). 137–152.Google Scholar
Digital Library
- [27] . 2015. Architecting to achieve a billion requests per second throughput on a single key-value store server platform. In Proceedings of the 42nd Annual International Symposium on Computer Architecture (ISCA’15). 476–488.Google Scholar
Digital Library
- [28] . 2014. Algorithmic improvements for fast concurrent cuckoo hashing. In Proceedings of the 9th European Conference on Computer Systems (EuroSys’14). 1–14.Google Scholar
Digital Library
- [29] . 2017. Cicada: Dependably fast multi-core in-memory transactions. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD’17). 21–35.Google Scholar
Digital Library
- [30] . 2009. Disaggregated memory for expansion and sharing in blade servers. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). 267–278.Google Scholar
Digital Library
- [31] . 2012. System-level implications of disaggregated memory. In Proceedings of the IEEE International Symposium on High-Performance Comp Architecture (HPCA’12). IEEE, 1–12.Google Scholar
Digital Library
- [32] . 2012. Cache craftiness for fast multicore key-value storage. In Proceedings of the 7th ACM European Conference on Computer Systems (EuroSys’12). 183–196.Google Scholar
Digital Library
- [33] . 2013. Using one-sided RDMA reads to build a fast, CPU-efficient key-value store. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’15). 103–114.Google Scholar
- [34] . 2001. The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Syst. 12, 10 (2001), 1094–1104.Google Scholar
Digital Library
- [35] . 2019. Write-optimized dynamic hashing for persistent memory. In Proceedings of the 17th USENIX Conference on File and Storage Technologies (FAST’19). 31–44.Google Scholar
Digital Library
- [36] . 2014. Lock-free cuckoo hashing. In Proceedings of the IEEE 34th International Conference on Distributed Computing Systems (ICDCS’14). IEEE, 627–636.Google Scholar
Digital Library
- [37] . 2013. Scaling memcache at facebook. In Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI’13). 385–398.Google Scholar
- [38] . 2004. Cuckoo hashing. J. Algor. 51, 2 (2004), 122–144.Google Scholar
Digital Library
- [39] . 2020. AIFM: High-performance, application-integrated far memory. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI’20). 315–332.Google Scholar
- [40] . 2018. LegoOS: A disseminated, distributed OS for hardware resource disaggregation. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI’18). 69–87.Google Scholar
- [41] . 2019. Shoal: A network architecture for disaggregated racks. In Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI’19). 255–270.Google Scholar
- [42] . 2020. Disaggregating persistent memory and controlling them remotely: An exploration of passive disaggregated key-value stores. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC’20). 33–48.Google Scholar
- [43] . 2017. Lite kernel rdma support for datacenter applications. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP’17). 306–324.Google Scholar
Digital Library
- [44] . 2020. Semeru: A memory-disaggregated managed runtime. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI’20). 261–280.Google Scholar
- [45] . 2018. Easy lock-free indexing in non-volatile memory. In Proceedings of the IEEE 34th International Conference on Data Engineering (ICDE’18). IEEE, 461–472.Google Scholar
Cross Ref
- [46] . 2015. Fast in-memory transaction processing using RDMA and HTM. In Proceedings of the 25th Symposium on Operating Systems Principles (SOSP’15). 87–104.Google Scholar
Digital Library
- [47] . 2014. Staring into the abyss: An evaluation of concurrency control with one thousand Cores. Proc. VLDB Endow. 8, 3 (
Nov. 2014), 209–220.Google ScholarDigital Library
- [48] . 2018. Write-optimized and high-performance hashing index scheme for persistent memory. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI’18). 461–476.Google Scholar
Index Terms
RACE: One-sided RDMA-conscious Extendible Hashing
Recommendations
Level Hashing: A High-performance and Flexible-resizing Persistent Hashing Index Structure
Systor 2018 Special Section on ATC 2018, Special Section on OSDI 2018 and Regular PapersNon-volatile memory (NVM) technologies as persistent memory are promising candidates to complement or replace DRAM for building future memory systems, due to having the advantages of high density, low power, and non-volatility. In main memory systems, ...
Fast In-Memory Transaction Processing Using RDMA and HTM
DrTM is a fast in-memory transaction processing system that exploits advanced hardware features such as remote direct memory access (RDMA) and hardware transactional memory (HTM). To achieve high efficiency, it mostly offloads concurrency control such ...
Pea Hash: A Performant Extendible Adaptive Hashing Index
PACMMODHashing index is widely used to support efficient point operations. We observe that there is a conflict between performance and memory utilization goals. Existing hashing indices often have to trade off hash table access latency for better memory ...






Comments