Abstract
Consolidated server racks are quickly becoming the standard infrastructure for engineering, business, medicine, and science. Such servers are still designed much in the way when they were organized as individual, distributed systems. Given that many fields rely on big-data analytics substantially, its cost-effectiveness and performance should be improved, which can be achieved by flexibly allowing resources to be shared across nodes. Here we describe Venice, a family of data-center server architectures that includes a strong communication substrate as a first-class resource. Venice supports a diverse set of resource-joining mechanisms that enables applications to leverage non-local resources efficiently.
We have constructed a hardware prototype to better understand the implications of design decisions about system support for resource sharing. We use it to measure the performance of at-scale applications and to explore performance, power, and resource-sharing transparency tradeoffs (i.e., how many programming changes are needed). We analyze these tradeoffs for sharing memory, accelerators, and NICs. We find that reducing/hiding latency is particularly important, the chosen communication channels should match the sharing access patterns of the applications, and of which we can improve performance by exploiting inter-channel collaboration.
- 2014. HUAWEI DC3.0. Retrieved on December 20, 2018 from http://www.huawei.com/ilink/en/download/HW_349607.Google Scholar
- 2014. Zynq®-7000 All Programmable SoC. Retrieved on December 20, 2018 from www.xilinx.com/products/silicon-devices/soc/zynq-7000.html.Google Scholar
- 2017. OpenCAPI Consortium. Retrieved on December 20, 2018 from https://opencapi.org/.Google Scholar
- 2018. CCIX Consortium. Retrieved on December 20, 2018 from https://www.ccixconsortium.com/.Google Scholar
- 2018. Gen-Z Consortium. Retrieved on December 20, 2018 from http://genzconsortium.org/.Google Scholar
- 2018. Infiniband Performance Benchmarks. Retrieved on December 20, 2018 from http://www.mellanox.com/page/performance_infiniband.Google Scholar
- Y. Ajima, Y. Takagi, T. Inoue, S. Hiramoto, and T. Shimizu. 2011. The tofu interconnect. In Proc. IEEE Annual Symposium on High Performance Interconnects. 87--94. Google Scholar
Digital Library
- C. Amza, A. L. Cox, S. Dwarkadas, P. Keleher, Honghui Lu, R. Rajamony, W. Yu, and W. Zwaenepoel. 1996. TreadMarks: Shared memory computing on networks of workstations. Computer 29, 2 (Feb. 1996), 18--28. Google Scholar
Digital Library
- E. Anderson, J. Brooks, C. Grassl, and S. Scott. 1997. Performance of the CRAY T3E multiprocessor. In Proc. ACM/IEEE International Conference on Supercomputing. 39--39. Google Scholar
Digital Library
- B. Arimilli, R. Arimilli, V. Chung, S. Clark, W. Denzel, B. Drerup, T. Hoefler, J. Joyner, J. Lewis, J. Li, N. Ni, and R. Rajamony. 2010. The PERCS high-performance interconnect. In Proc. IEEE Annual Symposium on High Performance Interconnects. 75--82. Google Scholar
Digital Library
- T. Benson, A. Akella, and D. A. Maltz. 2010. Network traffic characteristics of data centers in the wild. In Proc. ACM SIGCOMM Conference on Internet Measurement. 267--280. Google Scholar
Digital Library
- N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, A. Saidi, A. Basu, J. Hestness, D. R. Hower, T. Krishna, S. Sardashti, R. Sen, K. Sewell, M. Shoaib, N. Vaish, M. D. Hill, and D. A. Wood. 2011. The gem5 simulator. SIGARCH Computer Architecture News 39, 2 (May 2011), 1--7. Google Scholar
Digital Library
- Marco Ceriani, Simone Secchi, Oreste Villa, Antonino Tumeo, and Gianluca Palermo. 2017. Exploring Efficient Hardware Support for Applications with Irregular Memory Patterns on Multinode Manycore Architectures. IEEE Transactions on Parallel and Distributed Systems 28, 6 (2017), 1635–1648. Google Scholar
Digital Library
- Yisong Chang, Ke Zhang, Sally A. McKee, Lixin Zhang, Mingyu Chen, Liqiang Ren, and Zhiwei Xu. 2016. Extending on-chip interconnects for rack-level remote resource access. In Proc. 2016 IEEE 34th International Conference on Computer Design (ICCD’16). IEEE, 56--63.Google Scholar
Cross Ref
- Michael D. Dahlin, Randolph Y. Wang, Thomas E. Anderson, and David A. Patterson. 1994. Cooperative caching: Using remote client memory to improve file system performance. In Proc. USENIX Conference on Operating Systems Design and Implementation. 19. Google Scholar
Digital Library
- Oracle Berkeley DB. 2017. Retrieved on December 20, 2018 from http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html.Google Scholar
- J. Dean and S. Ghemawat. 2008. MapReduce: Simplified data processing on large clusters. Commun ications of the ACM 51, 1 (Jan. 2008), 107--113. Google Scholar
Digital Library
- M. J. Feeley, W. E. Morgan, E. P. Pighin, A. R. Karlin, H. M. Levy, and C. A. Thekkath. 1995. Implementing global memory management in a workstation cluster. In Proc. ACM Symposium on Operating Systems Principles. 201--212. Google Scholar
Digital Library
- Andrew V. Goldberg. 1997. An efficient implementation of a scaling minimum-cost flow algorithm. Journal of Algorithms 22, 1 (Jan. 1997), 1--29. Google Scholar
Digital Library
- Graph500. 2016. Retrieved on December 20, 2018 from http://www.graph500.org/.Google Scholar
- Juncheng Gu, Youngmoon Lee, Yiwen Zhang, Mosharaf Chowdhury, and Kang G. Shin. 2017. Efficient memory disaggregation with infiniswap. In Proc. NSDI. 649--667. Google Scholar
Digital Library
- M. R. Hines, M. Lewandowski, and K. Gopalan. 2005. Anemone: Adaptive network memory engine. In Proc. ACM Symposium on Operating Systems Principles. 1. Google Scholar
Digital Library
- Rui Hou, Tao Jiang, Liuhang Zhang, Pengfei Qi, Jianbo Dong, Haibin Wang, Xiongli Gu, and Shujie Zhang. 2013. Cost effective data center servers. In Proc. IEEE International Symposium on High Performance Computer Architecture. 179--187. Google Scholar
Digital Library
- Iperf. 2014. Retrieved on December 20, 2018 from http://iperf.fr/.Google Scholar
- H. Jin, X.-H. Sun, Y. Chen, and T. Ke. 2010. REMEM: Remote memory as checkpointing storage. In Proc. IEEE International Conference on Cloud Computing Technology and Science. 319--326. Google Scholar
Digital Library
- M. J. Kumar. 2013. Rack scale architecture for cloud. In Intel Developer Forum.Google Scholar
- J. Laudon and D. Lenoski. 1997. The SGI origin: A ccNUMA highly scalable server. In Proc. ACM International Symposium on Computer Architecture. 241--251. Google Scholar
Digital Library
- D. Lenoski, J. Laudon, K. Gharachorloo, W.-D. Weber, A. Gupta, J. Hennessy, M. Horowitz, and M. S. Lam. 1992. The Stanford Dash multiprocessor. IEEE Computer 25, 3 (March 1992), 63--79. Google Scholar
Digital Library
- K. Li and P. Hudak. 1989. Memory coherence in shared virtual memory systems. ACM Transactions on Computer Systems 7, 4 (Nov. 1989), 321--359. Google Scholar
Digital Library
- K. Lim, J. Chang, T. Mudge, P. Ranganathan, S. K. Reinhardt, and T. F. Wenisch. 2009. Disaggregated memory for expansion and sharing in blade servers. In Proc. ACM International Symposium on Computer Architecture. 267--278. Google Scholar
Digital Library
- K. Lim, P. Ranganathan, Jichuan Chang, C. Patel, T. Mudge, and S. Reinhardt. 2008. Understanding and designing new server architectures for emerging warehouse-computing environments. In Proc. ACM International Symposium on Computer Architecture. 315--326. Google Scholar
Digital Library
- Kevin Lim, Yoshio Turner, Jose Renato Santos, Alvin AuYoung, Jichuan Chang, Parthasarathy Ranganathan, and Thomas F. Wenisch. 2012. System-level implications of disaggregated memory. In IEEE International Symposium on High-Performance Comp Architecture. IEEE, 1--12. Google Scholar
Digital Library
- David Mayhew and Venkata Krishnan. 2003. PCI express and advanced switching: Evolutionary path to building next generation interconnects. In Proc. Symposium on High Performance Interconnects. 21--29.Google Scholar
Cross Ref
- Timothy Prickett Morgan. 2014. On-Chip Networking May Survive Calxeda Shutdown. Retrieved January 2014 from http://www.enterprisetech.com/2014/01/02/chip-networking-may-survive-calxeda-shutdown.Google Scholar
- Michael Nelson, Beng-Hong Lim, and Greg Hutchins. 2005. Fast transparent migration for virtual machines. In Proc. USENIX Annual Technical Conference. 391--394. Google Scholar
Digital Library
- S. Novakovic, A. Daglis, E. Bugnion, B. Falsafi, and B. Grot. 2014. Scale-out NUMA. In Proc. ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 3--18. Google Scholar
Digital Library
- J. Oleszkiewicz, L. Xiao, and Y. Liu. 2004. Parallel network RAM: Effectively utilizing global cluster memory for large data-intensive parallel programs. In Proc. International Conference on Parallel Processing, Vol. 1. 353--360. Google Scholar
Digital Library
- Oracle Corp. 2014. MySQL: The World’s Most Popular Open-Source Database. Retrieved from http://www.mysql.com.Google Scholar
- A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. Gopal, J. Gray, M. Haselman, S Hauck, S. Heil, A. Hormati, J.-Y. Kim, S. Lanka, J. Larus, E. Peterson, S. Pope, A. Smith, J. Thong, P. Xiao, and D. Burger. 2014. A reconfigurable fabric for accelerating large-scale datacenter services. In Proc. ACM International Symposium on Computer Architecuture. 13--24. Google Scholar
Digital Library
- Anil Rao. 2012. AMD | SeaMicro Technology Overview. Retrieved October 10, 2018 from http://www.seamicro.com/sites/default/files/SM_TO01_64_v2.7.pdf.Google Scholar
- J. Regula. 2013. Integrating rack level connectivity into a PCI express switch. In Proc. Hot Chips: A Symposium on High Performance Chips. 259--266.Google Scholar
- ScaleMP. 2011. Versatile SMP (vSMP) Architecture. Retrieved October 10, 2018 from http://www.scalemp.com/technology/versatile-smp-vsmp-architecture/.Google Scholar
- T. Sherwood, E. Perelman, G. Hamerly, and B. Calder. 2002. Automatically characterizing large scale program behavior. In Proc. International Conference on Architectural Support for Programming Languages and Operating Systems. 319--326. Google Scholar
Digital Library
- L. Wang, J. Zhan, C. Luo, Y. Zhu, Q. Yang, Y. He, W. Gao, Z. Jia, Y. Shi, S. Zhang, C. Zheng, G. Lu, K. Zhan, X. Li, and B. Qiu. 2014. BigDataBench: A big data benchmark suite from internet services. In Proc. IEEE International Symposium On High Performance Computer Architecture. 488--499.Google Scholar
- Wiki. 2017. Intel Xeon Microprocessors. Retrieved October 10, 2018 from http://en.wikipedia.org/wiki/List_of_Intel_Xeon_microprocessors#Haswell-based_Xeons.Google Scholar
- Steven Cameron Woo, Moriyoshi Ohara, Evan Torrie, Jaswinder Pal Singh, and Anoop Gupta. 1995. The SPLASH-2 programs: Characterization and methodological considerations. In Proc. ACM International Symposium on Computer Architecture. 24--36. Google Scholar
Digital Library
- M. Xie, Y. Lu, K. Wang, L. Liu, H. Cao, and X. Yang. 2012. Tianhe-1A interconnect and message-passing services. IEEE Micro 32, 1 (Jan. 2012), 8--20. Google Scholar
Digital Library
- Di Xu, Chenggang Wu, and Pen-Chung Yew. 2010. On mitigating memory bandwidth contention through bandwidth-aware scheduling. In Proc. IEEE/ACM/IFIP International Conference on Parallel Architectures and Compilation Techniques. 237--248. Google Scholar
Digital Library
- M. Zaharia, M. F. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. 2012. Spark: Cluster computing with working sets. In Proc. USENIX Conference on Hot Topics in Cloud Computing. 10. Google Scholar
Digital Library
- E. W. Felten and J. Zahorjan. 1991. Issues in the Implementation of a Remote Memory Paging System. Technical Report 91-03-09, University of Washington, Department of Computer Science and Engineering.Google Scholar
- J. Zawodny. 2009. Redis: Lightweight key/value store that goes the extra mile. Linux Magazine 79 (Aug. 2009).Google Scholar
Index Terms
Venice: An Effective Resource Sharing Architecture for Data Center Servers
Recommendations
Transparently bridging semantic gap in CPU management for virtualized environments
Consolidated environments are progressively accommodating diverse and unpredictable workloads in conjunction with virtual desktop infrastructure and cloud computing. Unpredictable workloads, however, aggravate the semantic gap between the virtual ...
Power and Performance Modeling in a Virtualized Server System
ICPPW '10: Proceedings of the 2010 39th International Conference on Parallel Processing WorkshopsVirtualization has become a very important technology which has been adopted in many enterprise computing systems and data centers. Virtualization makes resource management and maintenance easier, and can decrease energy consumption through resource ...
Application Performance Isolation in Virtualization
CLOUD '09: Proceedings of the 2009 IEEE International Conference on Cloud ComputingModern data centers use virtual machine based implementation for numerous advantages like resource isolation, hardware utilization, security and easy management. Applications are generally hosted on different virtual machines on a same physical machine. ...






Comments