skip to main content
research-article

Venice: An Effective Resource Sharing Architecture for Data Center Servers

Authors Info & Claims
Published:14 March 2019Publication History
Skip Abstract Section

Abstract

Consolidated server racks are quickly becoming the standard infrastructure for engineering, business, medicine, and science. Such servers are still designed much in the way when they were organized as individual, distributed systems. Given that many fields rely on big-data analytics substantially, its cost-effectiveness and performance should be improved, which can be achieved by flexibly allowing resources to be shared across nodes. Here we describe Venice, a family of data-center server architectures that includes a strong communication substrate as a first-class resource. Venice supports a diverse set of resource-joining mechanisms that enables applications to leverage non-local resources efficiently.

We have constructed a hardware prototype to better understand the implications of design decisions about system support for resource sharing. We use it to measure the performance of at-scale applications and to explore performance, power, and resource-sharing transparency tradeoffs (i.e., how many programming changes are needed). We analyze these tradeoffs for sharing memory, accelerators, and NICs. We find that reducing/hiding latency is particularly important, the chosen communication channels should match the sharing access patterns of the applications, and of which we can improve performance by exploiting inter-channel collaboration.

References

  1. 2014. HUAWEI DC3.0. Retrieved on December 20, 2018 from http://www.huawei.com/ilink/en/download/HW_349607.Google ScholarGoogle Scholar
  2. 2014. Zynq®-7000 All Programmable SoC. Retrieved on December 20, 2018 from www.xilinx.com/products/silicon-devices/soc/zynq-7000.html.Google ScholarGoogle Scholar
  3. 2017. OpenCAPI Consortium. Retrieved on December 20, 2018 from https://opencapi.org/.Google ScholarGoogle Scholar
  4. 2018. CCIX Consortium. Retrieved on December 20, 2018 from https://www.ccixconsortium.com/.Google ScholarGoogle Scholar
  5. 2018. Gen-Z Consortium. Retrieved on December 20, 2018 from http://genzconsortium.org/.Google ScholarGoogle Scholar
  6. 2018. Infiniband Performance Benchmarks. Retrieved on December 20, 2018 from http://www.mellanox.com/page/performance_infiniband.Google ScholarGoogle Scholar
  7. Y. Ajima, Y. Takagi, T. Inoue, S. Hiramoto, and T. Shimizu. 2011. The tofu interconnect. In Proc. IEEE Annual Symposium on High Performance Interconnects. 87--94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. C. Amza, A. L. Cox, S. Dwarkadas, P. Keleher, Honghui Lu, R. Rajamony, W. Yu, and W. Zwaenepoel. 1996. TreadMarks: Shared memory computing on networks of workstations. Computer 29, 2 (Feb. 1996), 18--28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. E. Anderson, J. Brooks, C. Grassl, and S. Scott. 1997. Performance of the CRAY T3E multiprocessor. In Proc. ACM/IEEE International Conference on Supercomputing. 39--39. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. B. Arimilli, R. Arimilli, V. Chung, S. Clark, W. Denzel, B. Drerup, T. Hoefler, J. Joyner, J. Lewis, J. Li, N. Ni, and R. Rajamony. 2010. The PERCS high-performance interconnect. In Proc. IEEE Annual Symposium on High Performance Interconnects. 75--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. T. Benson, A. Akella, and D. A. Maltz. 2010. Network traffic characteristics of data centers in the wild. In Proc. ACM SIGCOMM Conference on Internet Measurement. 267--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, A. Saidi, A. Basu, J. Hestness, D. R. Hower, T. Krishna, S. Sardashti, R. Sen, K. Sewell, M. Shoaib, N. Vaish, M. D. Hill, and D. A. Wood. 2011. The gem5 simulator. SIGARCH Computer Architecture News 39, 2 (May 2011), 1--7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Marco Ceriani, Simone Secchi, Oreste Villa, Antonino Tumeo, and Gianluca Palermo. 2017. Exploring Efficient Hardware Support for Applications with Irregular Memory Patterns on Multinode Manycore Architectures. IEEE Transactions on Parallel and Distributed Systems 28, 6 (2017), 1635–1648. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Yisong Chang, Ke Zhang, Sally A. McKee, Lixin Zhang, Mingyu Chen, Liqiang Ren, and Zhiwei Xu. 2016. Extending on-chip interconnects for rack-level remote resource access. In Proc. 2016 IEEE 34th International Conference on Computer Design (ICCD’16). IEEE, 56--63.Google ScholarGoogle ScholarCross RefCross Ref
  15. Michael D. Dahlin, Randolph Y. Wang, Thomas E. Anderson, and David A. Patterson. 1994. Cooperative caching: Using remote client memory to improve file system performance. In Proc. USENIX Conference on Operating Systems Design and Implementation. 19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Oracle Berkeley DB. 2017. Retrieved on December 20, 2018 from http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html.Google ScholarGoogle Scholar
  17. J. Dean and S. Ghemawat. 2008. MapReduce: Simplified data processing on large clusters. Commun ications of the ACM 51, 1 (Jan. 2008), 107--113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. M. J. Feeley, W. E. Morgan, E. P. Pighin, A. R. Karlin, H. M. Levy, and C. A. Thekkath. 1995. Implementing global memory management in a workstation cluster. In Proc. ACM Symposium on Operating Systems Principles. 201--212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Andrew V. Goldberg. 1997. An efficient implementation of a scaling minimum-cost flow algorithm. Journal of Algorithms 22, 1 (Jan. 1997), 1--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Graph500. 2016. Retrieved on December 20, 2018 from http://www.graph500.org/.Google ScholarGoogle Scholar
  21. Juncheng Gu, Youngmoon Lee, Yiwen Zhang, Mosharaf Chowdhury, and Kang G. Shin. 2017. Efficient memory disaggregation with infiniswap. In Proc. NSDI. 649--667. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. R. Hines, M. Lewandowski, and K. Gopalan. 2005. Anemone: Adaptive network memory engine. In Proc. ACM Symposium on Operating Systems Principles. 1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Rui Hou, Tao Jiang, Liuhang Zhang, Pengfei Qi, Jianbo Dong, Haibin Wang, Xiongli Gu, and Shujie Zhang. 2013. Cost effective data center servers. In Proc. IEEE International Symposium on High Performance Computer Architecture. 179--187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Iperf. 2014. Retrieved on December 20, 2018 from http://iperf.fr/.Google ScholarGoogle Scholar
  25. H. Jin, X.-H. Sun, Y. Chen, and T. Ke. 2010. REMEM: Remote memory as checkpointing storage. In Proc. IEEE International Conference on Cloud Computing Technology and Science. 319--326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. M. J. Kumar. 2013. Rack scale architecture for cloud. In Intel Developer Forum.Google ScholarGoogle Scholar
  27. J. Laudon and D. Lenoski. 1997. The SGI origin: A ccNUMA highly scalable server. In Proc. ACM International Symposium on Computer Architecture. 241--251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. D. Lenoski, J. Laudon, K. Gharachorloo, W.-D. Weber, A. Gupta, J. Hennessy, M. Horowitz, and M. S. Lam. 1992. The Stanford Dash multiprocessor. IEEE Computer 25, 3 (March 1992), 63--79. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. K. Li and P. Hudak. 1989. Memory coherence in shared virtual memory systems. ACM Transactions on Computer Systems 7, 4 (Nov. 1989), 321--359. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. K. Lim, J. Chang, T. Mudge, P. Ranganathan, S. K. Reinhardt, and T. F. Wenisch. 2009. Disaggregated memory for expansion and sharing in blade servers. In Proc. ACM International Symposium on Computer Architecture. 267--278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. K. Lim, P. Ranganathan, Jichuan Chang, C. Patel, T. Mudge, and S. Reinhardt. 2008. Understanding and designing new server architectures for emerging warehouse-computing environments. In Proc. ACM International Symposium on Computer Architecture. 315--326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kevin Lim, Yoshio Turner, Jose Renato Santos, Alvin AuYoung, Jichuan Chang, Parthasarathy Ranganathan, and Thomas F. Wenisch. 2012. System-level implications of disaggregated memory. In IEEE International Symposium on High-Performance Comp Architecture. IEEE, 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. David Mayhew and Venkata Krishnan. 2003. PCI express and advanced switching: Evolutionary path to building next generation interconnects. In Proc. Symposium on High Performance Interconnects. 21--29.Google ScholarGoogle ScholarCross RefCross Ref
  34. Timothy Prickett Morgan. 2014. On-Chip Networking May Survive Calxeda Shutdown. Retrieved January 2014 from http://www.enterprisetech.com/2014/01/02/chip-networking-may-survive-calxeda-shutdown.Google ScholarGoogle Scholar
  35. Michael Nelson, Beng-Hong Lim, and Greg Hutchins. 2005. Fast transparent migration for virtual machines. In Proc. USENIX Annual Technical Conference. 391--394. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. S. Novakovic, A. Daglis, E. Bugnion, B. Falsafi, and B. Grot. 2014. Scale-out NUMA. In Proc. ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 3--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. J. Oleszkiewicz, L. Xiao, and Y. Liu. 2004. Parallel network RAM: Effectively utilizing global cluster memory for large data-intensive parallel programs. In Proc. International Conference on Parallel Processing, Vol. 1. 353--360. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Oracle Corp. 2014. MySQL: The World’s Most Popular Open-Source Database. Retrieved from http://www.mysql.com.Google ScholarGoogle Scholar
  39. A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. Gopal, J. Gray, M. Haselman, S Hauck, S. Heil, A. Hormati, J.-Y. Kim, S. Lanka, J. Larus, E. Peterson, S. Pope, A. Smith, J. Thong, P. Xiao, and D. Burger. 2014. A reconfigurable fabric for accelerating large-scale datacenter services. In Proc. ACM International Symposium on Computer Architecuture. 13--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Anil Rao. 2012. AMD | SeaMicro Technology Overview. Retrieved October 10, 2018 from http://www.seamicro.com/sites/default/files/SM_TO01_64_v2.7.pdf.Google ScholarGoogle Scholar
  41. J. Regula. 2013. Integrating rack level connectivity into a PCI express switch. In Proc. Hot Chips: A Symposium on High Performance Chips. 259--266.Google ScholarGoogle Scholar
  42. ScaleMP. 2011. Versatile SMP (vSMP) Architecture. Retrieved October 10, 2018 from http://www.scalemp.com/technology/versatile-smp-vsmp-architecture/.Google ScholarGoogle Scholar
  43. T. Sherwood, E. Perelman, G. Hamerly, and B. Calder. 2002. Automatically characterizing large scale program behavior. In Proc. International Conference on Architectural Support for Programming Languages and Operating Systems. 319--326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. L. Wang, J. Zhan, C. Luo, Y. Zhu, Q. Yang, Y. He, W. Gao, Z. Jia, Y. Shi, S. Zhang, C. Zheng, G. Lu, K. Zhan, X. Li, and B. Qiu. 2014. BigDataBench: A big data benchmark suite from internet services. In Proc. IEEE International Symposium On High Performance Computer Architecture. 488--499.Google ScholarGoogle Scholar
  45. Wiki. 2017. Intel Xeon Microprocessors. Retrieved October 10, 2018 from http://en.wikipedia.org/wiki/List_of_Intel_Xeon_microprocessors#Haswell-based_Xeons.Google ScholarGoogle Scholar
  46. Steven Cameron Woo, Moriyoshi Ohara, Evan Torrie, Jaswinder Pal Singh, and Anoop Gupta. 1995. The SPLASH-2 programs: Characterization and methodological considerations. In Proc. ACM International Symposium on Computer Architecture. 24--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. M. Xie, Y. Lu, K. Wang, L. Liu, H. Cao, and X. Yang. 2012. Tianhe-1A interconnect and message-passing services. IEEE Micro 32, 1 (Jan. 2012), 8--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Di Xu, Chenggang Wu, and Pen-Chung Yew. 2010. On mitigating memory bandwidth contention through bandwidth-aware scheduling. In Proc. IEEE/ACM/IFIP International Conference on Parallel Architectures and Compilation Techniques. 237--248. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. M. Zaharia, M. F. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. 2012. Spark: Cluster computing with working sets. In Proc. USENIX Conference on Hot Topics in Cloud Computing. 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. E. W. Felten and J. Zahorjan. 1991. Issues in the Implementation of a Remote Memory Paging System. Technical Report 91-03-09, University of Washington, Department of Computer Science and Engineering.Google ScholarGoogle Scholar
  51. J. Zawodny. 2009. Redis: Lightweight key/value store that goes the extra mile. Linux Magazine 79 (Aug. 2009).Google ScholarGoogle Scholar

Index Terms

  1. Venice: An Effective Resource Sharing Architecture for Data Center Servers

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Computer Systems
      ACM Transactions on Computer Systems  Volume 36, Issue 1
      February 2018
      222 pages
      ISSN:0734-2071
      EISSN:1557-7333
      DOI:10.1145/3319851
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 March 2019
      • Accepted: 1 November 2018
      • Revised: 1 August 2018
      • Received: 1 May 2017
      Published in tocs Volume 36, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!