skip to main content
research-article

Page Fault Support for Network Controllers

Authors Info & Claims
Published:04 April 2017Publication History
Skip Abstract Section

Abstract

Direct network I/O allows network controllers (NICs) to expose multiple instances of themselves, to be used by untrusted software without a trusted intermediary. Direct I/O thus frees researchers from legacy software, fueling studies that innovate in multitenant setups. Such studies, however, overwhelmingly ignore one serious problem: direct memory accesses (DMAs) of NICs disallow page faults, forcing systems to either pin entire address spaces to physical memory and thereby hinder memory utilization, or resort to APIs that pin/unpin memory buffers before/after they are DMAed, which complicates the programming model and hampers performance.

We solve this problem by designing and implementing page fault support for InfiniBand and Ethernet NICs. A main challenge we tackle---unique to NICs---is handling receive DMAs that trigger page faults, leaving the NIC without memory to store the incoming data. We demonstrate that our solution provides all the benefits associated with "regular" virtual memory, notably (1) a simpler programming model that rids users from the need to pin, and (2) the ability to employ all the canonical memory optimizations, such as memory overcommitment and demand-paging based on actual use. We show that, as a result, benchmark performance improves by up to 1.9x.

References

  1. Brian Aker and Mingqiang Zhuang. Memaslap - load testing and benchmarking a server. http://docs.libmemcached.org/bin/memaslap.html. libmemcached 1.1.0 documentation. Accessed: May 2016.Google ScholarGoogle Scholar
  2. M. Alizadeh, B. Atikoglu, A. Kabbani, A. Lakshmikantha, Rong Pan, B. Prabhakar, and M. Seaman. Data center transport mechanisms: Congestion control theory and IEEE standardization. In Communication, Control, and Computing, 2008 46th Annual Allerton Conference on, pages 1270--1277, Sept 2008. http://dx.doi.org/10.1109/ALLERTON.2008.4797706. Google ScholarGoogle ScholarCross RefCross Ref
  3. M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. RFC 2581, Internet Engineering Task Force, April 1999.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. AMD Inc. AMD IOMMU architectural specification, rev 2.00. http://developer.amd.com/wordpress/media/2012/10/488821.pdf, Mar 2011. Accessed: May 2016.Google ScholarGoogle Scholar
  5. Nadav Amit, Muli Ben-Yehuda, Dan Tsafrir, and Assaf Schuster. vIOMMU: efficient IOMMU emulation. In USENIX Annual Technical Conference (ATC), pages 73--86, 2011. http://www.usenix.org/events/atc11/tech/final_files/Amit.pdf.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Nadav Amit, Abel Gordon, Nadav Har'El, Muli Ben-Yehuda, Alex Landau, Assaf Schuster, and Dan Tsafrir. Bare-metal performance for virtual machines with exitless interrupts. Communications of the ACM (CACM), 59(1):108--116, Jan 2016. http://dx.doi.org/10.1145/2845648. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Nadav Amit, Dan Tsafrir, and Assaf Schuster. VSwapper: A memory swapper for virtualized environments. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 349--366, 2014. http://dx.doi.org/10.1145/2541940.2541969. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Apple Inc. Thunderbolt device driver programming guide: Debugging VT-d I/O MMU virtualization. https://developer.apple.com/library/mac/documentation/HardwareDrivers/Conceptual/ThunderboltDevGuide/DebuggingThunderboltDrivers/DebuggingThunderboltDrivers.html, 2013. Accessed: May 2014.Google ScholarGoogle Scholar
  9. Andrea Arcangeli. Integrating KVM with the linux memory management. In KVM Forum, 2008.Google ScholarGoogle Scholar
  10. ARM Holdings. ARM system memory management unit architecture specification -- SMMU architecture version 2.0. http://infocenter.arm.com/help/topic/com.arm.doc.ihi0062c/IHI0062C_system_mmu_architecture_specification.pdf, 2013. Accessed: Jan 2015.Google ScholarGoogle Scholar
  11. Jens Axboe. Fio -- flexible IO tester. http://git.kernel.dk/?p=fio.git.Google ScholarGoogle Scholar
  12. Thomas Ball, Ella Bounimova, Byron Cook, Vladimir Levin, Jakob Lichtenberg, Con McGarvey, Bohus Ondrusek, Sriram K. Rajamani, and Abdullah Ustuner. Thorough static analysis of device drivers. In ACM Eurosys, pages 73--85, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. IX: A protected dataplane operating system for high throughput and low latency. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 49--65, 2014. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-belay.pdf.Google ScholarGoogle Scholar
  14. Christian Bell and Dan Bonachea. A new DMA registration strategy for pinning-based high performance networks. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2003. http://dx.doi.org/10.1109/IPDPS.2003.1213363. Google ScholarGoogle ScholarCross RefCross Ref
  15. Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, and Dan Tsafrir. The nom profit-maximizing operating system. In ACM International Conference on Virtual Execution Environments (VEE), pages 145--160, 2016. http://dx.doi.org/10.1145/2892242.2892250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Muli Ben-Yehuda, Michael D. Day, Zvi Dubitzky, Michael Factor, Nadav Har'El, Abel Gordon, Anthony Liguori, Orit Wasserman, and Ben-Ami Yassour. The Turtles project: Design and implementation of nested virtualization. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 423--436, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Ben-Yehuda.pdf.Google ScholarGoogle Scholar
  17. Muli Ben-Yehuda, Jimi Xenidis, Michal Ostrowski, Karl Rister, Alexis Bruemmer, and Leendert van Doorn. The price of safety: Evaluating IOMMU performance. In Ottawa Linux Symposium (OLS), pages 9--20, 2007. https://www.kernel.org/doc/mirror/ols2007v1.pdf#page=9.Google ScholarGoogle Scholar
  18. Ravi Bhargava, Benjamin Serebrin, Francesco Spadini, and Srilatha Manne. Accelerating two-dimensional page walks for virtualized systems. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 26--35, 2008. http://dx.doi.org/10.1145/1346281.1346286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Robert Birke, Lydia Y Chen, and Evgenia Smirni. Data centers in the wild: A large performance study. Technical Report RZ3820, IBM Research, 2012. http://domino.research.ibm.com/library/cyberdig.nsf/papers/0C306B31CF0D3861852579E40045F17F.Google ScholarGoogle Scholar
  20. James E.J. Bottomley. Dynamic DMA mapping using the generic device. https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/Documentation/DMA-API.txt?id=refs/tags/v3.18.3. Linux kernel documentation. Accessed: Jan 2015.Google ScholarGoogle Scholar
  21. Ethan Burns. Implementation and comparison of iSCSI over RDMA. PhD thesis, University of New Hampshire, 2008.Google ScholarGoogle Scholar
  22. Mallikarjun Chadalapaka, Uri Elzur, Michael Ku, Hemal Shah, and Patricia Thaler. A Study of iSCSI Extensions for RDMA. In Computer-Communication Networks, August 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yuqun Chen, Angelos Bilas, Stefanos N. Damianakis, Cezary Dubnicki, and Kai Li. UTLB: A mechanism for address translation on network interfaces. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 193--204, 1998. http://dx.doi.org/10.1145/291069.291046. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Andy Chou, Junfeng Yang, Benjamin Chelf, Seth Hallem, and Dawson Engler. An empirical study of operating systems errors. In ACM Symposium on Operating Systems Principles (SOSP), pages 73--88, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jonathan Corbet. Linux Device Drivers, chapter 15: Memory Mapping and DMA. O'Reilly, 3rd edition, 2005.Google ScholarGoogle Scholar
  26. Jonathan Corbet. Newer, newer NAPI. LWN https://lwn.net/Articles/244640/, Aug 2007. (Accessed: Aug 2016).Google ScholarGoogle Scholar
  27. Intel Corporation. Intel MPI benchmarks. https://software.intel.com/en-us/articles/intel-mpi-benchmarks, 2013.Google ScholarGoogle Scholar
  28. Crehan Research. Another year of robust growth and record shipments for branded data center switches. http://www.crehanresearch.com/wp-content/uploads/2015/03/CREHAN-2014-Data-Center-Switching-CR.pdf, Mar 2015. (Accessed: Aug 2015).Google ScholarGoogle Scholar
  29. Yaozu Dong, Yu Chen, Zhenhao Pan, Jinquan Dai, and Yunhong Jiang. ReNIC: Architectural extension to SR-IOV I/O virtualization for efficient replication. ACM Transactions on Architecture and Code Optimization (TACO), 8(4):40:1--40:22, Jan 2012. http://dx.doi.org/10.1145/2086696.2086719. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Aleksandar Dragojević, Dushyanth Narayanan, Miguel Castro, and Orion Hodson. FaRM: Fast remote memory. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 401--414, 2014. https://www.usenix.org/conference/nsdi14/technical-sessions/dragojevic.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Adam Dunkels. Design and implementation of the lwIP TCP/IP stack. Swedish Institute of Computer Science, 2:77, 2001.Google ScholarGoogle Scholar
  32. Montse Farreras, George Almasi, Calin Cascaval, and Toni Cortes. Scalable RDMA performance in PGAS languages. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2009. http://dx.doi.org/10.1109/IPDPS.2009.5161025. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Brad Fitzpatrick. Distributed caching with memcached. Linux Journal, 2004(124):5, Aug 2004. http://dl.acm.org/citation.cfm?id=1012889.1012894.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Sally Floyd, Dr. K. K. Ramakrishnan, and David L. Black. The Addition of Explicit Congestion Notification (ECN) to IP. RFC 3168, March 2013.Google ScholarGoogle Scholar
  35. Tomonori Fujita and Mike Christie. tgt: Framework for Storage Target Drivers. In Proceedings of the Linux Symposium, July 2006.Google ScholarGoogle Scholar
  36. GASNet 1.26.0. https://gasnet.lbl.gov/GASNet-1.26.0.tar.gz, October 2015. (Accessed: May 2016).Google ScholarGoogle Scholar
  37. Dror Goldenberg, Michael Kagan, Ran Ravid, and Michael S. Tsirkin. Zero copy sockets direct protocol over InfiniBand -- preliminary implementation and performance analysis. In IEEE Symposium on High Performance Interconnects (HOTI), pages 128--137, 2005. http://dx.doi.org/10.1109/CONECT.2005.35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Kinshuk Govil, Dan Teodosiu, Yongqiang Huang, and Mendel Rosenblum. Cellular disco: Resource management using virtual clusters on shared-memory multiprocessors. In ACM Symposium on Operating Systems Principles (SOSP), pages 154--169, 1999. http://dx.doi.org/10.1145/319344.319162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Diwaker Gupta, Sangmin Lee, Michael Vrable, Stefan Savage, Alex C. Snoeren, George Varghese, Geoffrey M. Voelker, and Amin Vahdat. Difference engine: Harnessing memory redundancy in virtual machines. Communications of the ACM (CACM), pages 85--93, 2010. http://dx.doi.org/10.1145/1831407.1831429. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. James Hamilton. AWS innovation at scale. https://www.youtube.com/watch?t=113&v=JIQETrFC_SQ, Nov 2014. (Accessed: Aug 2015).Google ScholarGoogle Scholar
  41. Nadav Har'El, Abel Gordon, Alex Landau, Muli Ben-Yehuda, Avishay Traeger, and Razya Ladelsky. Efficient and scalable paravirtual I/O system. In USENIX Annual Technical Conference (ATC), pages 231--242, 2013. https://www.usenix.org/system/files/conference/atc13/atc13-harel.pdf.Google ScholarGoogle Scholar
  42. Jorrit N. Herder, Herbert Bos, Ben Gras, Philip Homburg, and Andrew S. Tanenbaum. Failure resilience for device drivers. In IEEE/IFIP Annual International Conference on Dependable Systems and Networks (DSN), pages 41--50, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Gregory D. Hill and Albert H. Chen. High performance network multiplexing with IXGoogle ScholarGoogle Scholar
  44. . Research report, Stanford University, 2015. http://hselin.com/resources/CS344g_ixplusplus_final%20paper.pdf.Google ScholarGoogle Scholar
  45. Michael R. Hines, Abel Gordon, Marcio Silva, Dilma Da Silva, Kyung Ryu, and Muli Ben-Yehuda. Applications know best: Performance-driven memory overcommit with Ginkgo. In IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pages 130--137, 2011. http://dx.doi.org/10.1109/CloudCom.2011.27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Eric Horschman. Hypervisor memory management done right. http://blogs.vmware.com/virtualreality/2011/02/hypervisor-memory-management-done-right.html, 2011. (Accessed: May 2016).Google ScholarGoogle Scholar
  47. The HSA Foundation. http://www.hsafoundation.com/.Google ScholarGoogle Scholar
  48. HSA Foundation. HSA-Drivers-Linux-AMD. https://github.com/HSAFoundation/HSA-Drivers-Linux-AMD. (Accessed: May 2016).Google ScholarGoogle Scholar
  49. Woomin Hwang, Yangwoo Roh, Youngwoo Park, Ki-Woong Park, and Kyu Ho Park. HyperDealer: Reference pattern aware instant memory balancing for consolidated virtual machines. In IEEE International Conference on Cloud Computing (CLOUD), pages 426--434, 2014. http://dx.doi.org/10.1109/CLOUD.2010.70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. IBM Corporation. PowerLinux servers -- 64-bit DMA concepts. http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liabm/liabmconcepts.htm. Accessed: May 2014.Google ScholarGoogle Scholar
  51. IBM Corporation. AIX kernel extensions and device support programming concepts. https://publib.boulder.ibm.com/infocenter/aix/v7r1/topic/com.ibm.aix.kernelext/doc/kernextc/kernextc\_pdf.pdf, 2013. Accssed: May 2014.Google ScholarGoogle Scholar
  52. IEEE. Specification for 802.3 full duplex operation. IEEE Standard 802.3x http://dx.doi.org, 1997.Google ScholarGoogle Scholar
  53. VMware Inc. Configuring VMDirectPath I/O pass-through devices on a VMware ESX or VMware ESXi host. https://kb.vmware.com/kb/1010789. VMware Knowledge Base. Accessed: Aug 2016.Google ScholarGoogle Scholar
  54. InfiniBand Trade Association (IBTA). About InfiniBand. http://www.infinibandta.org/content/pages.php?pg=about\_us\_infiniband. (Accessed: May 2016).Google ScholarGoogle Scholar
  55. InfiniBand Trade Association (IBTA). About RoCE. http://www.infinibandta.org/content/pages.php?pg=about\_us\_RoCE. (Accessed: May 2016).Google ScholarGoogle Scholar
  56. Intel. PCI-SIG SR-IOV primer: An introduction to SR-IOV technology. http://www.intel.com/content/www/us/en/pci-express/pci-sig-sr-iov-primer-sr-iov-technology-paper.html, Jan 2011.Google ScholarGoogle Scholar
  57. Intel Corporation. DPDK: Data plane development kit. http://dpdk.org. (Accessed: May 2016).Google ScholarGoogle Scholar
  58. Intel Corporation. Intel virtualization technology for directed I/O - architecture specification - specification - Rev.\ 2.2. http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf, Sep 2013. Accessed: Jan 2015.Google ScholarGoogle Scholar
  59. Intel Corporation. Intel virtualization technology for directed I/O - architecture specification - Rev.\ 2.3. http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf, Oct 2014.Google ScholarGoogle Scholar
  60. Rick A. Jones. Netperf: A network performance benchmark (Revision 2.0). http://www.netperf.org/netperf/training/Netperf.html, 1995. Accessed: August, 2016.Google ScholarGoogle Scholar
  61. M. Frans Kaashoek, Dawson R. Engler, Gregory R. Ganger, Hector M. Brice\ no, Russell Hunt, David Mazières, Thomas Pinckney, Robert Grimm, John Jannotti, and Kenneth Mackenzie. Application performance and flexibility on exokernel systems. In ACM Symposium on Operating Systems Principles (SOSP), pages 52--65, 1997.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Anuj Kalia, Michael Kaminsky, and David G. Andersen. Design guidelines for high performance RDMA systems. In USENIX Annual Technical Conference (ATC), pages 437--450, 2016. https://www.usenix.org/conference/atc16/technical-sessions/presentation/kalia.Google ScholarGoogle Scholar
  63. Antoine Kaufmann, SImon Peter, Naveen Kr. Sharma, Thomas Anderson, and Arvind Krishnamurthy. High performance packet processing with FlexNIC. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 67--81, 2016. http://dx.doi.org/10.1145/2872362.2872367. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Alice E. Koniges, Rolf Rabenseifner, and Karl Solchenbach. Benchmark design for characterization of balanced high-performance architectures. In Proceedings of the 15th International Parallel & Distributed Processing Symposium, IPDPS '01, pages 196--, Washington, DC, USA, 2001. IEEE Computer Society. Google ScholarGoogle ScholarCross RefCross Ref
  65. Yossi Kuperman, Eyal Moscovici, Joel Nider, Razya Ladelsky, Abel Gordon, and Dan Tsafrir. Paravirtual remote I/O. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 49--65, 2016. http://dx.doi.org/10.1145/2872362.2872378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. George Kyriazis. Heterogeneous system architecture: A technical review. Technical report, AMD Inc., Aug 2012. Rev. 1.0 http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/hsa10.pdf (Accessed: May 2016).Google ScholarGoogle Scholar
  67. Joshua LeVasseur, Volkmar Uhlig, Jan Stoess, and Stefan Götz. Unmodified device driver reuse and improved system dependability via virtual machines. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 17--30, 2004. https://www.usenix.org/legacy/publications/library/proceedings/osdi04/tech/full_papers/levasseur/levasseur.pdf.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Sheng Li, Hyeontaek Lim, Victor W. Lee, Jung Ho Ahn, Anuj Kalia, Michael Kaminsky, David G. Andersen, O. Seongil, Sukhan Lee, and Pradeep Dubey. Architecting to achieve a billion requests per second throughput on a single key-value store server platform. In ACM International Symposium on Computer Architecture (ISCA), pages 476--488, 2015. https://doi.org/10.1145/2749469.2750416.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Hyeontaek Lim, Dongsu Han, David G. Andersen, and Michael Kaminsky. MICA: A holistic approach to fast in-memory key-value storage. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 429--444, 2014. https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/lim.Google ScholarGoogle Scholar
  70. getrlimit(2) -- Linux man page. http://linux.die.net/man/2/getrlimit. (Accessed: May 2016).Google ScholarGoogle Scholar
  71. mlock(2) -- Linux man page. http://linux.die.net/man/2/mlock. (Accessed: May 2016).Google ScholarGoogle Scholar
  72. The include/uapi/linux/resource.h header file of Linux 4.5. http://lxr.free-electrons.com/source/include/uapi/linux/resource.h?v=4.5\#L71. (Accessed: May 2016).Google ScholarGoogle Scholar
  73. Jiuxing Liu, Dhabaleswar K. Panda, Jiuxing Liu Dhabaleswar K. P, and Mohammad Banikazemi. Evaluating the impact of RDMA on storage I/O over InfiniBand. In SAN-03 Workshop (in conjunction with HPCA), 2004, 2004.Google ScholarGoogle Scholar
  74. Moshe Malka, Nadav Amit, Muli Ben-Yehuda, and Dan Tsafrir. rIOMMU: Efficient IOMMU for I/O devices that employ ring buffers. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 355--368, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Moshe Malka, Nadav Amit, and Dan Tsafrir. Efficient intra-operating system protection against harmful DMAs. In USENIX Conference on File and Storage Technologies (FAST), pages 29--44, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Vinod Mamtani. DMA directions and Windows. http://download.microsoft.com/download/a/f/d/afdfd50d-6eb9--425e-84e1-b4085a80e34e/sys-t304\_wh07.pptx, 2007. Accessed: May 2014.Google ScholarGoogle Scholar
  77. Alex Markuze, Adam Morrison, and Dan Tsafrir. It's DAMN time for overhead-free IOMMU protection. Submitted.Google ScholarGoogle Scholar
  78. Alex Markuze, Adam Morrison, and Dan Tsafrir. True IOMMU protection from DMA attacks: When copy is faster than zero copy. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 249--262, 2016. http://dx.doi.org/10.1145/2872362.2872379. Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Paul Menage. Cgroups. https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt.Google ScholarGoogle Scholar
  80. Frank Mietke, Robert Rex, Robert Baumgartl, Torsten Mehlan, Torsten Hoefler, and Wolfgang Rehm. Analysis of the memory registration process in the Mellanox InfiniBand software stack. In International European Conference on Parallel and Distributed Computing (Euro-Par), pages 124--133, 2006. http://dx.doi.org/10.1007/11823285_13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Juan Navarro, Sitaram Iyer, Peter Druschel, and Alan Cox. Practical, transparent operating system support for superpages. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 89--104, 2002. Google ScholarGoogle ScholarCross RefCross Ref
  82. Jarek Nieplocha, Vinod Tipparaju, Amina Saify, and Dhabaleswar K. Panda. Protocols and strategies for optimizing performance of remote memory operations on clusters. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2002. http://dx.doi.org/10.1109/IPDPS.2002.1016563. Google ScholarGoogle ScholarCross RefCross Ref
  83. Radhika Niranjan Mysore, George Porter, and Amin Vahdat. FasTrak: Enabling express lanes in multi-tenant data centers. In ACM Conference on Emerging Networking Experiments and Technologies (CoNEXT), pages 139--150, 2013. http://dx.doi.org/10.1145/2535372.2535386. Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. Li Ou, Xubin He, and Jizhong Han. An efficient design for fast memory registration in RDMA. Journal of Network and Computer Applications, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Shoumik Palkar, Chang Lan, Sangjin Han, Keon Jang, Aurojit Panda, Sylvia Ratnasamy, Luigi Rizzo, and Scott Shenker. E2: A framework for NFV applications. In ACM Symposium on Operating Systems Principles (SOSP), pages 121--136, 2015. https://doi.org/10.1145/2815400.2815423. Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. PCI-SIG. Single root I/O virtualization and sharing 1.0 specification. http://www.pcisig.com/specifications/iov/single_root/, Sep 2007. (Accessed: Aug 2016).Google ScholarGoogle Scholar
  87. PCI-SIG. Address Translation Services Revision 1.1. http://www.pcisig.com/specifications/iov/ats/, 2009.Google ScholarGoogle Scholar
  88. PCI-SIG. Single root I/O virtualization and sharing 1.1 specification. http://www.pcisig.com/specifications/iov/single_root/, Jan 2010. (Accessed: Aug 2016).Google ScholarGoogle Scholar
  89. Omer Peleg, Adam Morrison, Benjamin Serebrin, and Dan Tsafrir. Utilizing the IOMMU Scalably. In USENIX Annual Technical Conference (ATC), 2015.Google ScholarGoogle Scholar
  90. Simon Peter, Jialin Li, Doug Woos, Irene Zhang, Dan R. K. Ports, Thomas Anderson, Arvind Krishnamurthy, and Mark Zbikowski. Towards high-performance application-level storage management. In USENIX Workshop on Hot Topics in Storage and File Systems (HOTSTORAGE), 2014. https://www.usenix.org/system/files/conference/hotstorage14/hotstorage14-paper-peter.pdf.Google ScholarGoogle Scholar
  91. Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. Arrakis: The operating system is the control plane. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 1--16, 2014. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-peter_simon.pdf.Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Renato J. Recio, Bernard Metzler, Paul R. Culley, Jeff Hilland, and Dave Garcia. A remote direct memory access protocol specification. RFC 5040, The Internet Engineering Task Force (IETF) Network Working Group, 2007. https://tools.ietf.org/html/rfc5040 (Accessed: May 2016).Google ScholarGoogle Scholar
  93. Bruce Richardson. [dpdk-dev] memory pinning. http://dpdk.org/ml/archives/dev/2014-June/003937.html, 2014. (Accessed: Aug 2016).Google ScholarGoogle Scholar
  94. Luigi Rizzo. Netmap: A novel framework for fast packet I/O. In USENIX Annual Technical Conference (ATC), pages 101--112, 2012. https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo.Google ScholarGoogle Scholar
  95. Phil Rogers. Heterogeneous System Architecture (HSA): Overview and implementation. In Hot Chips, 2013. HC25. http://www.hotchips.org/wp-content/uploads/hc_archives/hc25/HC25.0T1-Hetero-epub/HC25.25.100-Intro-Rogers-HSA%20Intro%20HotChips2013_Final.pdf (Accessed: May 2016).Google ScholarGoogle Scholar
  96. Arvind Seshadri, Mark Luk, Ning Qu, and Adrian Perrig. SecVisor: A tiny hypervisor to provide lifetime kernel code integrity for commodity OSes. In ACM Symposium on Operating Systems Principles (SOSP), pages 335--350, 2007. http://dx.doi.org/10.1145/1294261.1294294. Google ScholarGoogle ScholarDigital LibraryDigital Library
  97. Jiaxin Shi, Youyang Yao, Rong Chen, Haibo Chen, and Feifei Li. Fast and concurrent RDF queries with RDMA-based distributed graph exploration. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 317--332, 2016. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/shi.Google ScholarGoogle Scholar
  98. Igor Smolyar, Muli Ben-Yehuda, and Dan Tsafrir. Securing self-virtualizing Ethernet devices. In USENIX Security Symposium, pages 335--350, 2015.Google ScholarGoogle Scholar
  99. Livio Soares and Michael Stumm. FlexSC: Flexible system call scheduling with exception-less system calls. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 33--46, 2010. https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Soares.pdf.Google ScholarGoogle ScholarDigital LibraryDigital Library
  100. Vaidyanathan Srinivasan, Anand K. Santhanam, and Madhavan Srinivasan. Cell Broadband Engine processor DMA engines, Part 1: The little engines that move data. http://www.ibm.com/developerworks/library/pa-celldmas, 2005. (Accessed: May 2016).Google ScholarGoogle Scholar
  101. B. Stephens, A.L. Cox, A. Singla, J. Carter, C. Dixon, and W. Felter. Practical DCB for improved data center networks. In INFOCOM, 2014 Proceedings IEEE, pages 1824--1832, April 2014. Google ScholarGoogle ScholarCross RefCross Ref
  102. Michael Swift, Brian Bershad, and Henry Levy. Improving the reliability of commodity operating systems. ACM Transactions on Computer Systems (TOCS), 23(1):77--110, Feb 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  103. Taneja Group. Hypervisor shootout: Maximizing workload density in the virtualization platform. http://www.vmware.com/files/pdf/vmware-maximize-workload-density-tg.pdf, 2010. (Accessed: May 2016).Google ScholarGoogle Scholar
  104. Hiroshi Tezuka, Francis O'Carroll, Atsushi Hori, and Yutaka Ishikawa. Pin-down cache: A virtual memory management technique for zero-copy communication. In IEEE International Parallel Processing Symposium (IPPS), pages 308--314, 1998. http://dx.doi.org/10.1109/IPPS.1998.669932. Google ScholarGoogle ScholarCross RefCross Ref
  105. Animesh Trivedi. Remote direct memory access (RDMA) 101 -- quick history lesson and introduction. http://0x8086.blogspot.com/2011/11/remote-direct-memory-access-rdma-101.html, 2011. (Accessed: May 2016).Google ScholarGoogle Scholar
  106. Cheng-Chun Tu, Michael Ferdman, Chao-tung Lee, and Tzi-cker Chiueh. A comprehensive implementation and evaluation of direct interrupt delivery. In ACM International Conference on Virtual Execution Environments (VEE), pages 1--15, 2016. http://dx.doi.org/10.1145/2731186.2731189. Google ScholarGoogle ScholarDigital LibraryDigital Library
  107. Cheng-Chun Tu, Chao-tang Lee, and Tzi-cker Chiueh. Marlin: A memory-based rack area network. In ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 125--136, 2014. http://doi.acm.org/10.1145/2658260.2658262.Google ScholarGoogle Scholar
  108. Gabrie van Zanten. Memory overcommit in production? YES YES YES. http://www.gabesvirtualworld.com/memory-overcommit-in-production-yes-yes-yes/, 2010. (Accessed: May 2016).Google ScholarGoogle Scholar
  109. Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. Large-scale cluster management at Google with Borg. In Proceedings of the Tenth European Conference on Computer Systems, EuroSys '15, pages 18:1--18:17, New York, NY, USA, 2015. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  110. Carl A. Waldspurger. Memory resource management in VMware ESX server. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 181--194, 2002. https://www.usenix.org/legacy/events/osdi02/tech/waldspurger.html. Google ScholarGoogle ScholarCross RefCross Ref
  111. Xingda Wei, Jiaxin Shi, Yanzhe Chen, Rong Chen, and Haibo Chen. Fast in-memory transaction processing using RDMA and H™. In ACM Symposium on Operating Systems Principles (SOSP), pages 87--104, 2015. https://doi.org/10.1145/2815400.2815419. Google ScholarGoogle ScholarDigital LibraryDigital Library
  112. Wikipedia. iWARP -- internet Wide Area RDMA Protocol. https://en.wikipedia.org/wiki/IWARP. (Accessed: Aug 2016).Google ScholarGoogle Scholar
  113. Dan Williams, Patrick Reynolds, Kevin Walsh, Emin Gün Sirer, and Fred B. Schneider. Device driver safety through a reference validation mechanism. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 241--254, 2008.Google ScholarGoogle ScholarDigital LibraryDigital Library
  114. Alex Williamson. VFIO: A user's perspective. In KVM Forum, 2012. http://www.linux-kvm.org/images/b/b4/2012-forum-VFIO.pdf. (Accsessed: May 2016).Google ScholarGoogle Scholar
  115. Alex Williamson. [qemu-devel] Intel IOMMU guest emulation and vfio-pci passthrough. https://lists.gnu.org/archive/html/qemu-devel/2015--11/msg04284.html, Nov 2015. (Accessed: Aug 2016).Google ScholarGoogle Scholar
  116. Paul Willmann, Scott Rixner, and Alan L. Cox. Protection strategies for direct access to virtualized I/O devices. In USENIX Annual Technical Conference (ATC), pages 15--28, 2008. https://www.usenix.org/legacy/event/usenix08/tech/full_papers/willmann/willmann.pdf.Google ScholarGoogle ScholarDigital LibraryDigital Library
  117. Timothy Wood, Gabriel Tarasuk-Levin, Prashant Shenoy, Peter Desnoyers, Emmanuel Cecchet, and Mark D. Corner. Memory buddies: Exploiting page sharing for smart colocation in virtualized data centers. In ACM International Conference on Virtual Execution Environments (VEE), pages 31--40, 2009. http://dx.doi.org/10.1145/1508293.1508299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  118. Jiesheng Wu, Pete Wyckoff, and Dhabaleswar Panda. PVFS over InfiniBand: Design and performance evaluation. In International Conference on Parallel Processing (ICPP), pages 125--132, 2003. http://dx.doi.org/10.1109/ICPP.2003.1240573. Google ScholarGoogle ScholarCross RefCross Ref
  119. Xiaowei Yang, Chuan Ye, and Qiangmin Lin. Evaluation and enhancement to memory sharing and swapping in Xen 4.1. In Xen Summit, 2011. http://tinyurl.com/xen-mem-share-swap (Accessed: May 2016).Google ScholarGoogle Scholar
  120. Ben-Ami Yassour, Muli Ben-Yehuda, and Orit Wasserman. On the DMA mapping problem in direct device assignment. In ACM International Systems and Storage Conference (SYSTOR), pages 18:1--18:12, 2010. http://dx.doi.org/10.1145/1815695.1815718. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Page Fault Support for Network Controllers

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!