skip to main content
research-article

I/o paravirtualization at the device file boundary

Published:24 February 2014Publication History
Skip Abstract Section

Abstract

Paravirtualization is an important I/O virtualization technology since it uniquely provides all of the following benefits: the ability to share the device between multiple VMs, support for legacy devices without virtualization hardware, and high performance. However, existing paravirtualization solutions have one main limitation: they only support one I/O device class, and would require significant engineering effort to support new device classes and features. In this paper, we present Paradice, a solution that vastly simplifies I/O paravirtualization by using a common paravirtualization boundary for various I/O device classes: Unix device files. Using this boundary, the paravirtual drivers simply act as a class-agnostic indirection layer between the application and the actual device driver.

We address two fundamental challenges: supporting cross-VM driver memory operations without changes to applications or device drivers and providing fault and device data isolation between guest VMs despite device driver bugs. We implement Paradice for x86, the Xen hypervisor, and the Linux and FreeBSD OSes. Our implementation paravirtualizes various GPUs, input devices, cameras, an audio device, and an Ethernet card for the netmap framework with ~7700 LoC, of which only ~900 are device class-specific. Our measurements show that Paradice achieves performance close to native for different devices and applications including netmap, 3D HD games, and OpenCL applications.

References

  1. Privilege escalation using NVIDIA GPU driver bug. http://www.securelist.com/en/advisories/50085.Google ScholarGoogle Scholar
  2. Privilege escalation using DRM/Radeon GPU driver bug. https://lkml.org/lkml/2010/1/18/106.Google ScholarGoogle Scholar
  3. CLOC. http://cloc.sourceforge.net/.Google ScholarGoogle Scholar
  4. OMAP4 Face Detection Module, Chapter 9 of the TRM. http://focus.ti.com/pdfs/wtbu/OMAP4460_ES1.0_PUBLIC_TRM_vF.zip.Google ScholarGoogle Scholar
  5. GalliumCompute. http://dri.freedesktop.org/wiki/GalliumCompute/.Google ScholarGoogle Scholar
  6. Guvcview. http://guvcview.sourceforge.net/.Google ScholarGoogle Scholar
  7. Nexuiz. http://www.alientrap.org/games/nexuiz.Google ScholarGoogle Scholar
  8. Tremulous. http://www.tremulous.net/.Google ScholarGoogle Scholar
  9. Everything is a file in Unix. http://ph7spot.com/musings/in-unix-everything-is-a-file.Google ScholarGoogle Scholar
  10. VGX. http://www.nvidia.com/object/vgx-hypervisor.html.Google ScholarGoogle Scholar
  11. VMDq. http://www.intel.com/content/www/us/en/network-adapters/gigabit-network-adapters/io-acceleration-technology-vmdq.html.Google ScholarGoogle Scholar
  12. Clang: a C Language Family Frontend for LLVM. http://clang.llvm.org/.Google ScholarGoogle Scholar
  13. OpenGL Microbenchmarks: Display List. http://www.songho.ca/opengl/gl_displaylist.html.Google ScholarGoogle Scholar
  14. GPU Benchmarking. http://www.phoronix.com/scan.php?page=article&item=virtualbox_4_opengl&num=2.Google ScholarGoogle Scholar
  15. OpenArena. http://openarena.ws/smfnews.php.Google ScholarGoogle Scholar
  16. Phoronix Test Suite. http://www.phoronix-test-suite.com/.Google ScholarGoogle Scholar
  17. Touchscreen Latency. http://www.engadget.com/2012/03/10/microsoft-cuts-touchscreen-lag-to-1ms/.Google ScholarGoogle Scholar
  18. OpenGL Microbenchmarks: Vertex Buffer Object and Vertex Array. http://www.songho.ca/opengl/gl_vbo.html.Google ScholarGoogle Scholar
  19. D. Abramson. Intel Virtualization Technology for Directed I/O. Intel Technology Journal, 10(3):179--192, 2006.Google ScholarGoogle ScholarCross RefCross Ref
  20. A. Amiri Sani, S. Nair, L. Zhong, and Q. Jacobson. Making I/O Virtualization Easy with Device Files. Technical Report 2013-04--13, Rice University, 2013.Google ScholarGoogle Scholar
  21. J. Andrus, C. Dall, A. V. Hof, O. Laadan, and J. Nieh. Cells: a Virtual Mobile Smartphone Architecture. In Proc. ACM SOSP, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the Art of Virtualization. In Proc. ACM SOSP, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. F. Bellard. QEMU, a Fast and Portable Dynamic Translator. In USENIX ATC, FREENIX Track, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Ben-Yehuda, M. D. Day, Z. Dubitzky, M. Factor, N. Har'El, A. Gordon, A. Liguori, O. Wasserman, and B. A. Yassour. The Turtles Project: Design and Implementation of Nested Virtualization. In Proc. USENIX OSDI, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An Empirical Study of Operating Systems Errors. In Proc. ACM SOSP, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Y. Dong, Z. Yu, and G. Rose. SR-IOV Networking in Xen: Architecture, Design and Implementation. In Proc. USENIX Workshop on I/O Virtualization (WIOV), 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. M. Dowty and J. Sugerman. GPU Virtualization on VMware's Hosted I/O Architecture. ACM SIGOPS Operating Systems Review, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. K. Fraser, S. Hand, R. Neugebauer, I. Pratt, A. Warfield, and M. Williamson. Safe Hardware Access with the Xen Virtual Machine Monitor. In Proc. Wrkshp. Operating System and Architectural Support for the On demand IT InfraStructure (OASIS), 2004.Google ScholarGoogle Scholar
  29. A. Ganapathi, V. Ganapathi, and D. Patterson. Windows XP Kernel Crash Analysis. In Proc. USENIX LISA, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. A. Gordon, N. Amit, N. Har'El, M. Ben-Yehuda, A. Landau, D. Tsafrir, and A. Schuster. ELI: Bare-Metal Performance for I/O Virtualization. In Proc. ACM ASPLOS, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. A. Gordon, N. Har'El, A. Landau, M. Ben-Yehuda, and A. Traeger. Towards Exitless and Efficient Paravirtual I/O. In Proc. SYSTOR, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. J. G. Hansen. Blink: Advanced Display Multiplexing for Virtualized Applications. In Proc. ACM Network and Operating System Support for Digital Audio and Video (NOSSDAV), 2007.Google ScholarGoogle Scholar
  33. N. HarEl, A. Gordon, A. Landau, M. Ben-Yehuda, A. Traeger, and R. Ladelsky. Efficient and Scalable Paravirtual I/O System. In Proc. USENIX ATC, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. S. Kato, K. Lakshmanan, R. R. Rajkumar, and Y. Ishikawa. TimeGraph: GPU Scheduling for Real-time Multi-tasking Environments. In Proc. USENIX ATC, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori. kvm: the Linux Virtual Machine Monitor. In Proc. Linux Symposium, 2007.Google ScholarGoogle Scholar
  36. H. A. Lagar-Cavilla, N. Tolia, M. Satyanarayanan, and E. D. Lara. VMM-Independent Graphics Acceleration. In Proc. ACM VEE, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proc. IEEE Int. Conf. on Code Generation and Optimization, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. J. LeVasseur, V. Uhlig, J. Stoess, and S. Götz. Unmodified Device Driver Reuse and Improved System Dependability via Virtual Machines. In Proc. USENIX OSDI, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. J. Liu, W. Huang, B. Abali, and D. K. Panda. High Performance VMM-Bypass I/O in Virtual Machines. In Proc. USENIX ATC, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. R. Nikolaev and G. Back. VirtuOS: An Operating System with Kernel Virtualization. In Proc. ACM SOSP, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. D. Presotto, R. Pike, K. Thompson, and H. Trickey. Plan 9, a Distributed System. In Proc. of the Spring 1991 EurOpen Conf., 1991.Google ScholarGoogle Scholar
  42. H. Raj and K. Schwan. High Performance and Scalable I/O Virtualization via Self-Virtualized Devices. In Proc. ACM HPDC, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. L. Rizzo. netmap: a Novel Framework for Fast Packet I/O. In Proc. USENIX ATC, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. R. Russel. virtio: Towards a De-Facto Standard for Virtual I/O Devices. ACM SIGOPS Operating Systems Review, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. L. Shi, H. Chen, and J. Sun. vCUDA: GPU Accelerated High Performance Computing in Virtual Machines. In IEEE Int. Symp. Parallel & Distributed Processing, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. C. Smowton. Secure 3D Graphics for Virtual Machines. In Proc. ACM European Wrkshp. System Security, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. J. Sugerman, G. Venkitachalam, and B. H. Lim. Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor. In Proc. USENIX ATC, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. M. M. Swift, B. N. Bershad, and H. M. Levy. Improving the Reliability of Commodity Operating Systems. In Proc. ACM SOSP, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. M. M. Swift, M. Annamalai, B. N. Bershad, and H. M. Levy. Recovering Device Drivers. In Proc. USENIX OSDI, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. L. Tan, E. M. Chan, R. Farivar, N. Mallick, J. C. Carlyle, F. M. David, and R. H. Campbell. iKernel: Isolating buggy and malicious device drivers using hardware virtualization support. In Proc. IEEE Int. Symp. Dependable, Autonomic and Secure Computing (DASC), 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. M. Weiser. Program slicing. In Proc. IEEE Int. Conf. on Software engineering.Google ScholarGoogle Scholar
  52. P. Willmann, J. Shafer, D. Carr, A. Menon, S. Rixner, A. L. Cox, andW. Zwaenepoel. Concurrent Direct Network Access for Virtual Machine Monitors. In Proc. IEEE High Performance Computer Architecture (HPCA), 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. P. Willmann, S. Rixner, and A. L. Cox. Protection Strategies for Direct Access to Virtualized I/O Devices. In Proc. USENIX ATC, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. I/o paravirtualization at the device file boundary

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGARCH Computer Architecture News
            ACM SIGARCH Computer Architecture News  Volume 42, Issue 1
            ASPLOS '14
            March 2014
            729 pages
            ISSN:0163-5964
            DOI:10.1145/2654822
            Issue’s Table of Contents
            • cover image ACM Conferences
              ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systems
              February 2014
              780 pages
              ISBN:9781450323055
              DOI:10.1145/2541940

            Copyright © 2014 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 24 February 2014

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!