Abstract
Streamline is a stream-based OS communication subsystem that spans from peripheral hardware to userspace processes. It improves performance of I/O-bound applications (such as webservers and streaming media applications) by constructing tailor-made I/O paths through the operating system for each application at runtime. Path optimization removes unnecessary copying, context switching and cache replacement and integrates specialized hardware. Streamline automates optimization and only presents users a clear, concise job control language based on Unix pipelines. For backward compatibility Streamline also presents well known files, pipes and sockets abstractions. Observed throughput improvement over Linux 2.6.24 for networking applications is up to 30-fold, but two-fold is more typical.
- Bavier, A., Voigt, T., Wawrzoniak, M., Peterson, L., and Gunningberg, P. 2002. Silk: Scout paths in the linux kernel. Tech. rep., Uppsala University.Google Scholar
- Bershad, B. N., Savage, S., Pardyak, P., Sirer, E. G., Fiuczynski, M., Becker, D., Eggers, S., and Chambers, C. 1995. Extensibility, safety and performance in the spin operating system. In Proceedings of the 15th Symposium on Operating Systems Principles. 267--284. Google Scholar
Digital Library
- Blevins, P. R. and Ramamoorthy, C. V. 1976. Aspects of a dynamically adaptive operating system. IEEE Trans. Comput. 25, 7, 713--725. Google Scholar
Digital Library
- Bos, H., de Bruijn, W., Cristea, M., Nguyen, T., and Portokalidis, G. 2004. Ffpf: Fairly fast packet filters. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). Google Scholar
Digital Library
- Brustoloni, J. C. and Steenkiste, P. 1996. Effects of buffering semantics on i/o performance. In Proceedings of the Symposium on Operating Systems Design and Implementation. 277--291. Google Scholar
Digital Library
- Campbell, R. H., Islam, N., Johnson, R., Kougiouris, P., and Madany, P. 1991. Choices, frameworks and refinement. In Proceedings of the International Workshop on Object Orientation in Operating Systems. 9--15.Google Scholar
- Carter, J. B., Ford, B., Hibler, M., Kuramkote, R., Law, J., Lepreau, J., Orr, D. B., Stoller, L., and Swanson, M. 1993. FLEX: A tool for building efficient and flexible systems. In Proceedings of the 4th IEEE Workshop on Workstation Operating Systems.Google Scholar
- Clark, D. D. 1985. The structuring of systems using upcalls. In Proceedings of the 10th ACM Symposium on Operating Systems Principles (SOSP’85). ACM Press, New York, NY, USA, 171--180. Google Scholar
Digital Library
- Clark, D. D. and Tennenhouse, D. L. 1990. Architectural considerations for a new generation of protocols. In Proceedings of the ACM Symposium on Communications Architectures and Protocols SIGCOMM’90. Google Scholar
Digital Library
- Cleary, J., Donnelly, S., Graham, I., McGregor, A., and Pearson, M. 2000. Design principles for accurate passive measurement. In Proceedings of the Passive and Active Measurement Conference (PAM).Google Scholar
- Cristea, M.-L., Gommans, L., Xu, L., and Bos, H. 2007. The token based switch: Per-packet access authorisation to optical shortcuts. In Proceedings of the 6th International IFIP-TC6 Networking Conference. Lecture Notes in Computer Science Series, vol. 4479. Google Scholar
Digital Library
- de Bruijn, W. 2010. Adaptive operating system design for high throughput i/o. Ph.D. thesis, Vrije Universiteit Amsterdam.Google Scholar
- de Bruijn, W. and Bos, H. 2008a. Beltway buffers: Avoiding the os traffic jam. In Proceedings of the Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM).Google Scholar
- de Bruijn, W. and Bos, H. 2008b. Pipesfs: Fast linux i/o in the unix tradition. ACM SigOps Oper. Syst. Rev. 42, 5, (Special Issue on R&D in the Linux Kernel). Google Scholar
Digital Library
- de Bruijn, W., Slowinska, A., van Reeuwijk, K., Hruby, T., Xu, L., and Bos, H. 2006. Safecard: A gigabit ips on the network card. In Proceedings of the 9th International Symposium on Recent Advances in Intrusion Detection (RAID’06). Google Scholar
Digital Library
- Druschel, P., Peterson, L. L., and Hutchinson, N. C. 1992. Beyond micro-kernel design: Decoupling modularity and protection in Lipto. In Proceedings of the 12th International Conference on Distributed Computing Systems. 512--520.Google Scholar
- Druschel, P. and Peterson, L. L. 1993. Fbufs: A high-bandwidth cross-domain transfer facility. In Proceedings of the Symposium on Operating Systems Principles. 189--202. Google Scholar
Digital Library
- Druschel, P., Abbott, M. B., Pagals, M. A., and Peterson, L. L. 1993. Network subsystems design. IEEE Netw. 7, 4, 8--17.Google Scholar
Digital Library
- Fedorova, A., Seltzer, M., Small, C., and Nussbaum, D. 2004. Throughput-oriented scheduling on chip multithreading systems. Tech. rep. TR-17-04, Harvard University.Google Scholar
- Fiuczynski, M. E., Martin, R. P., Owa, T., and Bershad, B. N. 1998. Spine: A safe programmable and integrated network environment. In Proceedings of the 8th ACM SIGOPS European Workshop on Support for Composing Distributed Applications. ACM Press, New York, NY, 7--12. Google Scholar
Digital Library
- Fraser, K., Hand, S., Neugebauer, R., Pratt, I., Warfield, A., and Williamson, M. 2004. Safe hardware access with the xen virtual machine monitor. In Proceedings of the OASIS ASPLOS Workshop (OASIS).Google Scholar
- Fröhlich, A. A. and Schröder-Preikschat, W. 1999. Tailor-made operating systems for embedded parallel applications. In Proceedings of the 4th IPPS/SPDP Workshop on Embedded HPC Systems and Applications. Google Scholar
Digital Library
- Govindan, R. and Anderson, D. P. 1991. Scheduling and ipc mechanisms for continuous media. In Proceedings of 13th ACM Symposium on Operating Systems Principles. ACM SIGOPS, 68--80. Google Scholar
Digital Library
- Hruby, T., van Reeuwijk, K., and Bos, H. 2007. Ruler: High-speed packet matching and rewriting on npus. In Proceedings of the 3rd ACM/IEEE Symposium on Architecture for Networking and Communications Systems (ANCS’07). ACM, New York, NY, 1--10. Google Scholar
Digital Library
- Hutchinson, N. C. and Peterson, L. L. 1991. The x-kernel: An architecture for implementing network protocols. IEEE Trans. Softw. Engin. 17, 1, 64--76. Google Scholar
Digital Library
- Isard, M., Budiu, M., Yu, Y., Birrell, A., and Fetterly, D. 2007. Dryad: Distributed data-parallel programs from sequential building blocks. In Proceedings of Eurosys’07. Google Scholar
Digital Library
- Jacobson, V. and Felderman, B. 2006. A modest proposal to help speed up & scale up the linux networking stack. http://www.linux.org.au/conf/2006/abstract8204.html?id=382.Google Scholar
- Karlin, S. and Peterson, L. 2002. Vera: an extensible router architecture. Comput. Netw. 38, 3, 277--293. Google Scholar
Digital Library
- Kohler, E., Morris, R., Chen, B., Jannotti, J., and Kaashoek, M. F. 2000. The click modular router. ACM Trans. Comput. Syst. 18, 3, 263--297. Google Scholar
Digital Library
- Lea, R., Yokote, Y., and Itoh, J.-I. 1995. Adaptive operating system design using reflection. In Proceedings of the 5th Workshop on Hot Topics in Operating Systems (HOTOS 95). IEEE Computer Society, Los Alamiros, CA, 95. Google Scholar
Digital Library
- Liu, X., Kreitz, C., van Renesse, R., Hickey, J., Hayden, M., Birman, K. P., and Constable, R. L. 1999. Building reliable, high-performance communication systems from components. In Proceeding of the 17th ACM Symposium on Operating Systems Principles. 80--92. Google Scholar
Digital Library
- McCanne, S. and Jacobson, V. 1993. The BSD Packet Filter: A new architecture for user-level packet capture. In Proceedings of the Winter USENIX Conference. Google Scholar
Digital Library
- McVoy, L. 1998. The splice I/O model. www.bitmover.com/lm/papers/splice.ps.Google Scholar
- Montz, A. B., Mosberger, D., O’Malley, S. W., Peterson, L. L., Proebsting, T. A., and Hartman, J. H. 1994. Scout: A communications-oriented operating system. Tech. rep. 9420, University of Arizona.Google Scholar
- NAPI. Linux napi, or “new” network api. http://www.linuxfoundation.org/en/Net:NAPI.Google Scholar
- Nguyen, T., Cristea, M., de Bruijn, W., and Bos, H. 2004. Scalable network monitors for high-speed links: a bottom-up approach. In Proceedings of the IEEE Workshop on IP Operations and Management (IPOM).Google Scholar
- Pai, V. S., Druschel, P., and Zwaenepoel, W. 2000. Io-lite: A unified i/o buffering and caching system. ACM Trans. Comput. Syst. 18, 1, 37--66. Google Scholar
Digital Library
- Pasquale, J., Anderson, E. W., and Muller, K. 1994. Container shipping: Operating system support for i/o-intensive applications. IEEE Comput. 27, 3, 84--93. Google Scholar
Digital Library
- Polychronakis, M., Anagnostakis, K. G., and Markatos, E. P. 2007. Emulation-based detection of non-self-contained polymorphic shellcode. In Proceedings of the 10th International Symposium on Recent Advances in Intrusion Detection (RAID’07). 87--106. Google Scholar
Digital Library
- Presotto, D. and Winterbottom, P. 1993. The organization of networks in plan 9. In Proceedings of the USENIX Conference. Usenix.Google Scholar
- Pu, C., Autrey, T., Black, A., Consel, C., Cowan, C., Inouye, J., Kethana, L., Walpole, J., and Zhang, K. 1995. Optimistic incremental specialization: Streamlining a commercial operating system. In Proceedings of the 15th ACM Symposium on Operating Systems Principles. Google Scholar
Digital Library
- Reid, A., Flatt, M., Stoller, L., Lepreau, J., and Eide, E. 2000. Knit: Component composition for systems software. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). USENIX Association, 24--24. Google Scholar
Digital Library
- Ritchie, D. M. 1984. A stream input-output system. AT&T Bell Labo. Tech. J. 63, 8, 1897--1910.Google Scholar
Cross Ref
- Schapbach, A., Peter, S., Baumann, A., Roscoe, T., Barham, P., Harris, T., and Isaacs, R. 2008. Embracing diversity in the barrelfish manycore operating system. In Proceedings of the Workshop on Managed Many-Core Systems (MMCS’08).Google Scholar
- Sermulins, J., Thies, W., Rabbah, R., and Amarasinghe, S. 2005. Cache aware optimization of stream programs. SIGPLAN Not. 40, 7, 115--126. Google Scholar
Digital Library
- Sullivan, M. and Heybey, A. 1998. Tribeca: A system for managing large databases of network traffic. In Proceedings of the USENIX Conference. 13--24. Google Scholar
Digital Library
- Sun Microsystems, I. 2005. STREAMS Programming Guide. Sun Microsystems, Inc.Google Scholar
- Tanenbaum, A. S., van Renesse, R., van Staveren, H., Sharp, G. J., and Mullender, S. J. 1990. Experiences with the amoeba distributed operating system. Comm. ACM 33, 12, 46--63. Google Scholar
Digital Library
- Traw, C. B. S. and Smith, J. M. 1993. Hardware/software organization of a high-performance ATM host interface. D. Select. Areas Comm. 11, 2, 240--253.Google Scholar
Digital Library
- Veitch, A. C. 1998. A dynamically reconfigurable and extensibe operating system. Ph.D. thesis, Univ. of British Columbia. Google Scholar
Digital Library
- Welsh, M., Culler, D. E., and Brewer, E. A. 2001. Seda: An architecture for well-conditioned, scalable internet services. In Proceedings of the Symposium on Operating Systems Principles. 230--243. Google Scholar
Digital Library
- Wikipedia. 2008. Wikipedia statistics. http://en.wikipedia.org/wiki/Wikipedia:Statistics.Google Scholar
- Wulf, W. A. and McKee, S. A. 1995. Hitting the memory wall: Implications of the obvious. Comput. Archit. News 23, 1, 20--24. Google Scholar
Digital Library
Index Terms
Application-Tailored I/O with Streamline
Recommendations
IO-Lite: a unified I/O buffering and caching system
This article presents the design, implementation, and evaluation of IO -Lite, a unified I/O buffering and caching system for general-purpose operating systems. IO-Lite unifies all buffering and caching in the system, to the extent permitted by the ...
Development and performance study of a zero-copy file transfer mechanism for VIA-Based PC cluster systems
HPCC'06: Proceedings of the Second international conference on High Performance Computing and CommunicationsThis paper presents the development and implementation of a zero-copy file transfer mechanism that improves the efficiency of file transfers for PC cluster systems using hardware-based VIA (Virtual Interface Architecture) network adapters. VIA is one of ...
Design and implementation of an improved zero-copy file transfer mechanism
PDCAT'04: Proceedings of the 5th international conference on Parallel and Distributed Computing: applications and TechnologiesThis paper presents the design and implementation of an improved zero-copy file transfer mechanism that improves the efficiency of file transfers for Linux-based PC cluster systems. This mechanism, based on the Virtual Interface Architecture (VIA), ...






Comments