skip to main content
research-article

SmartCon: SmartCon: Smart Context Switching for Fast Storage Devices

Published:20 March 2015Publication History
Skip Abstract Section

Abstract

Handling of storage IO in modern operating systems assumes that such devices are slow and CPU cycles are valuable. Consequently, to effectively exploit the underlying hardware resources, for example, CPU cycles, storage bandwidth and the like, whenever an IO request is issued to such device, the requesting thread is switched out in favor of another thread that may be ready to execute. Recent advances in nonvolatile storage technologies and multicore CPUs make both of these assumptions increasingly questionable, and an unconditional context switch is no longer desirable. In this article, we propose a novel mechanism called SmartCon, which intelligently decides whether to service a given IO request in interrupt-driven manner or busy-wait--based manner based on not only the device characteristics but also dynamic parameters such as IO latency, CPU utilization, and IO size. We develop an analytic performance model to project the performance of SmartCon for forthcoming devices. We implement SmartCon mechanism on Linux 2.6 and perform detailed evaluation using three different IO devices: Ramdisk, low-end SSD, and high-end SSD. We find that SmartCon yields up to a 39% performance gain over the mainstream block device approach for Ramdisk, and up to a 45% gain for PCIe-based SSD and SATA-based SSDs. We examine the detailed behavior of TLB, L1, L2 cache and show that SmartCon achieves significant improvement in all cache misbehaviors.

References

  1. Anant Agarwal, John Hennessy, and Mark Horowitz. 1988. Cache performance of operating system and multiprogramming workloads. ACM Trans. Comput. Syst. 6, 4, 393--431. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. M. R. Ahmadi and D. Maleki. 2010. Performance evaluation of server virtualization in data center applications. In Proceedings of the International Symposium on Telecommunications (IST’10).Google ScholarGoogle Scholar
  3. Ameen Akel, Adrian M. Caufield, Todor l. Mollov, Rajesh K. Gupta, and Steven Swanson. 2011. Onyx: A prototype phase change memory storage array. In Proceedings of USENX HotStorage. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Suparna Bhattacharya, John Tran, Mike Sullivan, and Chris Mason. 2004. Linux AIO performance and robustness for enterprise workloads. In Proceedings of the Linux Symposium.Google ScholarGoogle Scholar
  5. G. W. Burr, B. N. Kurdi, J. C. Scott, C. H. Lam, K. Gopalakrishnan, and R. S. Shenoy. 2008. Overview of candidate device technologies for storage-class memory. IBM Journal of Research and Development 52, 4--5, 449--464. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Adrian M. Caulfield, Arup De, Joel Coburn, Todor I. Mollow, Rajesh K. Gupta, and Steven Swanson. 2010. Moneta: A high-performance storage array architecture for next-generation, non-volatile memories. In Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’43). IEEE Computer Society, Washington, DC, 385--395. DOI: http://dx.doi.org/10.1109/MICRO.2010.33 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Feng Chen, Michael P. Mesnier, and Scott Hahn. 2014. A protected block device for persistent memory. In Proceedings of the 30th International Conference on Massive Storage Systems and Technology (MSST’14).Google ScholarGoogle ScholarCross RefCross Ref
  8. Joel Coburn, Adrian M. Caufield, Ameen Akel, Laura M. Grupp, Rajesh K. Gupta, Ranjit Jhala, and Steven Swanson. 2011. NV-Heaps: Making persistent objects fast and safe with next-generation, non-volatile memories. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J. Corbet, A. Rubini, and G. Kroah-Hartman. 2005. Linux Device Drivers. O’Reilly Media, Inc. Sebastopol, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Francis M. David, Jeffrey C. Carlyle, and Roy H. Campbell. 2007. Context switch overheads for Linux on ARM platforms. In Proceedings of the Workshop on Experimental Computer Science (ExpCS’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Khaled Elmeleegy, Anupam Chanda, and Alan L. Cox. 2004. Lazy asynchronous i/o for event-driven servers. In Proceedings of the USENIX Annual Technical Conference (ATC’04). Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. R. F. Freitas and W. W. Wilcke. 2008. Storage class memory: The next storage system technology. IBM Journal of Research and Development 52, 4/5, 439--447. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. FusionIO. 2010. ioDrive product family user guide—Linux for driver release 2.1.0.Google ScholarGoogle Scholar
  14. HenkPoley. 2014. A Look Back at Single-Threaded CPU Performance. Retrieved March 1, 2015 from http://preshing.com/20120208/a-look-back-at-single-threaded-cpu-performance/.Google ScholarGoogle Scholar
  15. M. Hosomi, H. Yamagishi, T. Yamamoto, K. Bessho, Y. Higo, K. Yamane, H. Yamada, M. Shoji, H. Hachino, C. Fukumoto, N. Nagao, and H. Kano. 2005. A novel nonvolatile memory with spin torque transfer magnetization switching: spin-ram. In Proceedings of the IEEE International Electron Devices Meeting. 459--462.Google ScholarGoogle Scholar
  16. A. Huffman. 2012. NVM express, revision 1.0 c. Intel Corporation.Google ScholarGoogle Scholar
  17. Intel. 2011. Light Peak Technology. Retrieved March 1, 2015 from http://en.wikipedia.org/wiki/Light_Peak.Google ScholarGoogle Scholar
  18. E. Ipek, J. Condit, B. Lee, E. B. Nightingale, D. Burger, C. Frost, and D. Coetzee. 2009. Better I/O through byte-addressable, persistent memory. In Proceedings of the ACM SIGOPS Symposium on Operating Systems Principles (SOSP’09). Big Sky, MT. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Sooman Jeong, Kisung Lee, Seongjin Lee, Seoungbum Son, and Youjip Won. 2013. I/O stack optimization for smartphones. In Proceedings of the 2013 USENIX Annual Technical Conference (USENIX ATC’13). USENIX, San Jose, CA, 309--320. Retrieved from https://www.usenix.org/conference/atc13/technical-sessions/presentation/jeong. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Ryan Johnson, Manos Athanassoulis, Radu Stoica, and Anastasia Ailamaki. 2009. A new look at the roles of spinning and blocking. In Proceedings of the International Workshop on Data Management on New Hardware (DaMoN’09). ACM, New York, NY, 21--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. Tim Jones. 2006. Linux Initial RAM Disk (initrd) Overview. Retrieved March 1, 2015 from https://www.ibm.com/developerworks/library/l-initrd.Google ScholarGoogle Scholar
  22. J. Jung, J. Choi, Y. Won, and S. Kang. 2009. Shadow block: Imposing block device abstraction on storage class memory. In Proceedings of the International Workshop on Software Support for Portable Storage (IWSSPS’09).Google ScholarGoogle Scholar
  23. J. Jung, Y. Won, E. Kim, H. Shin, and B. Jeon. 2010. FRASH: Exploiting storage class memory in hybrid file system for hierarchical storage. ACM Trans. on Storage 6, 1, 1--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. K. Kant. 2008. Exploiting NVRAM for building multi-level memory systems. In Proceedings of the International Workshop on OS Technologies for Large Scale NVRAM (NVRAMOS’08): Presentation.Google ScholarGoogle Scholar
  25. K. Kant and Y. Won. 1999. Server capacity planning for Web traffic workload. IEEE Transactions on Knowledge and Data Engineering 11, 5, 731--747. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Anna R. Karlin, Kai Li, Mark S. Manasse, and Susan Owicki. 1991. Empirical studies of competitive spinning for a shared-memory multiprocessor. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP’91). ACM, New York, NY, 41--55. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Baris Kasikci, Cristian Zamfir, and George Candea. 2013. RaceMob: Crowdsourced data race detection. In Proceedings of the 24th ACM Symposium on Operating Systems Principles. ACM, New York, NY, 406--422. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. Katcher. 1997. Postmark: A new file system benchmark. Network Appliance, Inc.Google ScholarGoogle Scholar
  29. M. Kobayashi. 1986. An empirical study of task switching locality in MVS. IEEE Trans. on Computers 100, 35, 720--731. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Philip Lantz, Subramanya Dulloor, Sanjay Kumar, Rajesh Sankaran, and Jeff Jackson. 2014. Yat: A validation framework for persistent memory software. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIX ATC 14). USENIX Association, Philadelphia, PA, 433--438. Retrieved from https://www.usenix.org/conference/atc14/technical-sessions/presentation/lantz. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Benjamin C. Lee, Engin Ipek, Onur Mutlu, and Doug Burger. 2009. Architecting phase change memory as a scalable dram alternative. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Chuanpeng Li, Chen Ding, and Kai Shen. 2007. Quantifying the cost of context switch. In Proceedings of the Workshop on Experimental Computer Science (ExpCS’07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Jing Li, Charles Augustine, Sayeef Salahuddin, and Kaushik Roy. 2008. Modeling of failure probability and statistical design of spin-torque transfer magnetic random access memory (STT-MRAM) array for yield enhancement. In Proceedings of the Design Automation Conference (DAC’08). 278--283. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Fang Liu, Fei Guo, Yan Solihin, Seongbeom Kim, and Abdulaziz Eker. 2008. Characterizing and modeling the behavior of context switch misses. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT’08). 91--101. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. R. McDougall and J. Mauro. 2005. FileBench. Retrieved March 1, 2015 from http://www.nfsv4bat.org/Documents/nasconf/2004/filebench.pdf.Google ScholarGoogle Scholar
  36. L. McVoy and C. Staelin. 1996. LMBENCH: Portable tools for performance analysis. In Proceedings of the USENIX Annual Technical Conference (ATC’96). 279--294. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Microsoft Corp, MSDN. 2010. RAMDisk Storage Driver Sample. Retrieved March 1, 2015 from https://code.msdn.microsoft.com/windowshardware/RAMDisk-Storage-Driver-9ce5f699.Google ScholarGoogle Scholar
  38. Kevin OBrien. 2013. OCZ RevoDrive 3 X2 480GB Review. Retrieved March 1, 2015 from http://www.storagereview.com/ocz_revodrive_3_x2_480gb_review.Google ScholarGoogle Scholar
  39. Doron Orenstien and Ronny Ronen. 2004. Low-power processor hint, such as from a PAUSE instruction.Google ScholarGoogle Scholar
  40. J. K. Ousterhout. 1990. Why are not operating systems getting faster as fast as hardware. In Proceedings of the Summer 1990 USENIX Conference. 247--256.Google ScholarGoogle Scholar
  41. S. Park and K. Shen. 2009. A performance evaluation of scientific i/o workloads on flash-based SSDs. In Proceedings of the Workshop on Interfaces and Architectures for Scientific Data Storage (IASDS’09).Google ScholarGoogle Scholar
  42. Moinuddin K. Qureshi, Michele M. Franceschini, Luis A. Lastras-Montaño, and John P. Karidis. 2010. Morphable memory system: A robust architecture for exploiting multi-level phase change memories. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA’10). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Moinuddin K. Qureshi, Vijayalakshmi Srinivasan, and Jude A. Revers. 2009. Scalable high performance main memory system using phase-change memory technology. In Proceedings of the ACM/IEEE International Symposium on Computer Architecture (ISCA’09). 24--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. K. Salah and A. Qahtan. 2009. Implementation and experimental performance evaluation of a hybrid interrupt-handling scheme. Computer Communications 32, 1, 179--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Samsung Electronics. June 2005. OneNAND Specification ver.1.2.Google ScholarGoogle Scholar
  46. Samsung Electronics. June 2007. 1Gb C-die DDR3 SDRAM specification.Google ScholarGoogle Scholar
  47. Patrick Schmid and Achim Roos. 2008. Intel X25-M Solid State Drive Reviewed. Retrieved March 1, 2015 from http://www.tomshardware.com/reviews/Intel-x25-m-SSD,2012.html.Google ScholarGoogle Scholar
  48. Johan Starner and Lars Asplund. 2004. Measuring the cache interference cost in preemptive real-time systems. In Proceedings of the 2004 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES’04). 146--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. D. Tam, R. Azimi, L. Soares, and M. Stumm. 2007. Managing shared L2 caches on multicore systems in software. In Proceedings of the Workshop on the Interaction between Operating Systems and Computer Architecture (WIOSCA’07).Google ScholarGoogle Scholar
  50. The Open Group Base Specifications Issues 6 IEEE Std 1003.1. 2003. Retrieved March 1, 2015 from http://www.opengroup.org/onlinepubs/007904975.Google ScholarGoogle Scholar
  51. Shivaram Venkataraman, Niraj Tolia, Parthasarathy Ranganathan, and Roy H. Campbell. 2011. Consistent and durable data structures for non-volatile byte-addressable memory. In Proceedings of the Usenix Conference on File and Storage Technologies (FAST’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. G. Venkatasubramanian, R. J. Figueiredo, R. Illikkal, and D. Newell. 2009. TMT-A TLB tag management framework for virtualized platforms. In Proc. of 21st Intl. Symp. on Computer Arch. and High. Perf. Computing. 153--160. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Haris Volos, Nadres Jann Tack, and Michael M. Swift. 2011. Mnemosyne: Lightweight persistent memory. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. R. P. Weicker. 1984. Dhrystone: a synthetic systems programming benchmark. Commun. ACM 27, 10, 1013--1030. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. A. Wiggins, H. Tuch, V. Uhlig, and G. Heiser. 2003. Implementation of fast address-space switching and TLB sharing on the StrongARM processor. In Proceedings of the 8th Asia-Pacific Computer Systems Architecture Conference (ACSAC’03).Google ScholarGoogle Scholar
  56. Xiaoxia Wu, Jian Li, Lixin Zhang, Evan Speight, Ram Rajamony, and Yuan Xie. 2009. Hybrid cache architecture with disparate memory technologies. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA’09). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. S. Yamada and S. Kusakabe. 2008. Effect of context aware scheduler on TLB. In Proceedings of the IEEE International Symposium on Parallel and Distributed Processing. 1--8.Google ScholarGoogle Scholar
  58. J. Yan and W. Zhang. 2008. WCET analysis for multi-core processors with shared L2 instruction caches. In Proceedings of the IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS’08). 80--89. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Jisoo Yang, Dave B. Minturn, and Frank Hady. 2012. When poll is better than interrupt. In Proceedings of the 10th USENIX Conference on FAST (FAST’12). 3. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. SmartCon: SmartCon: Smart Context Switching for Fast Storage Devices

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Storage
          ACM Transactions on Storage  Volume 11, Issue 2
          March 2015
          123 pages
          ISSN:1553-3077
          EISSN:1553-3093
          DOI:10.1145/2747982
          • Editor:
          • Darrell Long
          Issue’s Table of Contents

          Copyright © 2015 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 20 March 2015
          • Accepted: 1 May 2014
          • Revised: 1 February 2014
          • Received: 1 May 2013
          Published in tos Volume 11, Issue 2

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!