skip to main content
research-article

Comparative evaluation of overlap strategies with study of I/O overlap in MPI-IO

Published:01 October 2008Publication History
Skip Abstract Section

Abstract

Many scientific applications use parallel I/O to meet the low latency and high bandwidth I/O requirement. Among many available parallel I/O operations, collective I/O is one of the most popular methods when the storage layouts and access patterns of data do not match. The implementation of collective I/O typically involves disk I/O operations followed by interprocessor communications. Also, in many I/O-intensive applications, parallel I/O operations are usually followed by parallel computations. This paper presents a comparative study of different overlap strategies in parallel applications. We have experimented with four different overlap strategies 1) Overlapping I/O and communication; 2) Overlapping I/O and computation; 3) Overlapping computation and communication; and 4) Overlapping I/O, communication, and computation. All experiments have been conducted on a Linux Cluster and the performance results obtained are very encouraging. On an average, we have enhanced the performance of a generic collective read call by 38%, the MxM benchmark by 26%, and the FFT benchmark by 34%.

References

  1. Caglar, Benson, Huang, and Chu. Usfmpi: A multi-threaded implementation of mpi for linux clusters. In Proc's of the 15th Inter. Conf. on Paral. and Dist. Comp. and Sys., pages 92--103, 2003.Google ScholarGoogle Scholar
  2. Carns, Ligon, Ross, and Thakur. Pvfs: A parallel file system for linux clusters. In Proc's of the 4th Annual Linux Showcase and Conf., pages 317--327, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Caron, Desprez, and Suter. Overlapping computations and communications with i/o in wavefront algorithms. Technical Report RR-5410, Institut National de Recherche en Informatique et en Automatique (INRIA), 2004.Google ScholarGoogle Scholar
  4. Choudhary, Bordawekar, More, and Sivaram. Passion runtime library for the intel paragon. In Proc's of the Intel Supercomputer User's Group Conf., pages 119--128, 1995.Google ScholarGoogle Scholar
  5. Danalis, Kim, Pollock, and Swany. Transformations to parallel codes for communication-computation overlap. In SC '05: Proc's of the 2005 ACM/IEEE conf. on Supercomputing, page 58, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Dickens and Thakur. Improving collective i/o performance using threads. 13th Inter. and 10th Symp. on Paral. and Dist. Processing, pages 38--45, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Dickens and Thakur. Evaluation of collective i/o implementations on parallel architectures. J. Paral. Dist. Comp., 61(8):1052--1076, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Fernandez, Frachtenberg, and Petrini. Bcs-mpi: A new approach in the system software design for large-scale parallel computers. In SC'03, page 57, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Foster, Kesselman, and Tuecke. The nexus task-parallel runtime system. In Proc's of the 1st Inter. Workshop on Paral. Proc., 1994.Google ScholarGoogle Scholar
  10. Gropp and Thakur. Issues in developing a thread-safe mpi implementation. In Proc's of the 13th European PVM/MPI Users' Group Meeting, volume 4192, pages 12--21, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Hoefler, Squyres, Rehm, and Lumsdaine. A case for non-blocking collective operations. In Book Frontiers of High Perf. Comp. and Networking ISPA Workshops, volume 4331, pages 155--164, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Hsu and Smith. The performance impact of i/o optimizations and disk improvements. IBM J. Res. Dev., 48(2):255--289, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Kandemir and Choudhary. Compiler-directed i/o optimization. In IPDPS '02: Proc's of the 16th Inter. Symp. on Paral. and Dist. Proc., page 19.2, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Kandemir, Choudhary, and Ramanujam. An i/o-conscious tiling strategy for disk-resident data sets. J. Super., 21(3):257--284, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Kotz. Disk-directed i/o for an out-of-core computation. In HPDC '95: Proc's of the 4th IEEE Inter. Symp. on High Perf. Dist. Comp., page 159, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Krempel. Tracing the connections between mpi-io calls and their corresponding pvfs2 disk operations. Bachelor's thesis, Ruprecht-Karls Universitt Heidelberg, 2006.Google ScholarGoogle Scholar
  17. Message Passing Interface Forum. MPI-2: Extensions to the Message Passing Interface. 1997.Google ScholarGoogle Scholar
  18. More, Choudhary, Foster, and Xu. Mtio - a multi-threaded parallel i/o system. In IPPS '97, pages 368--373, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Patrick, Son, and Kandemir. Enhancing the performance of mpi-io applications by overlapping i/o, computation and communication. In PPoPP '08, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Ross, Thakur, and Choudhary. Achievements and challenges for i/o in computational science. J. of Physics: Conf. Series, 16:501--509, 2005.Google ScholarGoogle ScholarCross RefCross Ref
  21. Sancho, Barker, Kerbyson, and Davis. Quantifying the potential benefit of overlapping communication and computation in large-scale scientific applications. In SC '06, page 125, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Seamons, Chen, Jones, Jozwiak, and Winslett. Server-directed collective i/o in panda. In Supercomputing '95, page 57, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Sur, Jin, Chai, and Panda. Rdma read based rendezvous protocol for mpi over infiniband: design alternatives and benefits. In PPoPP '06: Proc's of the 11th ACM SIGPLAN symp. on Principles and practice of paral. prog., pages 32--39, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Thakur, Gropp, and Lusk. Data sieving and collective i/o in romio. In FRONTIERS '99, page 182, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Thakur, Gropp, and Lusk. On implementing mpi-io portably and with high performance. In Proc's of the 6th workshop on I/O in paral. and dist. sys., pages 23--32, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Thakur, Gropp, and Lusk. Optimizing noncontiguous accesses in mpi-io. Paral. Comp., 28(1):83--105, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Thakur, Lusk, and Gropp. I/o in parallel applications: The weakest link. The Inter. J. of High Perf. Comp. Appls., 12(4):389--395, 1998.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Thakur, Lusk, and Gropp. Users guide for romio: A high-performance, portable mpi-io implementation, 2002.Google ScholarGoogle Scholar
  29. Tsujita. Effective nonblocking mpi-i/o in remote i/o operations using a multithreaded mechanism. Technical report, 2004Google ScholarGoogle Scholar

Index Terms

  1. Comparative evaluation of overlap strategies with study of I/O overlap in MPI-IO

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Published in

              cover image ACM SIGOPS Operating Systems Review
              ACM SIGOPS Operating Systems Review  Volume 42, Issue 6
              October 2008
              111 pages
              ISSN:0163-5980
              DOI:10.1145/1453775
              Issue’s Table of Contents

              Copyright © 2008 Authors

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 1 October 2008

              Check for updates

              Qualifiers

              • research-article

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!