skip to main content
poster

Automatic datatype generation and optimization

Authors Info & Claims
Published:25 February 2012Publication History
Skip Abstract Section

Abstract

Many high performance applications spend considerable time packing noncontiguous data into contiguous communication buffers. MPI Datatypes provide an alternative by describing noncontiguous data layouts. This allows sophisticated hardware to retrieve data directly from application data structures. However, packing codes in real-world applications are often complex and specifying equivalent datatypes is difficult, time-consuming, and error prone. We present an algorithm that automates the transformation. We have implemented the algorithm in a tool that transforms packing code to MPI Datatypes, and evaluated it by transforming 90 packing codes from the NAS Parallel Benchmarks. The transformation allows easy porting of applications to new machines that benefit from datatypes, thus improving programmer productivity.

References

  1. Automatic datatype generation tool download. URL http://people.csail.mit.edu/fred/datatypes. Last Accessed 12/17/2011.Google ScholarGoogle Scholar
  2. D. Bailey, T. Harris, W. Saphir, R. Wijngaart, A. Woo, and M. Yarrow. The NAS parallel benchmarks 2.0. Technical report, NASA, 1995.Google ScholarGoogle Scholar
  3. F. Kjolstad, T. Hoefler, and M. Snir. A transformation to convert packing code to compact datatypes for efficient zero-copy data transfer. Technical report, University of Illinois at Urbana-Champaign, 2011.Google ScholarGoogle Scholar
  4. R. Ross, N. Miller, and W. Gropp. Implementing fast and reusable datatype processing. In EuroPVM/MPI, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  5. G. Santhanaraman, J. Wu, and D. K. Panda. Zero-copy MPI derived datatype communication over InfiniBand. In EuroPVM/MPI, 2004.Google ScholarGoogle ScholarCross RefCross Ref
  6. W. Tansey and E. Tilevich. Efficient automated marshaling of C++ data structures for MPI applications. In IPDPS, 2008.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Automatic datatype generation and optimization

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 47, Issue 8
        PPOPP '12
        August 2012
        334 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/2370036
        Issue’s Table of Contents
        • cover image ACM Conferences
          PPoPP '12: Proceedings of the 17th ACM SIGPLAN symposium on Principles and Practice of Parallel Programming
          February 2012
          352 pages
          ISBN:9781450311601
          DOI:10.1145/2145816

        Copyright © 2012 Authors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 25 February 2012

        Check for updates

        Qualifiers

        • poster

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!