Abstract
The steady increase of parallelism in high-performance computing platforms implies that communication will be most important in large-scale applications. In this work, we tackle the problem of transparent optimization of large-scale communication patterns using online compilation techniques. We utilize the Group Operation Assembly Language (GOAL), an abstract parallel dataflow definition language, to specify our transformations in a device-independent manner. We develop fast schemes that analyze dataflow and synchronization semantics in GOAL and detect if parts of the (or the whole) communication pattern express a known collective communication operation. The detection of collective operations allows us to replace the detected patterns with highly optimized algorithms or low-level hardware calls and thus improve performance significantly. Benchmark results suggest that our technique can lead to a performance improvement of orders of magnitude compared with various optimized algorithms written in Co-Array Fortran. Detecting collective operations also improves the programmability of parallel languages in that the user does not have to understand the detailed semantics of high-level communication operations in order to generate efficient and scalable code.
- G. Bronevetsky. Communication-sensitive static dataflow for parallel message passing applications. In Proc. of the 7th IEEE/ACM Intl. Symp. on Code Generation and Optimization, CGO '09, pages 1--12, 2009. Google Scholar
Digital Library
- C. Coarfa, Y. Dotsenko, J. Mellor-Crummey, F. Cantonnet, T. El-Ghazawi, A. Mohanti, Y. Yao, and D. Chavarría-Miranda. An evaluation of global address space languages: co-array fortran and unified parallel c. In Proc. of the tenth ACM SIGPLAN Symp. on Princ. and Practice of Par. Progr., PPoPP '05, pages 36--47. ACM, 2005. Google Scholar
Digital Library
- T. Hoefler, C. Siebert, and A. Lumsdaine. Group Operation Assembly Language - A Flexible Way to Express Collective Communication. In 38th Intl. Conf. on Par. Proc., ICPP'09, 2009. Google Scholar
Digital Library
- A. Knüpfer, D. Kranzlmüller, and W. E. Nagel. Detection of Collective MPI Operation Patterns. In Proc. of EuroPVM/MPI'04, volume 3241 of LNCS, pages 259--267. 2004.Google Scholar
- R. W. Numrich and J. Reid. Co-array fortran for parallel programming. SIGPLAN Fortran Forum, 17: 1--31, August 1998. ISSN 1061--7264. Google Scholar
Digital Library
- R. Preissl, T. Köckerbauer, M. Schulz, D. Kranzlmüller, B. R. d. Supinski, and D. J. Quinlan. Detecting Patterns in MPI Communication Traces. In Proc. of the 37th Intl. Conf. on Par. Proc., ICPP'08, pages 230--237, 2008. Google Scholar
Digital Library
- UPC Consortium. UPC Language Specifications, v1.2. Technical report, Lawrence Berkeley National Laboratory, 2005. LBNL-59208.Google Scholar
Index Terms
Communication-centric optimizations by dynamically detecting collective operations
Recommendations
Communication-centric optimizations by dynamically detecting collective operations
PPoPP '12: Proceedings of the 17th ACM SIGPLAN symposium on Principles and Practice of Parallel ProgrammingThe steady increase of parallelism in high-performance computing platforms implies that communication will be most important in large-scale applications. In this work, we tackle the problem of transparent optimization of large-scale communication ...
Optimization of Collective Communication Operations in MPICH
We describe our work on improving the performance of collective communication operations in MPICH for clusters connected by switched networks. For each collective operation, we use multiple algorithms depending on the message size, with the goal of ...
Optimization of MPI collective communication on BlueGene/L systems
ICS '05: Proceedings of the 19th annual international conference on SupercomputingBlueGene/L is currently the world's fastest supercomputer. It consists of a large number of low power dual-processor compute nodes interconnected by high speed torus and collective networks, Because compute nodes do not have shared memory, MPI is the ...







Comments