research-article

Open access

A large-scale study of MPI usage in open-source HPC applications

Authors:

Ignacio Laguna,

Kathryn Mohror,

Martin Ruefenacht,

Anthony Skjellum,

Nawrin SultanaAuthors Info & Claims

SC '19: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 31, Pages 1 - 14

https://doi.org/10.1145/3295500.3356176

Published: 17 November 2019 Publication History

Abstract

Understanding the state-of-the-practice in MPI usage is paramount for many aspects of supercomputing, including optimizing the communication of HPC applications and informing standardization bodies and HPC systems procurements regarding the most important MPI features. Unfortunately, no previous study has characterized the use of MPI on applications at a significant scale; previous surveys focus either on small data samples or on MPI jobs of specific HPC centers. This paper presents the first comprehensive study of MPI usage in applications. We survey more than one hundred distinct MPI programs covering a significantly large space of the population of MPI applications. We focus on understanding the characteristics of MPI usage with respect to the most used features, code complexity, and programming models and languages. Our study corroborates certain findings previously reported on smaller data samples and presents a number of interesting, previously un-reported insights.

References

[1]

David E Bernholdt, Swen Boehm, George Bosilca, Manjunath Gorentla Venkata, Ryan E Grant, Thomas Naughton, Howard P Pritchard, Martin Schulz, and Geoffroy R Vallee. 2017. A survey of MPI usage in the US exascale computing project. Concurrency and Computation: Practice and Experience (2017), e4851.

[2]

S. Chunduri, S. Parker, P. Balaji, K. Harms, and K. Kumaran. 2018. Characterization of MPI Usage on a Production Supercomputer. In SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. 386--400.

Digital Library

[3]

Al Danial. 2019. CLOC: Count Lines of Code. https://github.com/AlDanial/cloc. Online; accessed Jan/10/2019.

[4]

H Carter Edwards, Christian R Trott, and Daniel Sunderland. 2014. Kokkos: Enabling manycore performance portability through polymorphic memory access patterns. J. Parallel and Distrib. Comput. 74, 12 (2014), 3202--3216.

Digital Library

[5]

Message Passing Interface Forum. 2015. MPI: A Message-passing Interface Standard, Version 3.1; June 4, 2015. High-Performance Computing Center Stuttgart, University of Stuttgart. https://books.google.com/books?id=Fbv7jwEACAAJ

[6]

Robert B Grady. 1992. Practical software metrics for project management and process improvement. Prentice-Hall, Inc.

[7]

Richard D Hornung and Jeffrey A Keasler. 2014. The RAJA portability layer: overview and status. Technical Report. Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States).

[8]

Benjamin Klenk and Holger Fröning. 2017. An overview of MPI characteristics of exascale proxy applications. In International Supercomputing Conference. Springer, 217--236.

[9]

Luke Nguyen-Hoan, Shayne Flint, and Ramesh Sankaranarayana. 2010. A Survey of Scientific Software Development. In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '10). ACM, New York, NY, USA, Article 12, 10 pages.

Digital Library

[10]

Prakash Prabhu, Hanjun Kim, Taewook Oh, Thomas B Jablin, Nick P Johnson, Matthew Zoufaly, Arun Raman, Feng Liu, David Walker, Yun Zhang, et al. 2011. A survey of the practice of computational science. In SC'11: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1--12.

Digital Library

[11]

L. Prechelt. 2000. An empirical comparison of seven programming languages. Computer 33, 10 (Oct 2000), 23--29.

Digital Library

[12]

Rolf Rabenseifner. 1999. Automatic MPI counter profiling of all users: First results on a CRAY T3E 900-512. In Proceedings of the message passing interface developer's and user's conference, Vol. 1999. 77--85.

[13]

Nawrin Sultana, Anthony Skjellum, Purushotham Bangalore, Ignacio Laguna, and Kathryn Mohror. 2018. Understanding the Usage of MPI in Exascale Proxy Applications. Workshop on Exascale MPI (ExaMPI). (2018).

[14]

Steven P VanderWiel, Daphna Nathanson, and David J Lilja. 1997. Complexity and performance in parallel programming languages. In Proceedings Second International Workshop on High-Level Parallel Programming Models and Supportive Environments. IEEE, 3--12.

[15]

H. Zhang. 2009. An investigation of the relationships between lines of code and defects. In 2009 IEEE International Conference on Software Maintenance. 274--283.

Cited By

Gu TFei JCanini M(2024)OmNICCL: Zero-cost Sparse AllReduce with Direct Cache Access and SmartNICsProceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing10.1145/3672198.3673804(75-83)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672198.3673804
Luczynski PGianinazzi LIff PWilson LDe Sensi DHoefler TMencagli GDazzi PLowenthal DBadia R(2024)Near-Optimal Wafer-Scale ReduceProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658693(334-347)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658693
Dong YDai YXie MLu KWang RChen JShao MWang Z(2024)Faster and Scalable MPI Applications LaunchingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.321807735:2(264-279)Online publication date: Feb-2024
https://doi.org/10.1109/TPDS.2022.3218077
Show More Cited By

Index Terms

A large-scale study of MPI usage in open-source HPC applications
1. Computing methodologies
  1. Parallel computing methodologies
    1. Parallel programming languages
2. General and reference
  1. Document types
    1. Surveys and overviews

Recommendations

Investigating the Usage of MPI at Argument-Granularity in HPC Codes
EuroMPI '23: Proceedings of the 30th European MPI Users' Group Meeting

This study focuses on gaining insights into the usage of the Message-Passing Interface (MPI) in a large set of High-Performance Computing (HPC) codes by analyzing MPI function calls and their argument usage patterns. Previous work has focused on ...
An empirical investigation on MPI open source applications
EASE '14: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering

Parallel programming is a computing paradigm that gains more and more adopters each year. Embracing parallel programming means in most of the cases the usage of a particular library that allows to add parallelism and concurrency to applications. ...
Optimization and Performance of a Fortran 90 MPI-Based Unstructured Code on Large-Scale Parallel Systems

The message-passing interface (MPI) has become the standard in achieving effective results when using the message passing paradigm of parallelization. Codes written using MPI are extremely portable and are applicable to both clusters and massively ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '19: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2019

1921 pages

ISBN:9781450362290

DOI:10.1145/3295500

General Chair:
Michela Taufer,
Program Chairs:
Pavan Balaji,
Antonio J. Peña

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Available

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SC '19

Sponsor:

SIGHPC

SC '19: The International Conference for High Performance Computing, Networking, Storage, and Analysis

November 17 - 19, 2019

Colorado, Denver

Acceptance Rates

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

51
Total Citations
View Citations
2,919
Total Downloads

Downloads (Last 12 months)767
Downloads (Last 6 weeks)94

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gu TFei JCanini M(2024)OmNICCL: Zero-cost Sparse AllReduce with Direct Cache Access and SmartNICsProceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing10.1145/3672198.3673804(75-83)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672198.3673804
Luczynski PGianinazzi LIff PWilson LDe Sensi DHoefler TMencagli GDazzi PLowenthal DBadia R(2024)Near-Optimal Wafer-Scale ReduceProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658693(334-347)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658693
Dong YDai YXie MLu KWang RChen JShao MWang Z(2024)Faster and Scalable MPI Applications LaunchingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.321807735:2(264-279)Online publication date: Feb-2024
https://doi.org/10.1109/TPDS.2022.3218077
Noh SHong JLim CPark SKim JKim HKim YLee J(2024)PID-Comm: A Fast and Flexible Collective Communication Framework for Commodity Processing-in-DIMM Devices2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00027(245-260)Online publication date: 29-Jun-2024
https://doi.org/10.1109/ISCA59077.2024.00027
Li ZChen ZTang YAi XZhu YZhao ZShao JLiu GLiu SLiu BXu Y(2024)MUSE: A Runtime Incrementally Reconfigurable Network Adapting to HPC Real-Time Traffic2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00073(765-779)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPS57955.2024.00073
Tran TKuncham GRamesh BXu SSubramoni HAbduljabbar MPanda D(2024)OHIO: Improving RDMA Network Scalability in MPI_Alltoall Through Optimized Hierarchical and Intra/Inter-Node Communication Overlap Design2024 IEEE Symposium on High-Performance Interconnects (HOTI)10.1109/HOTI63208.2024.00019(47-56)Online publication date: 21-Aug-2024
https://doi.org/10.1109/HOTI63208.2024.00019
Jammer TSaillard ESchwitanski SJenke JVinayagame RHück ABischof C(2024)MPI-BugBench: A Framework for Assessing MPI Correctness ToolsRecent Advances in the Message Passing Interface10.1007/978-3-031-73370-3_8(121-137)Online publication date: 25-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-73370-3_8
Jammer TSchmidt ABischof C(2024)Annotation of Compiler Attributes for MPI FunctionsRecent Advances in the Message Passing Interface10.1007/978-3-031-73370-3_2(21-35)Online publication date: 25-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-73370-3_2
Posner JGoebel RFinnerty P(2024)Evolving APGAS Programs: Automatic and Transparent Resource Adjustments at RuntimeAsynchronous Many-Task Systems and Applications10.1007/978-3-031-61763-8_15(154-165)Online publication date: 14-Feb-2024
https://dl.acm.org/doi/10.1007/978-3-031-61763-8_15
Roa Perdomo DCeccato RNeveu RYviquel HLi XMonsalve Diaz JDoerfert J(2023)Memory Transfer Decomposition: Exploring Smart Data Movement Through Architecture-Aware StrategiesProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624609(1958-1967)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624609
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents