research-article

Network-accelerated non-contiguous memory transfers

Authors:

Salvatore Di Girolamo,

Konstantin Taranov,

Michael Schaffner,

Timo Schneider,

Jakub Beránek,

Torsten HoeflerAuthors Info & Claims

SC '19: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 56, Pages 1 - 14

https://doi.org/10.1145/3295500.3356189

Published: 17 November 2019 Publication History

Abstract

Applications often communicate data that is non-contiguous in the send- or the receive-buffer, e.g., when exchanging a column of a matrix stored in row-major order. While non-contiguous transfers are well supported in HPC (e.g., MPI derived datatypes), they can still be up to 5x slower than contiguous transfers of the same size. As we enter the era of network acceleration, we need to investigate which tasks to offload to the NIC: In this work we argue that non-contiguous memory transfers can be transparently network-accelerated, truly achieving zero-copy communications. We implement and extend sPIN, a packet streaming processor, within a Portals 4 NIC SST model, and evaluate strategies for NIC-offloaded processing of MPI datatypes, ranging from datatype-specific handlers to general solutions for any MPI datatype. We demonstrate up to 8x speedup in the unpack throughput of real applications, demonstrating that non-contiguous memory transfers are a first-class candidate for network acceleration.

References

[1]

T. Shanley. 2003. Infiniband Network Architecture. Addison-Wesley Professional.

[2]

2019. Mellanox Technologies. http://http://www.mellanox.com/. (2019).

[3]

B. Alverson, et al. 2012. Cray XC series network. Cray Inc., White Paper WP-Aries01-1112 (2012).

[4]

B. W Barrett, et al. 2018. The Portals 4.2 Network Programming Interface. Sandia National Laboratories, November 2012, Technical Report SAND2012-10087 (2018).

[5]

T. Schneider, et al. 2013. Protocols for Fully Offloaded Collective Operations on Accelerated Network Adapters. In Parallel Processing (ICPP), 2013 42nd International Conference on. 593--602.

Digital Library

[6]

S. Di Girolamo, et al. 2016. Exploiting Offload Enabled Network Interfaces. IEEE MICRO 36, 4 (Jul. 2016).

Digital Library

[7]

T. Schneider, R. Gerstenberger, and T. Hoefler. 2012. Micro-Applications for Communication Data Access Patterns and MPI Datatypes. In Recent Advances in the Message Passing Interface - Proceedings of the 19th European MPI Users' Group Meeting, EuroMPI 2012, 2012., Vol. 7490. Springer, 121--131.

[8]

T. Schneider, R. Gerstenberger, and T. Hoefler. 2014. Application-oriented ping-pong benchmarking: how to assess the real communication overheads. Journal of Computing 96, 4 (Apr. 2014), 279--292.

[9]

T. Hoefler and S. Gottlieb. 2010. Parallel Zero-Copy Algorithms for Fast Fourier Transform and Conjugate Gradient using MPI Datatypes. In Recent Advances in the Message Passing Interface (EuroMPI'10), Vol. LNCS 6305. Springer, 132--141.

[10]

W. Gropp, et al. 2011. Performance Expectations and Guidelines for MPI Derived Datatypes. In Recent Advances in the Message Passing Interface (EuroMPI'11), Vol. 6960. Springer, 150--159.

[11]

T. Schneider, F. Kjolstad, and T. Hoefler. 2013. MPI Datatype Processing using Runtime Compilation. In Proceedings of the 20th European MPI Users' Group Meeting. ACM, 19--24.

[12]

G. Santhanaraman, J. Wu, and D. K Panda. 2004. Zero-copy MPI derived datatype communication over InfiniBand. In European Parallel Virtual Machine/Message Passing Interface Users' Group Meeting. Springer, 47--56.

[13]

H. Wang, et al. 2011. Optimized non-contiguous MPI datatype communication for GPU clusters: Design, implementation and evaluation with MVAPICH2. In 2011 IEEE International Conference on Cluster Computing. IEEE, 308--316.

Digital Library

[14]

T. Hoefler, et al. 2017. sPIN: High-performance streaming Processing in the Network. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC17).

Digital Library

[15]

R. F Van der Wijngaart and P. Wong. 2002. NAS parallel benchmarks version 2.4. (2002).

[16]

J. Nieplocha, et al. 2006. High performance remote memory access communication: The ARMCI approach. The International Journal of High Performance Computing Applications 20, 2 (2006), 233--253.

Digital Library

[17]

B. Chapman, et al. 2010. Introducing OpenSHMEM: SHMEM for the PGAS community. In Proceedings of the Fourth Conference on Partitioned Global Address Space Programming Model. ACM, 2.

Digital Library

[18]

J. Mellor-Crummey, et al. 2009. A new vision for Coarray Fortran. In Proceedings of the Third Conference on Partitioned Global Address Space Programing Models. ACM, 5.

Digital Library

[19]

T. El-Ghazawi and L. Smith. 2006. UPC: unified parallel C. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing. ACM, 27.

[20]

Message Passing Interface Forum. 2012. MPI: A Message-Passing Interface Standard Version 3.0. (09 2012). Chapter author for Collective Communication, Process Topologies, and One Sided Communications.

[21]

W. Gropp, E. Lusk, and D. Swider. 1999. Improving the performance of MPI derived datatypes. In Proceedings of the Third MPI Developer's and User's Conference. MPI Software Technology Press, 25--30.

[22]

S. Byna, et al. 2006. Automatic memory optimizations for improving MPI derived datatype performance. In European Parallel Virtual Machine/Message Passing Interface Users' Group Meeting. Springer, 238--246.

Digital Library

[23]

N. Tanabe and H. Nakajo. 2008. Introduction to acceleration for MPI derived datatypes using an enhancer of memory and network. In European Parallel Virtual Machine/Message Passing Interface Users' Group Meeting. Springer, 324--325.

[24]

J. L. Träff. 2014. Optimal MPI Datatype Normalization for Vector and Index-block Types. In Proceedings of the 21st European MPI Users' Group Meeting (EuroMPI/ASIA '14). ACM, New York, NY, USA, Article 33, 6 pages.

Digital Library

[25]

R. Ross, et al. 2009. Processing MPI datatypes outside MPI. In European Parallel Virtual Machine/Message Passing Interface Users' Group Meeting. Springer, 42--53.

Digital Library

[26]

R. Ross, N. Miller, and W. D Gropp. 2003. Implementing fast and reusable datatype processing. In European Parallel Virtual Machine/Message Passing Interface Users' Group Meeting. Springer, 404--413.

[27]

A. Kurth, et al. 2017. HERO: Heterogeneous embedded research platform for exploring RISC-V manycore accelerators on FPGA. arXiv preprint arXiv:1712.06497 (2017).

[28]

D. Rossi, et al. 2017. Energy-Efficient Near-Threshold Parallel Computing: The PULPv2 Cluster. IEEE Micro 37, 5 (Sep. 2017), 20--31.

[29]

M. Gautschi, et al. 2017. Near-Threshold RISC-V Core With DSP Extensions for Scalable IoT Endpoint Devices. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 25, 10 (Oct 2017), 2700--2713.

[30]

Mellanox Technologies. 2019. Mellanox BlueField SmartNIC. http://www.mellanox.com/related-docs/prod_adapter_cards/PB_BlueField_Smart_NIC.pdf. (2019). Online; accessed 05. April 2019.

[31]

H. T. Mair, et al. 2016. 4.3 A 20nm 2.5GHz ultra-low-power tri-cluster CPU subsystem with adaptive power allocation for optimal mobile SoC performance. In IEEE International Solid-State Circuits Conference (ISSCC). 76--77.

[32]

J. Pyo, et al. 2015. 23.1 20nm high-K metal-gate heterogeneous 64b quad-core CPUs and hexa-core GPU for high-performance and energy-efficient mobile application processor. In 2015 IEEE International Solid-State Circuits Conference - (ISSCC) Digest of Technical Papers. 1--3.

[33]

R. Sohan, et al. 2010. Characterizing 10 Gbps network interface energy consumption. In IEEE Local Computer Network Conference. 268--271.

Digital Library

[34]

C. L Janssen, et al. 2010. A simulator for large-scale parallel computer architectures. International Journal of Distributed Systems and Technologies (IJDST) 1, 2 (2010), 57--73.

Digital Library

[35]

Nathan Binkert, et al. 2011. The gem5 simulator. ACM SIGARCH Computer Architecture News 39, 2 (2011), 1--7.

Digital Library

[36]

F. A Endo, D. Couroussé, and H. Charles. 2014. Micro-architectural simulation of in-order and out-of-order arm microprocessors with gem5. In 2014 International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS XIV). IEEE, 266--273.

[37]

A. Tousi and C. Zhu. 2017. Arm Research Starter Kit: System Modeling using gem5. (2017).

[38]

T. Hoefler, T. Schneider, and A. Lumsdaine. 2010. LogGOPSim - Simulating Large-Scale Applications in the LogGOPS Model. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing. ACM, 597--604.

[39]

Lawrence Livermore National Laboratory. 2018. Comb is a communication performance benchmarking tool. (2018). https://github.com/LLNL/Comb

[40]

S. Plimpton. 1995. Fast Parallel Algorithms for Short-range Molecular Dynamics. J. Comput. Phys. 117, 1 (March 1995), 1--19.

Digital Library

[41]

C. Bernard, et al. 1991. Studying quarks and gluons on MIMD parallel computers. The International Journal of Supercomputing Applications 5, 4 (1991), 61--70.

Digital Library

[42]

L. Carrington, et al. 2008. High-frequency Simulations of Global Seismic Wave Propagation Using SPECFEM3D GLOBE on 62K Processors. In Proceedings of the 2008 ACM/IEEE Conference on Supercomputing (SC '08). IEEE Press, Piscataway, NJ, USA, Article 60, 11 pages. http://dl.acm.org/citation.cfm?id=1413370.1413432

Digital Library

[43]

B Sjogreen. 2018. SW4 final report for iCOE. Technical Report. Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States).

[44]

W. C. Skamarock and J. B. Klemp. 2008. A Time-split Nonhydrostatic Atmospheric Model for Weather Research and Forecasting Applications. J. Comput. Phys. 227, 7 (March 2008), 3465--3485.

Digital Library

[45]

R. Neugebauer, et al. 2018. Understanding PCIe performance for end host networking. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. ACM, 327--341.

Digital Library

[46]

M. Martinasso, et al. 2016. A PCIe Congestion-Aware Performance Model for Densely Populated Accelerator Servers. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC16). IEEE Press, 63:1--63:11.

Digital Library

[47]

T. Hoefler, C. Siebert, and A. Lumsdaine. 2009. Group Operation Assembly Language - A Flexible Way to Express Collective Communication, In ICPP-2009 - The 38th International Conference on Parallel Processing. (Sep. 2009).

Digital Library

[48]

J. L. Träff, et al. 1999. Flattening on the Fly: efficient handling of MPI derived datatypes. In Recent Advances in Parallel Virtual Machine and Message Passing Interface, Jack Dongarra, Emilio Luque, and Tomàs Margalef (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 109--116.

[49]

T. Prabhu and W. Gropp. 2015. DAME: A runtime-compiled engine for derived datatypes. In Proceedings of the 22nd European MPI Users' Group Meeting. ACM, 4.

[50]

T. Schneider, R. Gerstenberger, and T. Hoefler. 2013. Compiler optimizations for non-contiguous remote data movement. In International Workshop on Languages and Compilers for Parallel Computing. Springer, 307--321.

[51]

TV Eicken, et al. 1992. Active messages: a mechanism for integrated communication and computation. In 1992 Proceedings the 19th Annual International Symposium on Computer Architecture. IEEE, 256--266.

[52]

M. Besta and T. Hoefler. 2015. Accelerating Irregular Computations with Hardware Transactional Memory and Active Messages. In Proceedings of the 24th Symposium on High-Performance Parallel and Distributed Computing (HPDC'15). ACM, 161--172.

[53]

M. Besta and T. Hoefler. 2015. Active Access: A Mechanism for High-Performance Distributed Data-Centric Computations. In Proceedings of the 29th International Conference on Supercomputing (ICS'15). ACM, 155--164.

Cited By

De Sensi DCosta Molero EDi Girolamo SVanbever LHoefler T(2024)Canary: Congestion-aware in-network allreduce using dynamic treesFuture Generation Computer Systems10.1016/j.future.2023.10.010152(70-82)Online publication date: Mar-2024
https://doi.org/10.1016/j.future.2023.10.010
Besta MGerstenberger RPeter EFischer MPodstawski MBarthels CAlonso GHoefler T(2023)Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph QueriesACM Computing Surveys10.1145/360493256:2(1-40)Online publication date: 15-Sep-2023
https://dl.acm.org/doi/10.1145/3604932
Zhang TZhou HHuang CTian CZhang WWang XWang YAbdelmoniem ATan MDou WChen G(2023)Achieving Zero-copy Serialization for Datacenter RPC2023 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC59175.2023.10253859(304-312)Online publication date: 17-Nov-2023
https://doi.org/10.1109/IPCCC59175.2023.10253859
Show More Cited By

Index Terms

Network-accelerated non-contiguous memory transfers
1. Networks
  1. Network architectures
    1. Programming interfaces

Recommendations

Non-contiguous processor allocation algorithms for distributed memory multicomputers
Supercomputing '94: Proceedings of the 1994 ACM/IEEE conference on Supercomputing

Current processor allocation techniques for highly parallel systems have thus far been restricted to contiguous allocation strategies for which performance suffers significantly due to the inherent problem of fragmentation. We are investigating ...
On contiguous and non-contiguous parallel task scheduling

In this paper we study differences between contiguous and non-contiguous parallel task schedules. Parallel tasks can be executed on more than one processor simultaneously. In the contiguous schedules, indices of the processors assigned to a task must be ...
Classification of Software Transfers
APSEC '12: Proceedings of the 2012 19th Asia-Pacific Software Engineering Conference - Volume 01

Many companies have development sites around the globe. This inevitably means that development work may be transferred between the sites. This paper defines a classification of software transfer types, it divides transfers into three main types: full, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '19: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2019

1921 pages

ISBN:9781450362290

DOI:10.1145/3295500

General Chair:
Michela Taufer,
Program Chairs:
Pavan Balaji,
Antonio J. Peña

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

European Science Foundation
European Research Council (ERC)

Conference

SC '19

Sponsor:

SIGHPC

SC '19: The International Conference for High Performance Computing, Networking, Storage, and Analysis

November 17 - 19, 2019

Colorado, Denver

Acceptance Rates

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
480
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)7

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

De Sensi DCosta Molero EDi Girolamo SVanbever LHoefler T(2024)Canary: Congestion-aware in-network allreduce using dynamic treesFuture Generation Computer Systems10.1016/j.future.2023.10.010152(70-82)Online publication date: Mar-2024
https://doi.org/10.1016/j.future.2023.10.010
Besta MGerstenberger RPeter EFischer MPodstawski MBarthels CAlonso GHoefler T(2023)Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph QueriesACM Computing Surveys10.1145/360493256:2(1-40)Online publication date: 15-Sep-2023
https://dl.acm.org/doi/10.1145/3604932
Zhang TZhou HHuang CTian CZhang WWang XWang YAbdelmoniem ATan MDou WChen G(2023)Achieving Zero-copy Serialization for Datacenter RPC2023 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC59175.2023.10253859(304-312)Online publication date: 17-Nov-2023
https://doi.org/10.1109/IPCCC59175.2023.10253859
Nölp KOden L(2023)Simplifying non-contiguous data transfer with MPI for PythonThe Journal of Supercomputing10.1007/s11227-023-05398-779:17(20019-20040)Online publication date: 7-Jun-2023
https://doi.org/10.1007/s11227-023-05398-7
Cao SDi Girolamo SHoefler T(2022)Accelerating Data Serialization/Deserialization Protocols with In-Network Compute2022 IEEE/ACM International Workshop on Exascale MPI (ExaMPI)10.1109/ExaMPI56604.2022.00008(22-30)Online publication date: Nov-2022
https://doi.org/10.1109/ExaMPI56604.2022.00008
Glines MPirgov PMullin LKhan R(2022)Strided DMA for Multidimensional Array Copy and TransposeIntelligent Computing10.1007/978-3-031-10461-9_26(375-393)Online publication date: 7-Jul-2022
https://doi.org/10.1007/978-3-031-10461-9_26
Besta MFischer MKalavri VKapralov MHoefler T(2021)Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.3131677(1-1)Online publication date: 2021
https://doi.org/10.1109/TPDS.2021.3131677
Besta MDomke JSchneider MKonieczny MGirolamo SSchneider TSingla AHoefler T(2021)High-Performance Routing With Multipathing and Path Diversity in Ethernet and HPC NetworksIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2020.303576132:4(943-959)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TPDS.2020.3035761
Di Girolamo SKurth ACalotoiu ABenz TSchneider TBeranek JBenini LHoefler T(2021)A RISC-V in-network accelerator for flexible high-performance low-power packet processing2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA52012.2021.00079(958-971)Online publication date: Jun-2021
https://doi.org/10.1109/ISCA52012.2021.00079
Suresh KRamesh BChen CGhazimirsaeed SBayatpour MShafi ASubramoni HPanda D(2021)Layout-aware Hardware-assisted Designs for Derived Data Types in MPI2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC53243.2021.00044(302-311)Online publication date: Dec-2021
https://doi.org/10.1109/HiPC53243.2021.00044
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents