MRG8: Random Number Generation for the Exascale Era

ABSTRACT

Pseudo random number generators (PRNGs) are crucial for numerous applications in HPC ranging from molecular dynamics to quantum chemistry, and hydrodynamics. These applications require high throughput and good statistical quality from the PRNGs - especially for parallel computing where long pseudo-random sequences can be exhausted rapidly. Although a handful PRNGs have been adapted to parallel computing, they do not fully exploit the features of wide-SIMD many-core processors and GPU accelerators in modern supercomputers. Multiple Recursive Generators (MRGs) are a family of random number generators based on higher order recursion, which provide statistically high-quality random number sequences with extremely long-recurrence lengths, and deterministic jump-ahead for effective parallelism. We reformulate the MRG8 (8th-order recursive implementation) for Intel's KNL and NVIDIA's P100 GPU - named MRG8-AVX512 and MRG8-GPU respectively. Our optimized implementation generates the same random number sequence as the original well-characterized MRG8. We evaluated MRG8-AVX512 and MRG8-GPU together with vender tuned random number generators for Intel KNL and GPU. MRG8-AVX512 achieves a substantial 69% improvement compared to Intel's MKL, and MRG8-GPU shows a maximum 3.36x speedup compared to NVIDIA's cuRAND library.

References

1. L Yu Barash and Lev N Shchur. 2014. PRAND: GPU accelerated parallel random number generation library: Using most reliable algorithms and applying parallelism of modern GPUs and CPUs. Computer Physics Communications 185, 4 (2014), 1343--1353.
2. Lih-Yuan Deng and Dale Bowman. 2017. Developments in pseudo-random number generators. Wiley Interdisciplinary Reviews: Computational Statistics 9, 5 (2017), e1404-n/a.
3. WMC Foulkes, L Mitas, RJ Needs, and G Rajagopal. 2001. Quantum Monte Carlo simulations of solids. Reviews of Modern Physics 73, 1 (2001), 33.
4. James E Gentle. 2006. Random Number Generation and Monte Carlo Methods. Springer Science & Business Media.
5. Brian L Hammond, William A Lester Jr, and Peter James Reynolds. 1994. Monte Carlo Methods In Ab Initio Quantum Chemistry. World Scientific.
6. Peter Hellekalek. 1998. Good random number generators are (not so) easy to find. Mathematics and Computers in Simulation 46, 5 (1998), 485--505.
7. Kenta Hongo, Ryo Maezono, and Kenichi Miura. 2010. Random number generators tested on quantum Monte Carlo simulations. Journal of computational chemistry 31, 11 (2010), 2186--2194.
8. Donald E Knuth. 1997. The art of computer programming, 3rd edn. seminumerical algorithms, vol. 2. (1997).
9. Pierre L'Ecuyer. 1996. Combined Multiple Recursive Random Number Generators. Operations Research 44, 5 (1996), 816--822.
10. Pierre L'ecuyer. 1999. Good Parameters and Implementations for Combined Multiple Recursive Random Number Generators. Operations Research 47, 1 (1999), 159--164.
11. Pierre L'Ecuyer, David Munger, Boris Oreshkin, and Richard Simard. 2016. Random numbers for parallel computers: Requirements and methods, with emphasis on GPUs. (06 2016).
12. Pierre L'Ecuyer and Richard Simard. 2007. TestU01: A C library for empirical testing of random number generators. ACM Transactions on Mathematical Software (TOMS) 33, 4 (2007), 22.
13. George Marsaglia et al. 2003. Xorshift RNGs. Journal of Statistical Software 8, 14 (2003), 1--6.
14. Makoto Matsumoto and Takuji Nishimura. 1998. Dynamic Creation of Pseudorandom Number Generators. Monte Carlo and Quasi-Monte Carlo Methods 2000 (1998), 56--69.
15. Makoto Matsumoto and Takuji Nishimura. 1998. Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Transactions on Modeling and Computer Simulation (TOMACS) 8, 1 (1998), 3--30.
16. Harald Niederreiter. 1992. Random Number Generation and Quasi-Monte Carlo Methods. SIAM.
17. Mutsuo Saito and Makoto Matsumoto. 2008. SIMD-Oriented Fast Mersenne Twister: a 128-bit Pseudorandom Number Generator. In Monte Carlo and Quasi-Monte Carlo Methods 2006. Springer, 607--622.
18. Mutsuo Saito and Makoto Matsumoto. 2013. Variants of Mersenne Twister Suitable for Graphic Processors. ACM Transactions on Mathematical Software (TOMS) 39, 2 (2013), 12.
19. John K Salmon, Mark A Moraes, Ron O Dror, and David E Shaw. 2011. Parallel Random Numbers: As Easy as 1, 2, 3. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2011. IEEE, 1--12.

Index Terms

• Mathematics of computing

  • Probability and statistics

    • Probabilistic reasoning algorithms

      • Random number generation