skip to main content

An approach to generate correctly rounded math libraries for new floating point variants

Published:04 January 2021Publication History
Skip Abstract Section

Abstract

Given the importance of floating point (FP) performance in numerous domains, several new variants of FP and its alternatives have been proposed (e.g., Bfloat16, TensorFloat32, and posits). These representations do not have correctly rounded math libraries. Further, the use of existing FP libraries for these new representations can produce incorrect results. This paper proposes a novel approach for generating polynomial approximations that can be used to implement correctly rounded math libraries. Existing methods generate polynomials that approximate the real value of an elementary function 𝑓 (𝑥) and produce wrong results due to approximation errors and rounding errors in the implementation. In contrast, our approach generates polynomials that approximate the correctly rounded value of 𝑓 (𝑥) (i.e., the value of 𝑓 (𝑥) rounded to the target representation). It provides more margin to identify efficient polynomials that produce correctly rounded results for all inputs. We frame the problem of generating efficient polynomials that produce correctly rounded results as a linear programming problem. Using our approach, we have developed correctly rounded, yet faster, implementations of elementary functions for multiple target representations.

References

  1. Denis Arzelier, Florent Bréhard, and Mioara Joldes. 2019. Exchange Algorithm for Evaluation and Approximation ErrorOptimized Polynomials. In 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH). 30-37. https://doi.org/10.1109/ ARITH. 2019.00014 Google ScholarGoogle ScholarCross RefCross Ref
  2. Florian Benz, Andreas Hildebrandt, and Sebastian Hack. 2012. A Dynamic Program Analysis to Find Floating-point Accuracy Problems. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (Beijing, China) ( PLDI '12). ACM, New York, NY, USA, 453-462. https://doi.org/10.1145/2345156.2254118 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Jeremy Bernstein, Jiawei Zhao, Markus Meister, Ming-Yu Liu, Anima Anandkumar, and Yisong Yue. 2020. Learning compositional functions via multiplicative weight updates. arXiv: 2006. 14560 [cs.NE]Google ScholarGoogle Scholar
  4. Sylvie Boldo, Marc Daumas, and Ren-Cang Li. 2009. Formally Verified Argument Reduction with a Fused Multiply-Add. In IEEE Transactions on Computers, Vol. 58. 1139-1145. https://doi.org/10.1109/TC. 2008.216 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Peter Borwein and Tamas Erdelyi. 1995. Polynomials and Polynomial Inequalities. Springer New York. https://doi.org/10. 1007/978-1-4612-0793-1 Google ScholarGoogle ScholarCross RefCross Ref
  6. Nicolas Brisebarre and Sylvvain Chevillard. 2007. Eficient polynomial L-approximations. In 18th IEEE Symposium on Computer Arithmetic (ARITH '07). https://doi.org/10.1109/ARITH. 2007.17 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Nicolas Brisebarre, Jean-Michel Muller, and Arnaud Tisserand. 2006. Computing Machine-Eficient Polynomial Approximations. In ACM ACM Transactions on Mathematical Software, Vol. 32. Association for Computing Machinery, New York, NY, USA, 236-256. https://doi.org/10.1145/1141885.1141890 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Nicolas Brunie, Florent de Dinechin, Olga Kupriianova, and Christoph Lauter. 2015. Code Generators for Mathematical Functions. In 2015 IEEE 22nd Symposium on Computer Arithmetic. 66-73. https://doi.org/10.1109/ARITH. 2015.22 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Hung Tien Bui and Sofiene Tahar. 1999. Design and synthesis of an IEEE-754 exponential function. In Engineering Solutions for the Next Millennium. 1999 IEEE Canadian Conference on Electrical and Computer Engineering, Vol. 1. 450-455 vol. 1. https://doi.org/10.1109/CCECE. 1999.807240 Google ScholarGoogle ScholarCross RefCross Ref
  10. Sylvain Chevillard, John Harrison, Mioara Joldes, and Christoph Lauter. 2011. Eficient and accurate computation of upper bounds of approximation errors. Theoretical Computer Science 412. https://doi.org/10.1016/j.tcs. 2010. 11.052 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Sylvain Chevillard, Mioara Joldes, and Christoph Lauter. 2010. Sollya: An Environment for the Development of Numerical Codes. In Mathematical Software-ICMS 2010 (Lecture Notes in Computer Science, Vol. 6327 ). Springer, Heidelberg, Germany, 28-31. https://doi.org/10.1007/978-3-642-15582-6_5 Google ScholarGoogle ScholarCross RefCross Ref
  12. Sylvain Chevillard and Christopher Lauter. 2007. A Certified Infinite Norm for the Implementation of Elementary Functions. In Seventh International Conference on Quality Software (QSIC 2007 ). 153-160. https://doi.org/10.1109/QSIC. 2007.4385491 Google ScholarGoogle ScholarCross RefCross Ref
  13. Sangeeta Chowdhary, Jay P. Lim, and Santosh Nagarakatte. 2020. Debugging and Detecting Numerical Errors in Computation with Posits. In 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'20). https: //doi.org/10.1145/3385412.3386004 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. William J Cody and William M Waite. 1980. Software manual for the elementary functions. Prentice-Hall, Englewood Clifs, NJ.Google ScholarGoogle Scholar
  15. Mike Cowlishaw. 2008. IEEE Standard for Floating-Point Arithmetic. IEEE 754-2008. IEEE Computer Society. 1-70 pages. https://doi.org/10.1109/IEEESTD. 2008.4610935 Google ScholarGoogle ScholarCross RefCross Ref
  16. Nasrine Damouche and Matthieu Martel. 2018. Salsa: An Automatic Tool to Improve the Numerical Accuracy of Programs. In Automated Formal Methods (Kalpa Publications in Computing, Vol. 5 ), Natarajan Shankar and Bruno Dutertre (Eds.). 63-76. https://doi.org/10.29007/j2fd Google ScholarGoogle ScholarCross RefCross Ref
  17. Catherine Daramy, David Defour, Florent Dinechin, and Jean-Michel Muller. 2003. CR-LIBM: A correctly rounded elementary function library. In Proceedings of SPIE Vol. 5205 : Advanced Signal Processing Algorithms, Architectures, and Implementations XIII, Vol. 5205. https://doi.org/10.1117/12.505591 Google ScholarGoogle ScholarCross RefCross Ref
  18. Marc Daumas, Guillaume Melquiond, and Cesar Munoz. 2005. Guaranteed proofs using interval arithmetic. In 17th IEEE Symposium on Computer Arithmetic (ARITH'05). 188-195. https://doi.org/10.1109/ARITH. 2005.25 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Florent de Dinechin, Christopher Lauter, and Guillaume Melquiond. 2011. Certifying the Floating-Point Implementation of an Elementary Function Using Gappa. In IEEE Transactions on Computers, Vol. 60. 242-253. https://doi.org/10.1109/TC. 2010.128 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Florent de Dinechin, Christoph Quirin Lauter, and Guillaume Melquiond. 2006. Assisted Verification of Elementary Functions Using Gappa. In Proceedings of the 2006 ACM Symposium on Applied Computing (Dijon, France) (SAC '06). Association for Computing Machinery, New York, NY, USA, 1318-1322. https://doi.org/10.1145/1141277.1141584 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Laurent Fousse, Guillaume Hanrot, Vincent Lefèvre, Patrick Pélissier, and Paul Zimmermann. 2007. MPFR: A Multipleprecision Binary Floating-point Library with Correct Rounding. ACM Trans. Math. Software 33, 2, Article 13 ( June 2007 ). https://doi.org/10.1145/1236463.1236468 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Zhoulai Fu and Zhendong Su. 2019. Efective Floating-point Analysis via Weak-distance Minimization. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (Phoenix, AZ, USA) ( PLDI 2019). ACM, New York, NY, USA, 439-452. https://doi.org/10.1145/3314221.3314632 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Ambros Gleixner, Daniel E. Stefy, and Kati Wolter. 2015. Iterative Refinement for Linear Programming. Technical Report 15-15. ZIB, Takustr. 7, 14195 Berlin. https://doi.org/10.1287/ijoc. 2016.0692 Google ScholarGoogle Scholar
  24. Ambros M. Gleixner, Daniel E. Stefy, and Kati Wolter. 2012. Improving the Accuracy of Linear Programming Solvers with Iterative Refinement. In Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation (Grenoble, France) (ISSAC '12). Association for Computing Machinery, New York, NY, USA, 187-194. https://doi.org/10. 1145/2442829.2442858 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Eric Goubault. 2001. Static Analyses of the Precision of Floating-Point Operations. In Proceedings of the 8th International Symposium on Static Analysis (SAS). Springer, 234-259. https://doi.org/10.1007/3-540-47764-0_14 Google ScholarGoogle ScholarCross RefCross Ref
  26. John Gustafson. 2017. Posit Arithmetic. https://posithub.org/docs/Posits4.pdfGoogle ScholarGoogle Scholar
  27. John Gustafson. 2020. The Minefield Method: A Uniformly Fast Solution to the Table-Maker's Dilemma. https://bit.ly/2ZP4kHjGoogle ScholarGoogle Scholar
  28. John Gustafson and Isaac Yonemoto. 2017. Beating Floating Point at Its Own Game: Posit Arithmetic. Supercomputing Frontiers and Innovations: an International Journal 4, 2 ( June 2017 ), 71-86. https://doi.org/10.14529/jsfi170206 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. John Harrison. 1997a. Floating point verification in HOL light: The exponential function. In Algebraic Methodology and Software Technology, Michael Johnson (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 246-260. https: //doi.org/10.1007/BFb0000475 Google ScholarGoogle ScholarCross RefCross Ref
  30. John Harrison. 1997b. Verifying the Accuracy of Polynomial Approximations in HOL. In International Conference on Theorem Proving in Higher Order Logics. https://doi.org/10.1007/BFb0028391 Google ScholarGoogle ScholarCross RefCross Ref
  31. John Harrison. 2009. HOL Light: An Overview. In Proceedings of the 22nd International Conference on Theorem Proving in Higher Order Logics, TPHOLs 2009 (Lecture Notes in Computer Science, Vol. 5674 ), Stefan Berghofer, Tobias Nipkow, Christian Urban, and Makarius Wenzel (Eds.). Springer-Verlag, Munich, Germany, 60-66. https://doi.org/10.1007/978-3-642-03359-9_4 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. IBM. 2008. Accurate Portable MathLib. http://oss.software.ibm.com/mathlib/Google ScholarGoogle Scholar
  33. Intel. 2019. Delivering a New Intelligence with AI at Scale. https://www.intel.com/content/www/us/en/artificial-intelligence/ posts/nnp-aisummit.htmlGoogle ScholarGoogle Scholar
  34. Claude-Pierre Jeannerod, Hervé Knochel, Christophe Monat, and Guillaume Revy. 2011. Computing Floating-Point Square Roots via Bivariate Polynomial Evaluation. IEEE Trans. Comput. 60. https://doi.org/10.1109/TC. 2010.152 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Jef Johnson. 2018. Rethinking floating point for deep learning. http://export.arxiv.org/abs/ 1811.01721Google ScholarGoogle Scholar
  36. William Kahan. 2004. A Logarithm Too Clever by Half. https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXTGoogle ScholarGoogle Scholar
  37. Dhiraj D. Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. 2019. A Study of BFLOAT16 for Deep Learning Training. arXiv: 1905.12322Google ScholarGoogle Scholar
  38. Olga Kupriianova and Christoph Lauter. 2014. Metalibm: A Mathematical Functions Code Generator. In 4th International Congress on Mathematical Software. https://doi.org/10.1007/978-3-662-44199-2_106 Google ScholarGoogle ScholarCross RefCross Ref
  39. Wonyeol Lee, Rahul Sharma, and Alex Aiken. 2017. On Automatically Proving the Correctness of Math.h Implementations. Proceedings of the ACM on Programming Languages 2, POPL, Article 47 ( Dec. 2017 ), 32 pages. https://doi.org/10.1145/ 3158135 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Vincent Lefèvre and Jean-Michel Muller. 2001. Worst Cases for Correct Rounding of the Elementary Functions in Double Precision. In 15th IEEE Symposium on Computer Arithmetic (Arith '01). 111-118. https://doi.org/10.1109/ARITH. 2001. 930110 Google ScholarGoogle ScholarCross RefCross Ref
  41. Vincent Lefèvre, Jean-Michel Muller, and Arnaud Tisserand. 1998. Toward correctly rounded transcendentals. IEEE Trans. Comput. 47, 11 ( 1998 ), 1235-1243. https://doi.org/10.1109/12.736435 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Cerlane Leong. 2019. SoftPosit-Math. https://gitlab.com/cerlane/softposit-mathGoogle ScholarGoogle Scholar
  43. Jay P. Lim, Mridul Aanjaneya, John Gustafson, and Santosh Nagarakatte. 2020a. A Novel Approach to Generate Correctly Rounded Math Libraries for New Floating Point Representations. arXiv: 2007. 05344 [cs.MS]Google ScholarGoogle Scholar
  44. Jay P. Lim and Santosh Nagarakatte. 2020a. RLibm. https://github.com/rutgers-apl/rlibmGoogle ScholarGoogle Scholar
  45. Jay P. Lim and Santosh Nagarakatte. 2020b. RLibm-generator. https://github.com/rutgers-apl/rlibm-generatorGoogle ScholarGoogle Scholar
  46. Jay P. Lim, Matan Shachnai, and Santosh Nagarakatte. 2020b. Approximating Trigonometric Functions for Posits Using the CORDIC Method. In Proceedings of the 17th ACM International Conference on Computing Frontiers (Catania, Sicily, Italy) ( CF '20). Association for Computing Machinery, New York, NY, USA, 19-28. https://doi.org/10.1145/3387902.3392632 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Guillaume Melquiond. 2019. Gappa. http://gappa.gforge.inria.frGoogle ScholarGoogle Scholar
  48. Sun Microsystems. 2008. LIBMCR 3 " 16 February 2008 " "libmcr-0.9". http://www.math.utah.edu/cgi-bin/man2html.cgi?/usr/ local/man/man3/libmcr.3Google ScholarGoogle Scholar
  49. Jean-Michel Muller. 2005. Elementary Functions: Algorithms and Implementation. Birkhauser. https://doi.org/10.1007/978-1-4899-7983-4 Google ScholarGoogle ScholarCross RefCross Ref
  50. NVIDIA. 2020. TensorFloat-32 in the A100 GPU Accelerates AI Training, HPC up to 20x. https://blogs.nvidia.com/blog/2020/ 05/14/tensorfloat-32-precision-format/Google ScholarGoogle Scholar
  51. Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015. Automatically Improving Accuracy for Floating Point Expressions. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Vol. 50. Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/ 2813885.2737959 Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Eugene Remes. 1934. Sur un procédé convergent d'approximations successives pour déterminer les polynômes d'approximation. Comptes rendus de l' Académie des Sciences 198 ( 1934 ), 2063-2065.Google ScholarGoogle Scholar
  53. Alex Sanchez-Stern, Pavel Panchekha, Sorin Lerner, and Zachary Tatlock. 2018. Finding Root Causes of Floating Point Error. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (Philadelphia, PA, USA) ( PLDI 2018). ACM, New York, NY, USA, 256-269. https://doi.org/10.1145/3296979.3192411 Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Joe Sawada. 2002. Formal verification of divide and square root algorithms using series calculation. In 3rd International Workshop on the ACL2 Theorem Prover and its Applications.Google ScholarGoogle Scholar
  55. Giuseppe Tagliavini, Stefan Mach, Davide Rossi, Andrea Marongiu, and Luca Benin. 2018. A transprecision floating-point platform for ultra-low power computing. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE). 1051-1056. https://doi.org/10.23919/DATE. 2018.8342167 Google ScholarGoogle ScholarCross RefCross Ref
  56. Ping-Tak Peter Tang. 1990. Table-Driven Implementation of the Logarithm Function in IEEE Floating-Point Arithmetic. ACM Trans. Math. Software 16, 4 (Dec. 1990 ), 378-400. https://doi.org/10.1145/98267.98294 Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Lloyd N. Trefethen. 2012. Approximation Theory and Approximation Practice (Other Titles in Applied Mathematics). Society for Industrial and Applied Mathematics, USA.Google ScholarGoogle Scholar
  58. Shibo Wang and Pankaj Kanwar. 2019. BFloat16: The secret to high performance on Cloud TPUs. https://cloud.google.com/ blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpusGoogle ScholarGoogle Scholar
  59. Xin Yi, Liqian Chen, Xiaoguang Mao, and Tao Ji. 2019. Eficient Automated Repair of High Floating-Point Errors in Numerical Libraries. Proceedings of the ACM on Programming Languages 3, POPL, Article 56 ( Jan. 2019 ), 29 pages. https://doi.org/10.1145/3290369 Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Abraham Ziv. 1991. Fast Evaluation of Elementary Mathematical Functions with Correctly Rounded Last Bit. ACM Trans. Math. Software 17, 3 (Sept. 1991 ), 410-423. https://doi.org/10.1145/114697.116813 Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Daming Zou, Muhan Zeng, Yingfei Xiong, Zhoulai Fu, Lu Zhang, and Zhendong Su. 2019. Detecting Floating-Point Errors via Atomic Conditions. Proceedings of the ACM on Programming Languages 4, POPL, Article 60 ( Dec. 2019 ), 27 pages. https://doi.org/10.1145/3371128 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An approach to generate correctly rounded math libraries for new floating point variants

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!