Abstract
Given the importance of floating point (FP) performance in numerous domains, several new variants of FP and its alternatives have been proposed (e.g., Bfloat16, TensorFloat32, and posits). These representations do not have correctly rounded math libraries. Further, the use of existing FP libraries for these new representations can produce incorrect results. This paper proposes a novel approach for generating polynomial approximations that can be used to implement correctly rounded math libraries. Existing methods generate polynomials that approximate the real value of an elementary function 𝑓 (𝑥) and produce wrong results due to approximation errors and rounding errors in the implementation. In contrast, our approach generates polynomials that approximate the correctly rounded value of 𝑓 (𝑥) (i.e., the value of 𝑓 (𝑥) rounded to the target representation). It provides more margin to identify efficient polynomials that produce correctly rounded results for all inputs. We frame the problem of generating efficient polynomials that produce correctly rounded results as a linear programming problem. Using our approach, we have developed correctly rounded, yet faster, implementations of elementary functions for multiple target representations.
- Denis Arzelier, Florent Bréhard, and Mioara Joldes. 2019. Exchange Algorithm for Evaluation and Approximation ErrorOptimized Polynomials. In 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH). 30-37. https://doi.org/10.1109/ ARITH. 2019.00014 Google Scholar
Cross Ref
- Florian Benz, Andreas Hildebrandt, and Sebastian Hack. 2012. A Dynamic Program Analysis to Find Floating-point Accuracy Problems. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (Beijing, China) ( PLDI '12). ACM, New York, NY, USA, 453-462. https://doi.org/10.1145/2345156.2254118 Google Scholar
Digital Library
- Jeremy Bernstein, Jiawei Zhao, Markus Meister, Ming-Yu Liu, Anima Anandkumar, and Yisong Yue. 2020. Learning compositional functions via multiplicative weight updates. arXiv: 2006. 14560 [cs.NE]Google Scholar
- Sylvie Boldo, Marc Daumas, and Ren-Cang Li. 2009. Formally Verified Argument Reduction with a Fused Multiply-Add. In IEEE Transactions on Computers, Vol. 58. 1139-1145. https://doi.org/10.1109/TC. 2008.216 Google Scholar
Digital Library
- Peter Borwein and Tamas Erdelyi. 1995. Polynomials and Polynomial Inequalities. Springer New York. https://doi.org/10. 1007/978-1-4612-0793-1 Google Scholar
Cross Ref
- Nicolas Brisebarre and Sylvvain Chevillard. 2007. Eficient polynomial L-approximations. In 18th IEEE Symposium on Computer Arithmetic (ARITH '07). https://doi.org/10.1109/ARITH. 2007.17 Google Scholar
Digital Library
- Nicolas Brisebarre, Jean-Michel Muller, and Arnaud Tisserand. 2006. Computing Machine-Eficient Polynomial Approximations. In ACM ACM Transactions on Mathematical Software, Vol. 32. Association for Computing Machinery, New York, NY, USA, 236-256. https://doi.org/10.1145/1141885.1141890 Google Scholar
Digital Library
- Nicolas Brunie, Florent de Dinechin, Olga Kupriianova, and Christoph Lauter. 2015. Code Generators for Mathematical Functions. In 2015 IEEE 22nd Symposium on Computer Arithmetic. 66-73. https://doi.org/10.1109/ARITH. 2015.22 Google Scholar
Digital Library
- Hung Tien Bui and Sofiene Tahar. 1999. Design and synthesis of an IEEE-754 exponential function. In Engineering Solutions for the Next Millennium. 1999 IEEE Canadian Conference on Electrical and Computer Engineering, Vol. 1. 450-455 vol. 1. https://doi.org/10.1109/CCECE. 1999.807240 Google Scholar
Cross Ref
- Sylvain Chevillard, John Harrison, Mioara Joldes, and Christoph Lauter. 2011. Eficient and accurate computation of upper bounds of approximation errors. Theoretical Computer Science 412. https://doi.org/10.1016/j.tcs. 2010. 11.052 Google Scholar
Digital Library
- Sylvain Chevillard, Mioara Joldes, and Christoph Lauter. 2010. Sollya: An Environment for the Development of Numerical Codes. In Mathematical Software-ICMS 2010 (Lecture Notes in Computer Science, Vol. 6327 ). Springer, Heidelberg, Germany, 28-31. https://doi.org/10.1007/978-3-642-15582-6_5 Google Scholar
Cross Ref
- Sylvain Chevillard and Christopher Lauter. 2007. A Certified Infinite Norm for the Implementation of Elementary Functions. In Seventh International Conference on Quality Software (QSIC 2007 ). 153-160. https://doi.org/10.1109/QSIC. 2007.4385491 Google Scholar
Cross Ref
- Sangeeta Chowdhary, Jay P. Lim, and Santosh Nagarakatte. 2020. Debugging and Detecting Numerical Errors in Computation with Posits. In 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'20). https: //doi.org/10.1145/3385412.3386004 Google Scholar
Digital Library
- William J Cody and William M Waite. 1980. Software manual for the elementary functions. Prentice-Hall, Englewood Clifs, NJ.Google Scholar
- Mike Cowlishaw. 2008. IEEE Standard for Floating-Point Arithmetic. IEEE 754-2008. IEEE Computer Society. 1-70 pages. https://doi.org/10.1109/IEEESTD. 2008.4610935 Google Scholar
Cross Ref
- Nasrine Damouche and Matthieu Martel. 2018. Salsa: An Automatic Tool to Improve the Numerical Accuracy of Programs. In Automated Formal Methods (Kalpa Publications in Computing, Vol. 5 ), Natarajan Shankar and Bruno Dutertre (Eds.). 63-76. https://doi.org/10.29007/j2fd Google Scholar
Cross Ref
- Catherine Daramy, David Defour, Florent Dinechin, and Jean-Michel Muller. 2003. CR-LIBM: A correctly rounded elementary function library. In Proceedings of SPIE Vol. 5205 : Advanced Signal Processing Algorithms, Architectures, and Implementations XIII, Vol. 5205. https://doi.org/10.1117/12.505591 Google Scholar
Cross Ref
- Marc Daumas, Guillaume Melquiond, and Cesar Munoz. 2005. Guaranteed proofs using interval arithmetic. In 17th IEEE Symposium on Computer Arithmetic (ARITH'05). 188-195. https://doi.org/10.1109/ARITH. 2005.25 Google Scholar
Digital Library
- Florent de Dinechin, Christopher Lauter, and Guillaume Melquiond. 2011. Certifying the Floating-Point Implementation of an Elementary Function Using Gappa. In IEEE Transactions on Computers, Vol. 60. 242-253. https://doi.org/10.1109/TC. 2010.128 Google Scholar
Digital Library
- Florent de Dinechin, Christoph Quirin Lauter, and Guillaume Melquiond. 2006. Assisted Verification of Elementary Functions Using Gappa. In Proceedings of the 2006 ACM Symposium on Applied Computing (Dijon, France) (SAC '06). Association for Computing Machinery, New York, NY, USA, 1318-1322. https://doi.org/10.1145/1141277.1141584 Google Scholar
Digital Library
- Laurent Fousse, Guillaume Hanrot, Vincent Lefèvre, Patrick Pélissier, and Paul Zimmermann. 2007. MPFR: A Multipleprecision Binary Floating-point Library with Correct Rounding. ACM Trans. Math. Software 33, 2, Article 13 ( June 2007 ). https://doi.org/10.1145/1236463.1236468 Google Scholar
Digital Library
- Zhoulai Fu and Zhendong Su. 2019. Efective Floating-point Analysis via Weak-distance Minimization. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (Phoenix, AZ, USA) ( PLDI 2019). ACM, New York, NY, USA, 439-452. https://doi.org/10.1145/3314221.3314632 Google Scholar
Digital Library
- Ambros Gleixner, Daniel E. Stefy, and Kati Wolter. 2015. Iterative Refinement for Linear Programming. Technical Report 15-15. ZIB, Takustr. 7, 14195 Berlin. https://doi.org/10.1287/ijoc. 2016.0692 Google Scholar
- Ambros M. Gleixner, Daniel E. Stefy, and Kati Wolter. 2012. Improving the Accuracy of Linear Programming Solvers with Iterative Refinement. In Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation (Grenoble, France) (ISSAC '12). Association for Computing Machinery, New York, NY, USA, 187-194. https://doi.org/10. 1145/2442829.2442858 Google Scholar
Digital Library
- Eric Goubault. 2001. Static Analyses of the Precision of Floating-Point Operations. In Proceedings of the 8th International Symposium on Static Analysis (SAS). Springer, 234-259. https://doi.org/10.1007/3-540-47764-0_14 Google Scholar
Cross Ref
- John Gustafson. 2017. Posit Arithmetic. https://posithub.org/docs/Posits4.pdfGoogle Scholar
- John Gustafson. 2020. The Minefield Method: A Uniformly Fast Solution to the Table-Maker's Dilemma. https://bit.ly/2ZP4kHjGoogle Scholar
- John Gustafson and Isaac Yonemoto. 2017. Beating Floating Point at Its Own Game: Posit Arithmetic. Supercomputing Frontiers and Innovations: an International Journal 4, 2 ( June 2017 ), 71-86. https://doi.org/10.14529/jsfi170206 Google Scholar
Digital Library
- John Harrison. 1997a. Floating point verification in HOL light: The exponential function. In Algebraic Methodology and Software Technology, Michael Johnson (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 246-260. https: //doi.org/10.1007/BFb0000475 Google Scholar
Cross Ref
- John Harrison. 1997b. Verifying the Accuracy of Polynomial Approximations in HOL. In International Conference on Theorem Proving in Higher Order Logics. https://doi.org/10.1007/BFb0028391 Google Scholar
Cross Ref
- John Harrison. 2009. HOL Light: An Overview. In Proceedings of the 22nd International Conference on Theorem Proving in Higher Order Logics, TPHOLs 2009 (Lecture Notes in Computer Science, Vol. 5674 ), Stefan Berghofer, Tobias Nipkow, Christian Urban, and Makarius Wenzel (Eds.). Springer-Verlag, Munich, Germany, 60-66. https://doi.org/10.1007/978-3-642-03359-9_4 Google Scholar
Digital Library
- IBM. 2008. Accurate Portable MathLib. http://oss.software.ibm.com/mathlib/Google Scholar
- Intel. 2019. Delivering a New Intelligence with AI at Scale. https://www.intel.com/content/www/us/en/artificial-intelligence/ posts/nnp-aisummit.htmlGoogle Scholar
- Claude-Pierre Jeannerod, Hervé Knochel, Christophe Monat, and Guillaume Revy. 2011. Computing Floating-Point Square Roots via Bivariate Polynomial Evaluation. IEEE Trans. Comput. 60. https://doi.org/10.1109/TC. 2010.152 Google Scholar
Digital Library
- Jef Johnson. 2018. Rethinking floating point for deep learning. http://export.arxiv.org/abs/ 1811.01721Google Scholar
- William Kahan. 2004. A Logarithm Too Clever by Half. https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXTGoogle Scholar
- Dhiraj D. Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, Jiyan Yang, Jongsoo Park, Alexander Heinecke, Evangelos Georganas, Sudarshan Srinivasan, Abhisek Kundu, Misha Smelyanskiy, Bharat Kaul, and Pradeep Dubey. 2019. A Study of BFLOAT16 for Deep Learning Training. arXiv: 1905.12322Google Scholar
- Olga Kupriianova and Christoph Lauter. 2014. Metalibm: A Mathematical Functions Code Generator. In 4th International Congress on Mathematical Software. https://doi.org/10.1007/978-3-662-44199-2_106 Google Scholar
Cross Ref
- Wonyeol Lee, Rahul Sharma, and Alex Aiken. 2017. On Automatically Proving the Correctness of Math.h Implementations. Proceedings of the ACM on Programming Languages 2, POPL, Article 47 ( Dec. 2017 ), 32 pages. https://doi.org/10.1145/ 3158135 Google Scholar
Digital Library
- Vincent Lefèvre and Jean-Michel Muller. 2001. Worst Cases for Correct Rounding of the Elementary Functions in Double Precision. In 15th IEEE Symposium on Computer Arithmetic (Arith '01). 111-118. https://doi.org/10.1109/ARITH. 2001. 930110 Google Scholar
Cross Ref
- Vincent Lefèvre, Jean-Michel Muller, and Arnaud Tisserand. 1998. Toward correctly rounded transcendentals. IEEE Trans. Comput. 47, 11 ( 1998 ), 1235-1243. https://doi.org/10.1109/12.736435 Google Scholar
Digital Library
- Cerlane Leong. 2019. SoftPosit-Math. https://gitlab.com/cerlane/softposit-mathGoogle Scholar
- Jay P. Lim, Mridul Aanjaneya, John Gustafson, and Santosh Nagarakatte. 2020a. A Novel Approach to Generate Correctly Rounded Math Libraries for New Floating Point Representations. arXiv: 2007. 05344 [cs.MS]Google Scholar
- Jay P. Lim and Santosh Nagarakatte. 2020a. RLibm. https://github.com/rutgers-apl/rlibmGoogle Scholar
- Jay P. Lim and Santosh Nagarakatte. 2020b. RLibm-generator. https://github.com/rutgers-apl/rlibm-generatorGoogle Scholar
- Jay P. Lim, Matan Shachnai, and Santosh Nagarakatte. 2020b. Approximating Trigonometric Functions for Posits Using the CORDIC Method. In Proceedings of the 17th ACM International Conference on Computing Frontiers (Catania, Sicily, Italy) ( CF '20). Association for Computing Machinery, New York, NY, USA, 19-28. https://doi.org/10.1145/3387902.3392632 Google Scholar
Digital Library
- Guillaume Melquiond. 2019. Gappa. http://gappa.gforge.inria.frGoogle Scholar
- Sun Microsystems. 2008. LIBMCR 3 " 16 February 2008 " "libmcr-0.9". http://www.math.utah.edu/cgi-bin/man2html.cgi?/usr/ local/man/man3/libmcr.3Google Scholar
- Jean-Michel Muller. 2005. Elementary Functions: Algorithms and Implementation. Birkhauser. https://doi.org/10.1007/978-1-4899-7983-4 Google Scholar
Cross Ref
- NVIDIA. 2020. TensorFloat-32 in the A100 GPU Accelerates AI Training, HPC up to 20x. https://blogs.nvidia.com/blog/2020/ 05/14/tensorfloat-32-precision-format/Google Scholar
- Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015. Automatically Improving Accuracy for Floating Point Expressions. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Vol. 50. Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/ 2813885.2737959 Google Scholar
Digital Library
- Eugene Remes. 1934. Sur un procédé convergent d'approximations successives pour déterminer les polynômes d'approximation. Comptes rendus de l' Académie des Sciences 198 ( 1934 ), 2063-2065.Google Scholar
- Alex Sanchez-Stern, Pavel Panchekha, Sorin Lerner, and Zachary Tatlock. 2018. Finding Root Causes of Floating Point Error. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (Philadelphia, PA, USA) ( PLDI 2018). ACM, New York, NY, USA, 256-269. https://doi.org/10.1145/3296979.3192411 Google Scholar
Digital Library
- Joe Sawada. 2002. Formal verification of divide and square root algorithms using series calculation. In 3rd International Workshop on the ACL2 Theorem Prover and its Applications.Google Scholar
- Giuseppe Tagliavini, Stefan Mach, Davide Rossi, Andrea Marongiu, and Luca Benin. 2018. A transprecision floating-point platform for ultra-low power computing. In 2018 Design, Automation Test in Europe Conference Exhibition (DATE). 1051-1056. https://doi.org/10.23919/DATE. 2018.8342167 Google Scholar
Cross Ref
- Ping-Tak Peter Tang. 1990. Table-Driven Implementation of the Logarithm Function in IEEE Floating-Point Arithmetic. ACM Trans. Math. Software 16, 4 (Dec. 1990 ), 378-400. https://doi.org/10.1145/98267.98294 Google Scholar
Digital Library
- Lloyd N. Trefethen. 2012. Approximation Theory and Approximation Practice (Other Titles in Applied Mathematics). Society for Industrial and Applied Mathematics, USA.Google Scholar
- Shibo Wang and Pankaj Kanwar. 2019. BFloat16: The secret to high performance on Cloud TPUs. https://cloud.google.com/ blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpusGoogle Scholar
- Xin Yi, Liqian Chen, Xiaoguang Mao, and Tao Ji. 2019. Eficient Automated Repair of High Floating-Point Errors in Numerical Libraries. Proceedings of the ACM on Programming Languages 3, POPL, Article 56 ( Jan. 2019 ), 29 pages. https://doi.org/10.1145/3290369 Google Scholar
Digital Library
- Abraham Ziv. 1991. Fast Evaluation of Elementary Mathematical Functions with Correctly Rounded Last Bit. ACM Trans. Math. Software 17, 3 (Sept. 1991 ), 410-423. https://doi.org/10.1145/114697.116813 Google Scholar
Digital Library
- Daming Zou, Muhan Zeng, Yingfei Xiong, Zhoulai Fu, Lu Zhang, and Zhendong Su. 2019. Detecting Floating-Point Errors via Atomic Conditions. Proceedings of the ACM on Programming Languages 4, POPL, Article 60 ( Dec. 2019 ), 27 pages. https://doi.org/10.1145/3371128 Google Scholar
Digital Library
Index Terms
An approach to generate correctly rounded math libraries for new floating point variants
Recommendations
High performance correctly rounded math libraries for 32-bit floating point representations
PLDI 2021: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and ImplementationThis paper proposes a set of techniques to develop correctly rounded math libraries for 32-bit float and posit types. It enhances our RLIBM approach that frames the problem of generating correctly rounded libraries as a linear programming problem in the ...
One polynomial approximation to produce correctly rounded results of an elementary function for multiple representations and rounding modes
Mainstream math libraries for floating point (FP) do not produce correctly rounded results for all inputs. In contrast, CR-LIBM and RLIBM provide correctly rounded implementations for a specific FP representation with one rounding mode. Using such ...
Progressive polynomial approximations for fast correctly rounded math libraries
PLDI 2022: Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and ImplementationThis paper presents a novel method for generating a single polynomial approximation that produces correctly rounded results for all inputs of an elementary function for multiple representations. The generated polynomial approximation has the nice ...






Comments