skip to main content
research-article

Faster Base64 Encoding and Decoding Using AVX2 Instructions

Published:17 July 2018Publication History
Skip Abstract Section

Abstract

Web developers use base64 formats to include images, fonts, sounds, and other resources directly inside HTML, JavaScript, JSON, and XML files. We estimate that billions of base64 messages are decoded every day. We are motivated to improve the efficiency of base64 encoding and decoding. Compared to state-of-the-art implementations, we multiply the speeds of both the encoding (≈ 10 ×0) and the decoding (≈ 7 ×). We achieve these good results by using the single-instruction-multiple-data instructions available on recent Intel processors (AVX2). Our accelerated software abides by the specification and reports errors when encountering characters outside of the base64 set. It is available online as free software under a liberal license.

References

  1. Amazon 2015. Amazon SimpleDB. Retrieved from http://docs.aws.amazon.com/AmazonSimpleDB/latest/DeveloperGuide/Welcome.html.Google ScholarGoogle Scholar
  2. Amazon 2017. Amazon DynamoDB. Retrieved from http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html.Google ScholarGoogle Scholar
  3. ARM 2017. The Scalable Vector Extension (SVE), for ARMv8-A. Technical Report. ARM Holdings, Cambridge, UK. Retrieved from https://static.docs.arm.com/ddi0584/a/DDI0584A_a_SVE_supp_armv8A.pdf.Google ScholarGoogle Scholar
  4. David Calhoun. 2011. When to base64 encode images (and when not to). Retrieved from http://davidbcalhoun.com/2011/when-to-base64-encode-images-and-when-not-to/.Google ScholarGoogle Scholar
  5. Hanson Char. 2014. A fast and correct base 64 codec. Retrieved from https://aws.amazon.com/blogs/developer/a-fast-and-correct-base-64-codec/.Google ScholarGoogle Scholar
  6. Matt Crane and Jimmy Lin. 2017. An exploration of serverless architectures for information retrieval. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR’17). ACM, New York, NY, 241--244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mark Davis. 2012. Unicode over 60 percent of the web. Retrieved from https://googleblog.blogspot.ca/2012/02/unicode-over-60-percent-of-web.html.Google ScholarGoogle Scholar
  8. Elastic 2017. Elasticsearch reference-Binary datatype. Retrieved from https://www.elastic.co/guide/en/elasticsearch/reference/current/binary.html.Google ScholarGoogle Scholar
  9. Pierre Estérie, Joel Falcou, Mathias Gaunard, and Jean-Thierry Lapresté. 2014. Boost.SIMD: Generic programming for portable SIMDization. In Proceedings of the Workshop on Programming Models for SIMD/Vector Processing (WPMVP’14). ACM, New York, NY, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Tammy Everts. 2013. Rules for mobile performance optimization. Commun. ACM 56, 8 (Aug. 2013), 52--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Roy T. Fielding, James Gettys, Jeffrey C. Mogul, Henrik Frystyk Nielsen, Larry Masinter, Paul J. Leach, and Tim Berners-Lee. 1999. Hypertext transfer protocol-HTTP/1.1. Retrieved from https://tools.ietf.org/html/rfc2616. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Agner Fog. 2016. Instruction Tables: Lists of Instruction Latencies, Throughputs and Micro-operation Breakdowns for Intel, AMD and VIA CPUs. Technical Report. Copenhagen University College of Engineering, Copenhagen, Denmark. Retrieved from http://www.agner.org/optimize/instruction_tables.pdf.Google ScholarGoogle Scholar
  13. Ned Freed and Nathaniel S. Borenstein. 1996. Multipurpose internet mail extensions (MIME) Part One: Format of internet message bodies. Retrieved from https://tools.ietf.org/html/rfc2045. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Nick Galbreath. 2016. Fast c-string transformations. Retrieved from https://github.com/client9/stringencoders.Google ScholarGoogle Scholar
  15. Ian Hickson. 2016. Web storage. Retrieved from https://www.w3.org/TR/webstorage/.Google ScholarGoogle Scholar
  16. Richard Duchatsch Johansen, Talita Cristina Pagani Britto, and Cesar Augusto Cusin. 2013. CSS browser selector plus: A JavaScript library to support cross-browser responsive design. In Proceedings of the 22nd International Conference on World Wide Web (WWW’13). ACM, New York, NY, 27--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Michael B. Jones, John Bradley, and Nat Sakimura. 2015. JSON web signature (JWS). Retrieved from https://tools.ietf.org/html/rfc7515.Google ScholarGoogle Scholar
  18. Simon Josefsson. 2006. The base16, base32, and base64 data encodings. Retrieved from https://tools.ietf.org/html/rfc4648.Google ScholarGoogle Scholar
  19. Alfred Klomp. 2014a. Fast base64 encoding/decoding with SSE vectorization. Retrieved from http://www.alfredklomp.com/programming/sse-base64/.Google ScholarGoogle Scholar
  20. Alfred Klomp. 2014b. Fast base64 stream encoder/decoder in C99, with SIMD acceleration. Retrieved from https://github.com/aklomp/base64.Google ScholarGoogle Scholar
  21. Nick Kopp. 2013. Base64 encoding on a GPU. Retrieved from https://www.codeproject.com/Articles/276993/Base-Encoding-on-a-GPU.Google ScholarGoogle Scholar
  22. John Linn. 1993. Privacy enhancement for internet electronic mail: Part I: Message encryption and authentication procedures. Retrieved from https://tools.ietf.org/html/rfc1421.Google ScholarGoogle Scholar
  23. Larry Masinter. 1998. The “data” URL scheme. Retrieved from https://tools.ietf.org/html/rfc2397.Google ScholarGoogle Scholar
  24. Microsoft 2017. XML data (SQL server)-Use the binary base64 option. Retrieved from https://www.elastic.co/guide/en/elasticsearch/reference/current/binary.html.Google ScholarGoogle Scholar
  25. MongoDB 2017. MongoDB extended JSON. Retrieved from https://docs.mongodb.com/manual/reference/mongodb-extended-json/.Google ScholarGoogle Scholar
  26. Thomas Nägele. 2015. Client-side Performance Profiling of JavaScript for Web Applications. Master’s thesis. Radboud University Nijmegen, The Netherlands.Google ScholarGoogle Scholar
  27. Gabriele Paoloni. 2010. How to Benchmark Code Execution Times on Intel IA-32 and IA-64 Instruction Set Architectures. Intel Corporation, Santa Clara, CA.Google ScholarGoogle Scholar
  28. Guru Prasad Srinivasa, Rizwana Begum, Scott Haseley, Mark Hempstead, and Geoffrey Challen. 2017. Separated by birth: Hidden differences between seemingly-identical smartphone CPUs. In Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications (HotMobile’17). ACM, New York, NY, 103--108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Yi Tang and Manjia Lin. 2015. EQPO: Obscuring encrypted web traffic with equal-sized pseudo-objects. In Proceedings of the International Conference on Information Security and Cryptology. Springer, New York, 227--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Matt Tierney, Ian Spiro, Christoph Bregler, and Lakshminarayanan Subramanian. 2013. Cryptagram: Photo privacy for online social media. In Proceedings of the 1st ACM Conference on Online Social Networks (COSN’13). ACM, New York, NY, 75--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Haichuan Wang, Peng Wu, Ilie Gabriel Tanase, Mauricio J. Serrano, and José E. Moreira. 2014. Simple, portable, and fast SIMD intrinsic programming: Generic SIMD library. In Proceedings of the Workshop on Programming Models for SIMD/Vector Processing (WPMVP’14). ACM, New York, NY, 9--16. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Faster Base64 Encoding and Decoding Using AVX2 Instructions

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on the Web
      ACM Transactions on the Web  Volume 12, Issue 3
      August 2018
      207 pages
      ISSN:1559-1131
      EISSN:1559-114X
      DOI:10.1145/3240924
      Issue’s Table of Contents

      Copyright © 2018 Owner/Author

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 17 July 2018
      • Accepted: 1 January 2018
      • Revised: 1 August 2017
      • Received: 1 March 2017
      Published in tweb Volume 12, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!