skip to main content
research-article

Adding approximate counters

Published:27 February 2016Publication History
Skip Abstract Section

Abstract

We describe a general framework for adding the values of two approximate counters to produce a new approximate counter value whose expected estimated value is equal to the sum of the expected estimated values of the given approximate counters. (To the best of our knowledge, this is the first published description of any algorithm for adding two approximate counters.) We then work out implementation details for five different kinds of approximate counter and provide optimized pseudocode. For three of them, we present proofs that the variance of a counter value produced by adding two counter values in this way is bounded, and in fact is no worse, or not much worse, than the variance of the value of a single counter to which the same total number of increment operations have been applied. Addition of approximate counters is useful in massively parallel divide-and-conquer algorithms that use a distributed representation for large arrays of counters. We describe two machine-learning algorithms for topic modeling that use millions of integer counters, and confirm that replacing the integer counters with approximate counters is effective, speeding up a GPU-based implementation by over 65% and a CPU-based by nearly 50%, as well as reducing memory requirements, without degrading their statistical effectiveness.

References

  1. A. C. Callahan. Random rounding: Some principles and applications. In ICASSP '76: IEEE Intl. Conf. Acoustics, Speech, and Signal Processing, volume 1, pages 501--504, Apr 1976.Google ScholarGoogle ScholarCross RefCross Ref
  2. Miklós Csűrös. Approximate counting with a floating-point counter. In COCOON '10: Proc. 16th Annual International Conf. Computing and Combinatorics, pages 358--367, Berlin, Heidelberg, 2010. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Andrej Cvetkovski. An algorithm for approximate counting using limited memory resources. In SIGMETRICS '07: Proc. 2007 ACM SIGMETRICS International Conf. Measurement and Modeling of Computer Systems, pages 181--190, New York, 2007. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Dave Dice, Yossi Lev, and Mark Moir. Scalable statistics counters. In PPoPP '13: Proc. 18th ACM SIGPLAN Symp. Principles and Practice of Parallel Programming, pages 307--308, New York, 2013. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Philippe Flajolet. Approximate counting: A detailed analysis. BIT Numerical Mathematics, 25(1):113--134, 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. George E. Forsythe. Reprint of a note on rounding-off errors. SIAM Review, 1(1):66--67, 1959. dx.doi.org/10.1137/1001011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Scott A. Mitchell and David M. Day. Flexible approximate counting. In IDEAS '11: Proc. 15th Symp. International Database Engineering & Applications, pages 233--239, New York, 2011. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Robert Morris. Counting large numbers of events in small registers. Commun. ACM, 21(10):840--842, October 1978. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. D. S. Parker, B. Pierce, and P. R. Eggert. Monte Carlo arithmetic: How to gamble with floating point and win. Computing in Science Engineering, 2(4):58--68, July 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. D. Stott Parker. Monte Carlo arithmetic: Exploiting randomness in floating-point arithmetic. Technical Report 970002, Computer Science Department, UCLA, Los Angeles, CA, March 1997. http://fmdb.cs.ucla.edu/Treports/970002.pdf.Google ScholarGoogle Scholar
  11. Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, and Jack Dongarra. MPI---The Complete Reference: Volume 1, The MPI Core. MIT Press, Cambridge, Massachusetts, second edition, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Rade Stanojevic. Small active counters. In 26th IEEE International Conf. Computer Communications (INFOCOM 2007), pages 2153--2161, May 2007.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jean-Baptiste Tristan, Joseph Tassarotti, and Guy L. Steele Jr. Efficient training of LDA on a GPU by Mean-For-Mode Gibbs sampling. In ICML 2015: 32nd International Conf. Machine Learning, volume 37, July 2015. Volume 37 of the Journal in Machine Learning Research: Workshop and Conference Proceedings.Google ScholarGoogle Scholar
  14. A. J. Walker. Fast generation of uniformly distributed pseudorandom numbers with floating-point representation. Electronics Letters, 10(25):533--534, December 1974.Google ScholarGoogle ScholarCross RefCross Ref
  15. Manzil Zaheer, Michael Wick, Jean-Baptiste Tristan, Alex Smola, and Guy L. Steele Jr. Exponential stochastic cellular automata for massively parallel inference. In LearningSys: Workshop on Machine Learning Systems at Neural Information Processing Systems (NIPS), December 2015. http://learningsys.org/papers/LearningSys_2015_paper_11.pdf.Google ScholarGoogle Scholar
  16. Manzil Zaheer, Michael Wick, Jean-Baptiste Tristan, Alex Smola, and Guy L. Steele Jr. Exponential stochastic cellular automata for massively parallel inference. In AISTATS: 19th Intl. Conf. Artificial Intelligence and Statistics, May 2016. To appear.Google ScholarGoogle Scholar

Index Terms

  1. Adding approximate counters

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader
              About Cookies On This Site

              We use cookies to ensure that we give you the best experience on our website.

              Learn more

              Got it!