skip to main content
research-article

Storage strategies for collections in dynamically typed languages

Published:29 October 2013Publication History
Skip Abstract Section

Abstract

Dynamically typed language implementations often use more memory and execute slower than their statically typed cousins, in part because operations on collections of elements are unoptimised. This paper describes storage strategies, which dynamically optimise collections whose elements are instances of the same primitive type. We implement storage strategies in the PyPy virtual machine, giving a performance increase of 18% on wide-ranging benchmarks of real Python programs. We show that storage strategies are simple to implement, needing only 1500LoC in PyPy, and have applicability to a wide range of virtual machines.

References

  1. C. S. Ananian and M. Rinard. Data size optimizations for Java programs. In phProc. LCTES, pages 59--68, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. A. W. Appel and M. J. Gonçalves. Hash-consing garbage collection. Technical Report CS-TR-412--93, Princeton University, Dept. of Computer Science, 1993.Google ScholarGoogle Scholar
  3. V. Bala, E. Duesterwald, and S. Banerjia. Dynamo: a transparent dynamic optimization system. In phProc. PLDI, pages 1--12, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M. Bayer. Orm2010, 2012. URL http://techspot.zzzeek.org. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  5. S. Bird, E. Klein, and E. Loper. phNatural Language Processing with Python. O'Reilly Media, July 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. F. Bolz and L. Tratt. The impact of meta-tracing on VM design and implementation. phTo appear, Science of Computer Programming, 2013.Google ScholarGoogle Scholar
  7. C. F. Bolz, A. Cuni, M. Fijałkowski, and A. Rigo. Tracing the meta-level: PyPy's tracing JIT compiler. In phProc. ICOOOLPS, pages 18--25, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. C. F. Bolz, A. Cuni, M. Fijałkowski, M. Leuschel, S. Pedroni, and A. Rigo. Allocation removal by partial evaluation in a tracing JIT. phProc. PEPM, Jan. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. O. Callaú, R. Robbes, E. Tanter, and D. Röthlisberger. How developers use the dynamic features of programming languages: the case of smalltalk. In phProc. MSR, page 23--32, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. B. Cannon. phLocalized type inference of atomic types in Python. Master thesis, California Polytechnic State University, 2005.Google ScholarGoogle Scholar
  11. C. Chambers, D. Ungar, and E. Lee. An efficient implementation of SELF a dynamically-typed object-oriented language based on prototypes. In phOOPSLA, volume 24, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. C. Chambers, D. Ungar, and E. Lee. An efficient implementation of self a dynamically-typed object-oriented language based on prototypes. phSIGPLAN Not., 24: 49--70, Sept. 1989\natexlabb. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. D. Clifford. URL http://v8-io12.appspot.com/. Talk at IO12 {Online; accessed 27-June-2013}.Google ScholarGoogle Scholar
  14. D. Clifford, V. Egorov, and M. Stanton. Personal communication, July 2013.Google ScholarGoogle Scholar
  15. J. da Silva and G. Lopes. A local maxima method and a fair dispersion normalization for extracting multi-word units from corpora. In phMeeting on Mathematics of Language, 1999.Google ScholarGoogle Scholar
  16. FeedParser Developers. Feedparser, 2012. URL http://code.google.com/p/feedparser/. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  17. P. Fleming and J. Wallace. How not to lie with statistics: the correct way to summarize benchmark results. phCommun. ACM, 29 (3): 218--221, Mar. 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Gal, B. Eich, M. Shaver, D. Anderson, D. Mandelin, M. R. Haghighat, B. Kaplan, G. Hoare, B. Zbarsky, J. Orendorff, J. Ruderman, E. W. Smith, R. Reitmaier, M. Bebenita, M. Chang, and M. Franz. Trace-based Just-In-Time type specialization for dynamic languages. In phProc. PLDI, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. E. Gamma, R. Helm, R. E. Johnson, and J. Vlissides. phDesign Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Longman, Amsterdam, Oct. 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Georges, D. Buytaert, and L. Eeckhout. Statistically rigorous Java performance evaluation. phSIGPLAN Notices, 42 (10): 57--76, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D. Gudeman. Representing type information in Dynamically-Typed languages. Technical Report TR93--27, University of Arizona at Tucson, 1993.Google ScholarGoogle Scholar
  22. B. Hackett and S.-y. Guo. Fast and precise hybrid type inference for JavaScript. In phProc. PLDI, pages 239--250, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. A. Holkner and J. Harland. Evaluating the dynamic behaviour of python applications. In phProc. ACSC, pages 19--28, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. T. Kotzmann and H. Mössenböck. Escape analysis in the context of dynamic compilation and deoptimization. In phProc. VEE, page 111--120, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Logilab. Pylint, 2012. URL http://www.logilab.org/project/pylint. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  26. NetworkX Developers. Networkx, 2012. URL http://networkx.lanl.gov. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  27. Excelerator Developers. pyexcelerator, 2012. URL http://sourceforge.net/projects/pyexcelerator. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  28. PyPy Team. Pypy speed benchmarks, 2013. URL http://speed.pypy.org/. {Online; accessed 27-June-2013}.Google ScholarGoogle Scholar
  29. P. Quentel. Pydblite, 2012. URL http://www.pydblite.net/en/index.html. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  30. A. Radziszewski and M. Piasecki. A preliminary Noun Phrase Chunker for Polish. In phProc. IOS, pages 169--180. Springer, 2010. Chunker available at http://nlp.pwr.wroc.pl/trac/private/disaster/.Google ScholarGoogle Scholar
  31. G. Richards, S. Lebresne, B. Burg, and J. Vitek. An analysis of the dynamic behavior of JavaScript programs. In phProc. PLDI, pages 1--12, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. A. Rigo and S. Pedroni. PyPy's approach to virtual machine construction. In phProc. DLS, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Rosetta Code. Inverted index, 2012. URL http://rosettacode.org/wiki/Inverted_index#Python. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar
  34. J. B. Sartor, M. Hirzel, and K. S. McKinley. No bit left behind: the limits of heap data compression. In phProc. ISSM, pages 111--120. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. L. Tratt. Dynamically typed languages. phAdvances in Computers, 77: 149--184, July 2009.Google ScholarGoogle ScholarCross RefCross Ref
  36. Whoosh Developers. Whoosh, 2012. URL https://bitbucket.org/mchaput/whoosh. {Online; accessed 26-March-2013}.Google ScholarGoogle Scholar

Index Terms

  1. Storage strategies for collections in dynamically typed languages

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGPLAN Notices
            ACM SIGPLAN Notices  Volume 48, Issue 10
            OOPSLA '13
            October 2013
            867 pages
            ISSN:0362-1340
            EISSN:1558-1160
            DOI:10.1145/2544173
            Issue’s Table of Contents
            • cover image ACM Conferences
              OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications
              October 2013
              904 pages
              ISBN:9781450323741
              DOI:10.1145/2509136

            Copyright © 2013 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 29 October 2013

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!