skip to main content
article

Parallel virtual machines with RPython

Published:01 November 2016Publication History
Skip Abstract Section

Abstract

The RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support parallel execution since the framework relies on a Global Interpreter Lock (GIL): PyPy serialises the execution of multi-threaded Python programs.

We describe the rationale and design of a new parallel execution model for RPython that allows the generation of parallel virtual machines while leaving the language semantics unchanged. This model then allows different implementations of concurrency control, and we discuss an implementation based on a GIL and an implementation based on Software Transactional Memory (STM).

To evaluate the benefits of either choice, we adapt PyPy to work with both implementations (GIL and STM). The evaluation shows that PyPy with STM improves the runtime of a set of multi-threaded Python programs over PyPy with a GIL by factors in the range of 1.87x up to 5.96x when executing on a processor with 8 cores.

References

  1. “The IronPython Project,” http://ironpython.net/.Google ScholarGoogle Scholar
  2. “The Jython Project,” http://www.jython.org/.Google ScholarGoogle Scholar
  3. Nicholas Riley and Craig Zilles, “Hardware Transactional Memory Support for Lightweight Dynamic Language Evolution,” OOPSLA ’06, pp. 998–1008, ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Rei Odaira, Jose G. Castanos, and Hisanobu Tomari, “Eliminating Global Interpreter Locks in Ruby Through Hardware Transactional Memory,” PPoPP ’14, pp. 131–142, ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Fuad Tabba, “Adding Concurrency in Python Using a Commercial Processor’s Hardware Transactional Memory Support,” SIGARCH Comput. Archit. News, vol. 38, no. 5, pp. 12–19, Apr. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yu David Liu, Xiaoqi Lu, and Scott F. Smith, “Coqa: Concurrent Objects with Quantized Atomicity,” CC’08/ETAPS’08, pp. 260–275, Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski, and Armin Rigo, “Tracing the Meta-level: PyPy’s Tracing JIT Compiler,” ICOOOLPS ’09, pp. 18–25, ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. “CPython,” https://www.python.org/.Google ScholarGoogle Scholar
  9. “The JRuby Project,” http://jruby.org/.Google ScholarGoogle Scholar
  10. Colin Blundell, E. Lewis, and Milo Martin, “Unrestricted Transactional Memory: Supporting I/O and System Calls Within Transactions,” Technical Reports (CIS), May 2006.Google ScholarGoogle Scholar
  11. “PyPy Source on Bitbucket,” https://bitbucket.org/ pypy/pypy.Google ScholarGoogle Scholar
  12. Remigius Meier, Armin Rigo, and Thomas Gross, “A Transactional Memory System for Parallel Python,” https://bitbucket.org/pypy/extradoc/raw/ 12bda771698925b8296808959e7b830aef8b78b8/ talk/dls2014/paper/paper.pdf, Aug. 2014.Google ScholarGoogle Scholar
  13. “PyPy Benchmarks on Bitbucket,” https://bitbucket. org/pypy/benchmarks.Google ScholarGoogle Scholar
  14. Christian Wimmer and Thomas Würthinger, “Truffle: A Selfoptimizing Runtime System,” SPLASH ’12, pp. 13–14, ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Parallel virtual machines with RPython

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!