Abstract
The RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support parallel execution since the framework relies on a Global Interpreter Lock (GIL): PyPy serialises the execution of multi-threaded Python programs.
We describe the rationale and design of a new parallel execution model for RPython that allows the generation of parallel virtual machines while leaving the language semantics unchanged. This model then allows different implementations of concurrency control, and we discuss an implementation based on a GIL and an implementation based on Software Transactional Memory (STM).
To evaluate the benefits of either choice, we adapt PyPy to work with both implementations (GIL and STM). The evaluation shows that PyPy with STM improves the runtime of a set of multi-threaded Python programs over PyPy with a GIL by factors in the range of 1.87x up to 5.96x when executing on a processor with 8 cores.
- “The IronPython Project,” http://ironpython.net/.Google Scholar
- “The Jython Project,” http://www.jython.org/.Google Scholar
- Nicholas Riley and Craig Zilles, “Hardware Transactional Memory Support for Lightweight Dynamic Language Evolution,” OOPSLA ’06, pp. 998–1008, ACM. Google Scholar
Digital Library
- Rei Odaira, Jose G. Castanos, and Hisanobu Tomari, “Eliminating Global Interpreter Locks in Ruby Through Hardware Transactional Memory,” PPoPP ’14, pp. 131–142, ACM. Google Scholar
Digital Library
- Fuad Tabba, “Adding Concurrency in Python Using a Commercial Processor’s Hardware Transactional Memory Support,” SIGARCH Comput. Archit. News, vol. 38, no. 5, pp. 12–19, Apr. 2010. Google Scholar
Digital Library
- Yu David Liu, Xiaoqi Lu, and Scott F. Smith, “Coqa: Concurrent Objects with Quantized Atomicity,” CC’08/ETAPS’08, pp. 260–275, Springer-Verlag. Google Scholar
Digital Library
- Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski, and Armin Rigo, “Tracing the Meta-level: PyPy’s Tracing JIT Compiler,” ICOOOLPS ’09, pp. 18–25, ACM. Google Scholar
Digital Library
- “CPython,” https://www.python.org/.Google Scholar
- “The JRuby Project,” http://jruby.org/.Google Scholar
- Colin Blundell, E. Lewis, and Milo Martin, “Unrestricted Transactional Memory: Supporting I/O and System Calls Within Transactions,” Technical Reports (CIS), May 2006.Google Scholar
- “PyPy Source on Bitbucket,” https://bitbucket.org/ pypy/pypy.Google Scholar
- Remigius Meier, Armin Rigo, and Thomas Gross, “A Transactional Memory System for Parallel Python,” https://bitbucket.org/pypy/extradoc/raw/ 12bda771698925b8296808959e7b830aef8b78b8/ talk/dls2014/paper/paper.pdf, Aug. 2014.Google Scholar
- “PyPy Benchmarks on Bitbucket,” https://bitbucket. org/pypy/benchmarks.Google Scholar
- Christian Wimmer and Thomas Würthinger, “Truffle: A Selfoptimizing Runtime System,” SPLASH ’12, pp. 13–14, ACM. Google Scholar
Digital Library
Index Terms
Parallel virtual machines with RPython
Recommendations
Virtual machine design for parallel dynamic programming languages
To leverage the benefits of modern hardware, dynamic languages must support parallelism, and parallelism requires a virtual machine (VM) capable of parallel execution — a parallel VM. However, unrestricted concurrency and the dynamism of dynamic ...
Parallel virtual machines with RPython
DLS 2016: Proceedings of the 12th Symposium on Dynamic LanguagesThe RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support ...
A way forward in parallelising dynamic languages
ICOOOLPS '14: Proceedings of the 9th International Workshop on Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems PLEDynamic languages became very popular in recent years. At some point, the need for concurrency arose, and many of them made the choice to use a single global interpreter lock (GIL) to synchronise the interpreter in a multithreading scenario. This choice,...







Comments