Abstract
This paper reports on our experiences of using a commercial processor's best-effort hardware transactional memory to improve concurrency in CPython, the reference Python implementation. CPython protects its data structures using a single global lock, which inhibits parallelism when running multiple threads.
We modified the CPython interpreter to use besteffort hardware transactions available in Sun's Rock processor, and fall back on the single global lock when unable to commit in hardware. The modifications were minimal; however, we had to restructure some of CPython's shared data structures to handle false conflicts arising from CPython's management of the shared data. Our results show that the modified CPython interpreter can run small, simple, workloads and scale almost linearly, while improving the concurrency of more complex workloads.
- TIOBE programming community index for May. TIOBE Software BV, 2010.Google Scholar
- D. Beazley. Inside the python GIL (slides). Presented at the Python Concurrency Workshop, 2009.Google Scholar
- C. Blundell, A. Raghavan, and M. M. K. Martin. RETCON: Transactional repair without replay. In the proceedings of the 37th annual international symposium on Computer architecture. ACM, 2010. Google Scholar
Digital Library
- S. Chaudhry, R. Cypher, M. Ekman, M. Karlsson, A. Landin, S. Yip, H. Zeffer, and M. Tremblay. Rock: A high-performance SPARC CMT processor. IEEE Micro, 2009. Google Scholar
Digital Library
- D. Dice, Y. Lev, M. Moir, and D. Nussbaum. Early experience with a commercial hardware transactional memory implementation. In the proceeding of the 14th international conference on Architectural support for programming languages and operating systems. ACM, 2009. Google Scholar
Digital Library
- D. Dice, Y. Lev, M. Moir, D. Nussbaum, and M. Olszewski. Early experience with a commercial hardware transactional memory implementation. TR-2009-180, Sun Microsystems Laboratories, 2009. Google Scholar
Digital Library
- F. Ellen, Y. Lev, V. Luchangco, and M. Moir. SNZI: scalable nonzero indicators. In the proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing. ACM, 2007. Google Scholar
Digital Library
- D. Geer. Chip makers turn to multicore processors. IEEE Computer, 2005. Google Scholar
Digital Library
- M. Herlihy and J. E. B. Moss. Transactional memory: architectural support for lock-free data structures. In the proceedings of the 20th annual international symposium on Computer architecture. ACM, 1993. Google Scholar
Digital Library
- J. Juneau, J. Baker, F. Wierzbicki, L. S. Munoz, and V. Ng. The Definitive Guide to Jython: Python for the Java Platform. Apress, 2010. Google Scholar
Digital Library
- J. R. Larus and R. Rajwar. Transactional Memory. Synthesis Lectures on Computer Architecture. Morgan and Claypool Publishers, 2007. Google Scholar
Digital Library
- M. Moir. Hybrid transactional memory. Sun Microsystems Laboratories, 2005.Google Scholar
- R. Rajwar and J. R. Goodman. Speculative lock elision: enabling highly concurrent multithreaded execution. In the proceedings of the 34th annual ACM/IEEE international symposium on Microarchitecture. IEEE Computer Society, 2001. Google Scholar
Digital Library
- H. E. Ramadan, C. J. Rossbach, and E. Witchel. Dependence-aware transactional memory for increased concurrency. In the proceedings of the 41st annual IEEE/ACM International Symposium on Microarchitecture. IEEE Computer Society, 2008. Google Scholar
Digital Library
- A. Rigo and S. Pedroni. PyPy's approach to virtual machine construction. In the companion to the 21st ACM SIGPLAN symposium on Object-oriented programming systems, languages, and applications. ACM, 2006. Google Scholar
Digital Library
- N. Riley and C. Zilles. Hardware tansactional memory support for lightweight dynamic language evolution. In the companion to the 21st ACM SIGPLAN symposium on Object-oriented programming systems, languages, and applications. ACM, 2006. Google Scholar
Digital Library
- G. Stein. Free threading. Python Software Foundation, 2001. http://mail.python.org/pipermail/python-dev/2001-August/017099.html.Google Scholar
- G. van Rossum. The Python Language Reference: Release 2.6.4. Python Software Foundation, 2009.Google Scholar
- G. van Rossum. The Python/C API: Release 2.6.4. Python Software Foundation, 2009.Google Scholar
- G. van Rossum. Extending and Embedding Python: Release 2.6.4. Python Software Foundation, 2009.Google Scholar
- G. van Rossum. The Python Standard Library: Release 2.6.4. Python Software Foundation, 2009.Google Scholar
- R. P. Weicker. Dhrystone: a synthetic systems programming benchmark. Communications of the ACM, 1984. Google Scholar
Digital Library
Index Terms
Adding concurrency in python using a commercial processor's hardware transactional memory support
Recommendations
Hardware Support for Relaxed Concurrency Control in Transactional Memory
MICRO '43: Proceedings of the 2010 43rd Annual IEEE/ACM International Symposium on MicroarchitectureToday's transactional memory systems implement the two-phase-locking (2PL) algorithm which aborts transactions every time a conflict happens. 2PL is a simple algorithm that provides fast transactional operations. However, it limits concurrency in ...
Dependence-aware transactional memory for increased concurrency
MICRO 41: Proceedings of the 41st annual IEEE/ACM International Symposium on MicroarchitectureTransactional memory (TM) is a promising paradigm for helping programmers take advantage of emerging multi-core platforms. Though they perform well under low contention, hardware TM systems have a reputation of not performing well under high contention, ...






Comments