Abstract
The MCS lock is one of the most prevalent queuing locks. It provides fair scheduling and high performance on massively parallel systems. However, the MCS lock mandates a bring-your-own-context policy: each lock user must provide an additional context (i.e., a queue node) to interact with the lock. This paper proposes MCSg, a variant of the MCS lock that relaxes this restriction.
Our key observation is that not all lock users are created equal. We analyzed how locks are used in massively-parallel modern systems, such as NUMA-aware operating systems and databases. We found that such systems often have a small number of "regular" code paths that enter the lock very frequently. Such code paths are the primary beneficiary of the high scalability of MCS locks.
However, there are also many "guest" code paths that infrequently enter the lock and do not need the same degree of fairness to access the lock (e.g., background tasks that only run periodically with lower priority). These guest users, which are typically spread out in various modules of the software, prefer context-free locks, such as ticket locks.
MCSg provides these guests a context-free interface while regular users still enjoy the benefits provided by MCS. It can also be used as a drop-in replacement of MCS for more advanced locks, such as cohort locking. We also propose MCSg++, an extended version of MCSg, which avoids guest starvation and non-FIFO behaviors that might happen with MCSg.
Our evaluation using microbenchmarks and the TPC-C database benchmark on a 16-socket, 240-core server shows that both MCSg and MCSg++ preserve the benefits of MCS for regular users while providing a context-free interface for guests.
- M. Auslander, D. Edelsohn, O. Krieger, B. Rosenburg, and R. Wisniewski. Enhancement to the MCS lock for increased functionality and improved programmability. U.S. patent application number 20030200457 (abandoned), 2003.Google Scholar
- D. Bueso and S. Norton. An overview of kernel lock improvements. LinuxCon North America, 2014. http://events.linuxfoundation.org/sites/events/files/slides/linuxcon-2014-locking-final.pdf.Google Scholar
- J. Corbet. MCS locks and qspinlocks. LWN, 2014. https://lwn.net/Articles/590243/.Google Scholar
- T. Craig. Building FIFO and priority-queuing spin locks from atomic swap. Technical Report TR 93-02-02, Department of Computer Science and Engineering, University of Washington, 1993. ftp://ftp.cs.washington.edu/tr/1993/02/UW-CSE-93-02-02.pdf.Google Scholar
- D. Dice, V. J. Marathe, and N. Shavit. Lock cohorting: a general technique for designing NUMA locks. PPoPP, pages 247--256, 2012. Google Scholar
Digital Library
- D. Gifford and A. Spector. Case Study: IBM's System/360-370 Architecture. CACM, 30(4):291--307, Apr. 1987. Google Scholar
Digital Library
- R. Johnson, I. Pandis, N. Hardavellas, A. Ailamaki, and B. Falsafi. Shore-MT: A Scalable Storage Manager for the Multicore Era. In EDBT, pages 24--35, 2009. Google Scholar
Digital Library
- H. Kimura. FOEDUS: OLTP engine for a thousand cores and NVRAM. SIGMOD, pages 691--706, 2015. Google Scholar
Digital Library
- L. Lamport. A new solution of dijkstra's concurrent programming problem. CACM, 17(8):453--455, Aug. 1974. Google Scholar
Digital Library
- S. Li, T. Hoefler, and M. Snir. NUMA-aware shared-memory collective communication for MPI. HPDC, pages 85--96, 2013. Google Scholar
Digital Library
- J. Low. Personal communication, 2015.Google Scholar
- J. Low et al. {patch} timer: Improve itimers scalability. LKML, 2015. https://lkml.org/lkml/2015/10/14/822.Google Scholar
- M. Luo, D. K. Panda, K. Z. Ibrahim, and C. Iancu. Congestion avoidance on manycore high performance computing systems. ICS, pages 121--132, 2012. Google Scholar
Digital Library
- P. S. Magnusson, A. Landin, and E. Hagersten. Queue locks on cache coherent multiprocessors. International Symposium on Parallel Processing, pages 165--171, 1994. Google Scholar
Digital Library
- J. M. Mellor-Crummey and M. L. Scott. Algorithms for scalable synchronization on shared-memory multiprocessors. ACM TOCS, 9 (1):21--65, Feb. 1991. Google Scholar
Digital Library
- J. M. Mellor-Crummey and M. L. Scott. Scalable reader-writer synchronization for shared-memory multiprocessors. PPoPP, pages 106--113, 1991. Google Scholar
Digital Library
- M. L. Scott. Shared-memory synchronization. Synthesis Lectures on Computer Architecture, 8(2):1--221, 2013. Google Scholar
Digital Library
- M. L. Scott and W. N. Scherer. Scalable queue-based spin locks with timeout. PPoPP, pages 44--52, 2001. Google Scholar
Digital Library
- Transaction Processing Performance Council. TPC benchmark C standard specification, revision 5.11. 2010.Google Scholar
- Y. Yan, S. Chatterjee, Z. Budimlic, and V. Sarkar. Integrating mpi with asynchronous task parallelism. In EuroMPI, pages 333--336, 2011. Google Scholar
Digital Library
Index Terms
Be my guest: MCS lock now welcomes guests
Recommendations
High performance locks for multi-level NUMA systems
PPoPP 2015: Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel ProgrammingEfficient locking mechanisms are critically important for high performance computers. On highly-threaded systems with a deep memory hierarchy, the throughput of traditional queueing locks, e.g., MCS locks, falls off due to NUMA effects. Two-level ...
Be my guest: MCS lock now welcomes guests
PPoPP '16: Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel ProgrammingThe MCS lock is one of the most prevalent queuing locks. It provides fair scheduling and high performance on massively parallel systems. However, the MCS lock mandates a bring-your-own-context policy: each lock user must provide an additional context (...
Non-blocking timeout in scalable queue-based spin locks
PODC '02: Proceedings of the twenty-first annual symposium on Principles of distributed computingQueue-based spin locks allow programs with busy-wait synchronization to scale to very large multiprocessors, without fear of starvation or performance-destroying contention. Timeout-capable spin locks allow a thread to abandon its attempt to acquire a ...






Comments