skip to main content
research-article

Turning nondeterminism into parallelism

Published:29 October 2013Publication History
Skip Abstract Section

Abstract

Nondeterminism is a useful and prevalent concept in the design and implementation of software systems. An important property of nondeterminism is its latent parallelism: A nondeterministic action can evaluate to multiple behaviors. If at least one of these behaviors does not conflict with concurrent tasks, then there is an admissible execution of the action in parallel with these tasks. Unfortunately, existing implementations of the atomic paradigm - optimistic as well as pessimistic - are unable to fully exhaust the parallelism potential of nondeterministic actions, lacking the means to guide concurrent tasks toward nondeterministic choices that minimize interference.

This paper investigates the problem of utilizing parallelism due to nondeterminism. We observe that nondeterminism occurs in many real-world codes. We motivate the need for devising coordination mechanisms that can utilize available nondeterminism. We have developed a system featuring such mechanisms, which leverages nondeterminism in a wide class of query operations, allowing a task to look into the future of concurrent tasks that mutate the shared state during query evaluation and reduce conflict accordingly. We evaluate our system on a suite of 12 algorithmic benchmarks of wide applicability, as well as an industrial application. The results are encouraging.

References

  1. The bloom language. http://www.bloom-lang.net/.Google ScholarGoogle Scholar
  2. S. G. Akl and G. T. Toussaint. A fast convex hull algorithm. Inf. Process. Lett., 7:219--222, 1978.Google ScholarGoogle ScholarCross RefCross Ref
  3. Arvind and D. E. Culler. Annual review of computer science vol. 1, 1986. chapter Dataflow architectures, pages 225--253. Annual Reviews Inc., 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. H. Attiya, L. Epstein, H. Shachnai, and T. Tamir. Transactional contention management as a non-clairvoyant scheduling problem. In PODC, pages 308--315, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. H. Attiya and A. Milani. Transactional scheduling for read-dominated workloads. In OPODIS, pages 3--17, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. G. E. Belloch. Nesl: A nested data-parallel language. regular tech report CMU-CS-92-103, Carnegie Mellon, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. G. E. Blelloch. Programming parallel algorithms. Commun. ACM, 39:85--97, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. G. E. Blelloch, J. T. Fineman, and J. Shun. Greedy sequential maximal independent set and matching are parallel on average. In SPAA, pages 308--317, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. M. Botincan, M. Dodds, and S. Jagannathan. Resource-sensitive synchronization inference by abduction. In POPL, pages 309--322, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. J. Burnim, T. Elmas, G. Necula, and K. Sen. Ndseq: runtime checking for nondeterministic sequential specifications of parallel correctness. In PLDI, pages 401--414, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Burnim, T. Elmas, G. C. Necula, and K. Sen. Ndetermin: inferring nondeterministic sequential specifications for parallelism correctness. In PPOPP, pages 329--330, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. S. Cherem, T. Chilimbi, and S. Gulwani. Inferring locks for atomic sections. In PLDI, pages 304--315, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. L. Cinque and C. diMaggio. A bsp realization of jarvis? algorithm. In ICIAP, pages 247--, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. N. Conway, W. Marczak, P. Alvaro, J. M. Hellerstein, and D. Maier. Logic and lattices for distributed programming. Technical Report UCB/EECS-2012-167, EECS Department, University of California, Berkeley, http://db.cs.berkeley.edu/papers/UCB-lattice-tr.pdf, june 2012.Google ScholarGoogle Scholar
  15. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to Algorithms, Third Edition. MIT Press, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. J. B. Dennis and D. P. Misunas. A preliminary architecture for a basic data-flow processor. In ISCA, pages 126--132, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. E. W. Dijkstra. Guarded commands, nondeterminacy and formal derivation of programs. Commun. ACM, 18:453--457, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. R. W. Floyd. Nondeterministic algorithms. J. ACM, 14:636--644, 1967. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. V. Gajinov, M. Milovanovic, O. Unsal, A. Cristal, E. Ayguade, and M. Valero. Integrating dataflow abstractions into transactional memory. First Workshop on Systems for Future Multi-Core Architectures, 2011.Google ScholarGoogle Scholar
  20. J. N. Gray, R. A. Lorie, G. R. Putzolu, and I. L. Traiger. Readings in database systems. chapter Granularity of locks and degrees of consistency in a shared data base, pages 94--121. Morgan Kaufmann Publishers Inc., 1988. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. R. Guerraoui, M. Herlihy, and B. Pochon. Toward a theory of transactional contention managers. In PODC, pages 258--264, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. R. Guerraoui, M. Kapalka, and J. Vitek. Stmbench7: a benchmark for software transactional memory. In EuroSys, pages 315--324, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. G. G. Gueta, G. Ramalingam, M. Sagiv, and E. Yahav. Concurrent libraries with foresight. In PLDI, pages 263--274, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. T. Harris and K. Fraser. Language support for lightweight transactions. In OOPSLA, pages 388--402, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. T. Harris, S. Marlow, S. Peyton-Jones, and M. Herlihy. Composable memory transactions. In PPOPP, pages 48--60, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. P. Hawkins, A. Aiken, K. Fisher, M. Rinard, and M. Sagiv. Concurrent data representation synthesis. In PLDI, pages 417--428, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. P. Hawkins, A. Aiken, K. Fisher, M. C. Rinard, and M. Sagiv. Data representation synthesis. In PLDI, pages 38--49, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. D. Hendler, I. Incze, N. Shavit, and M. Tzafrir. Flat combining and the synchronization-parallelism tradeoff. In SPAA, pages 355--364, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. M. Herlihy and E. Koskinen. Transactional boosting: a methodology for highly-concurrent transactional objects. In PPoPP. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Herlihy and E. B. Moss. Transactional memory: Architectural support for lock-free data structures. In ISCA, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. S. Hong, H. Chafi, E. Sedlar, and K. Olukotun. Green-marl: a dsl for easy and efficient graph analysis. In ASPLOS, pages 349--362, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys, pages 59--72, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. M. Isard and Y. Yu. Distributed data-parallel computing using a high-level programming language. In COMAD, pages 987--994, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. A. Katsigiannis, N. Anastopoulos, K. Nikas, and N. Koziris. An approach to parallelize kruskals algorithm using helper threads. Technical report, National Technical University of Athens, 2012.Google ScholarGoogle Scholar
  35. E. Koskinen, M. J. Parkinson, and M. Herlihy. Coarse-grained transactions. In POPL, pages 19--30, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. M. Kulkarni, D. Nguyen, D. Prountzos, X. Sui, and K. Pingali. Exploiting the commutativity lattice. In PLDI, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Kulkarni, K. Pingali, B. Walter, G. Ramanarayanan, K. Bala, and L. P. Chew. Optimistic parallelism requires abstractions. In PLDI, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Lesani and J. Palsberg. Communicating memory transactions. In PPOPP, pages 157--168, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. J. S. Foster M. Hicks and P. Prattikakis. Lock inference for atomic sections. TRANSACT, 2006.Google ScholarGoogle Scholar
  40. G. Malewicz, M. H. Austern, A. J.C. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: a system for large-scale graph processing. In COMAD, pages 135--146, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Y. Mansour. Pessimistic decision tree pruning based on tree size. In ICML, pages 195--201, 1997.Google ScholarGoogle Scholar
  42. B. McCloskey, F. Zhou, D. Gay, and E. A. Brewer. Autolocker: synchronization inference for atomic sections. In POPL, pages 346--358, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, and P. Schwarz. Aries: a transaction recovery method supporting fine-granularity locking and partial rollbacks using writeahead logging. ACM Trans. Database Syst., 17:94--162, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Y. Ni, V. S. Menon, A. Adl-Tabatabai, A. L. Hosking, R. L. Hudson, E. B. Moss, B. Saha, and T. Shpeisman. Open nesting in software transactional memory. In PPOPP, pages 68--78, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. K. Nikas, N. Anastopoulos, G. I. Goumas, and N. Koziris. Employing transactional memory and helper threads to speedup dijkstra?s algorithm. In ICPP, pages 388--395, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. P. Prabhu, G. Ramalingam, and K. Vaswani. Safe programmable speculative parallelism. In PLDI, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. C. Seaton, D. Goodman, M. Lujan, and I. Watson. Applying dataflow and transactions to lee routing. In MULTIPROG, 2012.Google ScholarGoogle Scholar
  49. Y. Smaragdakis, A. Kay, R. Behrends, and M. Young. Transactions with isolation and cooperation. In OOPSLA, pages 191--210, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. A. Stivala, P. J. Stuckey, M. Garcia de la Banda, M. Hermenegildo, and A. Wirth. Lock-free parallel dynamic programming. J. Parallel Distrib. Comput., 70:839--848, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. O. Tripp, R. Manevich, J. Field, and M. Sagiv. Janus: exploiting parallelism via hindsight. In PLDI, pages 145--156, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. O. Tripp, G. Yorsh, J. Field, and M. Sagiv. Hawkeye: effective discovery of dataflow impediments to parallelization. In OOPSLA, pages 207--224, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. A. Udupa, K. Rajan, andW. Thies. Alter: exploiting breakable dependences for parallelization. In PLDI, pages 480--491, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. V. N. Vapnik. The nature of statistical learning theory. Springer-Verlag New York, Inc., 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Turning nondeterminism into parallelism

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 48, Issue 10
      OOPSLA '13
      October 2013
      867 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2544173
      Issue’s Table of Contents
      • cover image ACM Conferences
        OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications
        October 2013
        904 pages
        ISBN:9781450323741
        DOI:10.1145/2509136

      Copyright © 2013 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 29 October 2013

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!