skip to main content
research-article
Open Access

Static prediction of parallel computation graphs

Published:12 January 2022Publication History
Skip Abstract Section

Abstract

Many algorithms for analyzing parallel programs, for example to detect deadlocks or data races or to calculate the execution cost, are based on a model variously known as a cost graph, computation graph or dependency graph, which captures the parallel structure of threads in a program. In modern parallel programs, computation graphs are highly dynamic and depend greatly on the program inputs and execution details. As such, most analyses that use these graphs are either dynamic analyses or are specialized static analyses that gather a subset of dependency information for a specific purpose.

This paper introduces graph types, which compactly represent all of the graphs that could arise from program execution. Graph types are inferred from a parallel program using a graph type system and inference algorithm, which we present drawing on ideas from Hindley-Milner type inference, affine logic and region type systems. We have implemented the inference algorithm over a subset of OCaml, extended with parallelism primitives, and we demonstrate how graph types can be used to accelerate the development of new graph-based static analyses by presenting proof-of-concept analyses for deadlock detection and cost analysis.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

Short video for the POPL 2022 paper "Static Prediction of Parallel Computation Graphs"

References

  1. Arvind and K. P. Gostelow. 1978. The Id Report: An Asychronous Language and Computing Machine. Department of Information and Computer Science, University of California, Irvine.Google ScholarGoogle Scholar
  2. Özalp Babaoğlu, Keith Marzullo, and Fred B. Schneider. 1993. A Formalization of Priority Inversion. Real-Time Systems, 5, 4 (1993), 285–303. https://doi.org/10.1007/BF01088832 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Utpal Banerjee, Brian Bliss, Zhiqiang Ma, and Paul Petersen. 2006. A Theory of Data Race Detection. In Proceedings of the 2006 Workshop on Parallel and Distributed Systems: Testing and Debugging (PADTAD ’06). Association for Computing Machinery, New York, NY, USA. 69–78. isbn:1595934146 https://doi.org/10.1145/1147403.1147416 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Sandrine Blazy, André Maroneze, and David Pichardie. 2013. Formal Verification of Loop Bound Estimation for WCET Analysis. In Revised Selected Papers of the 5th International Conference on Verified Software: Theories, Tools, Experiments - Volume 8164 (VSTTE 2013). Springer-Verlag, Berlin, Heidelberg. 281–303. isbn:9783642541070 https://doi.org/10.1007/978-3-642-54108-7_15 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Guy Blelloch and John Greiner. 1995. Parallelism in Sequential Functional Languages. In Proceedings of the Seventh International Conference on Functional Programming Languages and Computer Architecture (FPCA ’95). Association for Computing Machinery, New York, NY, USA. 226–237. isbn:0897917197 https://doi.org/10.1145/224164.224210 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Guy E. Blelloch and John Greiner. 1996. A Provable Time and Space Efficient Implementation of NESL. In Proceedings of the First ACM SIGPLAN International Conference on Functional Programming (ICFP ’96). Association for Computing Machinery, New York, NY, USA. 213–225. isbn:0897917707 https://doi.org/10.1145/232627.232650 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Chandrasekhar Boyapati, Robert Lee, and Martin Rinard. 2002. Ownership Types for Safe Programming: Preventing Data Races and Deadlocks. In Proceedings of the 17th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA ’02). Association for Computing Machinery, New York, NY, USA. 211–230. isbn:1581134711 https://doi.org/10.1145/582419.582440 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Zhenqiang Chen, Baowen Xu, Jianjun Zhao, and Hongji Yang. 2002. Static Dependency Analysis for Concurrent Ada 95 Programs. In Reliable Software Technologies — Ada-Europe 2002, Johann Blieberger and Alfred Strohmeier (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 219–230. isbn:978-3-540-48046-4 https://doi.org/10.1007/3-540-48046-3_17 Google ScholarGoogle ScholarCross RefCross Ref
  9. Jingde Cheng. 1993. Process dependence net of distributed programs and its applications in development of distributed systems. In Proceedings of 1993 IEEE 17th International Computer Software and Applications Conference COMPSAC ’93. 231–240. https://doi.org/10.1109/CMPSAC.1993.404187 Google ScholarGoogle ScholarCross RefCross Ref
  10. Tiago Cogumbreiro, Raymond Hu, Francisco Martins, and Nobuko Yoshida. 2018. Dynamic Deadlock Verification for General Barrier Synchronisation. ACM Trans. Program. Lang. Syst., 41, 1 (2018), Article 1, Dec., 38 pages. issn:0164-0925 https://doi.org/10.1145/3229060 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Tiago Cogumbreiro, Rishi Surendran, Francisco Martins, Vivek Sarkar, Vasco T. Vasconcelos, and Max Grossman. 2017. Deadlock Avoidance in Parallel Programs with Futures: Why Parallel Tasks Should Not Wait for Strangers. Proc. ACM Program. Lang., 1, OOPSLA (2017), Article 103, Oct., 26 pages. https://doi.org/10.1145/3143359 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Dawson Engler and Ken Ashcraft. 2003. RacerX: Effective, Static Detection of Race Conditions and Deadlocks. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (SOSP ’03). Association for Computing Machinery, New York, NY, USA. 237–252. isbn:1581137575 https://doi.org/10.1145/945445.945468 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Andreas Ermedahl, Christer Sandberg, Jan Gustafsson, Stefan Bygde, and Björn Lisper. 2007. Loop Bound Analysis based on a Combination of Program Slicing, Abstract Interpretation, and Invariant Analysis. In 7th International Workshop on Worst-Case Execution Time Analysis (WCET’07), Christine Rochange (Ed.) (OpenAccess Series in Informatics (OASIcs), Vol. 6). Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany. isbn:978-3-939897-05-7 issn:2190-6807 https://doi.org/10.4230/OASIcs.WCET.2007.1194 Google ScholarGoogle ScholarCross RefCross Ref
  14. Cormac Flanagan and Stephen N. Freund. 2009. FastTrack: Efficient and Precise Dynamic Race Detection. In Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’09). Association for Computing Machinery, New York, NY, USA. 121–133. isbn:9781605583921 https://doi.org/10.1145/1542476.1542490 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Emden R. Gansner and Stephen C. North. 2000. An Open Graph Visualization System and Its Applications to Software Engineering. Softw. Pract. Exper., 30, 11 (2000), Sept., 1203?1233. issn:0038-0644 https://doi.org/10.1002/1097-024X(200009)30:11<1203::AID-SPE338>3.0.CO;2-NGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  16. David K. Gifford and John M. Lucassen. 1986. Integrating Functional and Imperative Programming. In Proceedings of the 1986 ACM Conference on LISP and Functional Programming (LFP ’86). Association for Computing Machinery, New York, NY, USA. 28–38. isbn:0897912004 https://doi.org/10.1145/319838.319848 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Robert H. Halstead. 1985. MULTILISP: a language for concurrent symbolic computation. ACM Transactions on Programming Languages and Systems, 7 (1985), 501–538. https://doi.org/10.1145/4472.4478 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Fritz Henglein, Henning Makholm, and Henning Niss. 2005. Effect Types and Region-Based Memory Management. In Advanced Topics in Types and Programming Languages, Benjamin C. Pierce (Ed.). MIT Press, Cambridge, Massachusetts. 87–135.Google ScholarGoogle Scholar
  19. J. Roger Hindley. 1969. The Principal Type-Scheme of an Object in Combinatory Logic. Trans. Amer. Math. Soc., 146 (1969), 29–60. issn:00029947 http://www.jstor.org/stable/1995158Google ScholarGoogle Scholar
  20. Jan Hoffmann and Zhong Shao. 2015. Automatic Static Cost Analysis for Parallel Programs. In Programming Languages and Systems, Jan Vitek (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg. 132–157. isbn:978-3-662-46669-8 https://doi.org/10.1007/978-3-662-46669-8_6 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Martin Hofmann and Steffen Jost. 2003. Static Prediction of Heap Space Usage for First-Order Functional Programs. In 30th Symposium on Principles of Programming Languages (POPL’03). 185–197. https://doi.org/10.1145/604131.604148 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Paul Hudak and Steven Anderson. 1987. Pomset Interpretations of Parallel Functional Programs. In Proceedings of the Functional Programming Languages and Computer Architecture. Springer-Verlag, Berlin, Heidelberg. 234–256. isbn:3540183175 https://doi.org/10.1007/3-540-18317-5_14 Google ScholarGoogle ScholarCross RefCross Ref
  23. Pierre Jouvelot and David K. Gifford. 1989. Reasoning about Continuations with Control Effects. In Proceedings of the ACM SIGPLAN 1989 Conference on Programming Language Design and Implementation (PLDI ’89). Association for Computing Machinery, New York, NY, USA. 218–226. isbn:089791306X https://doi.org/10.1145/73141.74837 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Richard M. Karp and Rayamond E. Miller. 1966. Properties of a Model for Parallel Computations: Determinacy, Termination, Queueing. SIAM J. Appl. Math., 14, 6 (1966), 1390–1411. https://doi.org/10.1137/0114108 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Y. Kasahara, Y. Nomura, M. Kamachi, J. Cheng, and K. Ushijima. 1995. An integrated support environment for distributed software development based on unified program representations. In Proceedings 1995 Asia Pacific Software Engineering Conference. 254–263. https://doi.org/10.1109/APSEC.1995.496974 Google ScholarGoogle ScholarCross RefCross Ref
  26. Bogdan Korel. 1987. The program dependence graph in static program testing. Inform. Process. Lett., 24, 2 (1987), 103–108. issn:0020-0190 https://doi.org/10.1016/0020-0190(87)90102-5 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Leslie Lamport. 1978. Time, Clocks and the Ordering of Events in a Distributed System. Commun. ACM, 21 (1978), July, 558–565.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Yau-Tsun Steven Li and Sharad Malik. 1997. Performance analysis of embedded software using implicit path enumeration. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 16, 12 (1997), 1477–1487. https://doi.org/10.1109/43.664229 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. John M. Lucassen. 1987. Types and Effects toward the Integration of Functional and Imperative Programming. Massachusetts Institute of Technology. Cambridge, Massachusetts. Available as Technical Report MIT-LCS-TR-408.Google ScholarGoogle Scholar
  30. Robin Milner. 1978. A theory of type polymorphism in programming. J. Comput. System Sci., 17, 3 (1978), 348–375. issn:0022-0000 https://doi.org/10.1016/0022-0000(78)90014-4 Google ScholarGoogle ScholarCross RefCross Ref
  31. Stefan K. Muller, Umut A. Acar, and Robert Harper. 2018. Competitive Parallelism: Getting Your Priorities Right. Proc. ACM Program. Lang., 2, ICFP (2018), Article 95, July, 30 pages. issn:2475-1421 https://doi.org/10.1145/3236790 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Stefan K. Muller, Kyle Singer, Noah Goldstein, Umut A. Acar, Kunal Agrawal, and I-Ting Angelina Lee. 2020. Responsive Parallelism with Futures and State. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’20). Association for Computing Machinery, New York, NY, USA. 577–591. isbn:9781450376136 https://doi.org/10.1145/3385412.3386013 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Alan Mycroft. 1984. Polymorphic type schemes and recursive definitions. In International Symposium on Programming, M. Paul and B. Robinet (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 217–228. isbn:978-3-540-38809-8 https://doi.org/10.1007/3-540-12925-1_41 Google ScholarGoogle ScholarCross RefCross Ref
  34. V. Krishna Nandivada, Jun Shirako, Jisheng Zhao, and Vivek Sarkar. 2013. A Transformation Framework for Optimizing Task-Parallel Programs. ACM Trans. Program. Lang. Syst., 35, 1 (2013), Article 3, April, 48 pages. issn:0164-0925 https://doi.org/10.1145/2450136.2450138 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Armand Navabi, Xiangyu Zhang, and Suresh Jagannathan. 2008. Quasi-Static Scheduling for Safe Futures. In Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’08). Association for Computing Machinery, New York, NY, USA. 23–32. isbn:9781595937957 https://doi.org/10.1145/1345206.1345212 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Jorge E Rodriguez Bezos. 1969. A Graph Model for Parallel Computations. Ph.D. Dissertation. Massachusetts Institute of Technology. Cambridge, Massachusetts.Google ScholarGoogle Scholar
  37. Mads Rosendahl. 1989. Automatic complexity analysis. In FPCA ’89: Functional Programming Languages and Computer Architecture. ACM, 144–156.Google ScholarGoogle Scholar
  38. David Sands. 1990. Complexity Analysis for a Lazy Higher-Order Language. In ESOP ’90: Proceedings of the 3rd European Symposium on Programming. Springer-Verlag, London, UK. 361–376. https://doi.org/10.1007/3-540-52592-0_74 Google ScholarGoogle ScholarCross RefCross Ref
  39. Daniel Spoonhower. 2009. Scheduling Deterministic Parallel Programs. Ph.D. Dissertation. Carnegie Mellon University. Pittsburgh, PA, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Mads Tofte and Lars Birkedal. 1998. A Region Inference Algorithm. ACM Trans. Program. Lang. Syst., 20, 4 (1998), July, 724–767. issn:0164-0925 https://doi.org/10.1145/291891.291894 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Mads Tofte and Jean-Pierre Talpin. 1997. Region-Based Memory Management. Information and Computation, 132, 2 (1997), 109–176. issn:0890-5401 https://doi.org/10.1006/inco.1996.2613 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Robert Utterback, Kunal Agrawal, Jeremy Fineman, and I-Ting Angelina Lee. 2019. Efficient Race Detection with Futures. In Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming (PPoPP ’19). Association for Computing Machinery, New York, NY, USA. 340–354. isbn:9781450362252 https://doi.org/10.1145/3293883.3295732 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Robert Utterback, Kunal Agrawal, Jeremy T. Fineman, and I-Ting Angelina Lee. 2016. Provably Good and Practically Efficient Parallel Race Detection for Fork-Join Programs. In Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’16). Association for Computing Machinery, New York, NY, USA. 83–94. isbn:9781450342100 https://doi.org/10.1145/2935764.2935801 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Caleb Voss, Tiago Cogumbreiro, and Vivek Sarkar. 2019. Transitive Joins: A Sound and Efficient Online Deadlock-Avoidance Policy. PPoPP ’19. Association for Computing Machinery, New York, NY, USA. 378–390. isbn:9781450362252 https://doi.org/10.1145/3293883.3295724 Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Mark Weiser. 1984. Program Slicing. IEEE Transactions on Software Engineering, SE-10, 4 (1984), 352–357. https://doi.org/10.1109/TSE.1984.5010248 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Yifan Xu, Kyle Singer, and I-Ting Angelina Lee. 2020. Parallel Determinacy Race Detection for Futures. In Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’20). Association for Computing Machinery, New York, NY, USA. 217–231. isbn:9781450368186 https://doi.org/10.1145/3332466.3374536 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Static prediction of parallel computation graphs

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!