skip to main content

Taking Back Control in an Intermediate Representation for GPU Computing

Published:11 January 2023Publication History
Skip Abstract Section

Abstract

We describe our experiences successfully applying lightweight formal methods to substantially improve and reformulate an important part of Standard Portable Intermediate Representation SPIRV, an industry-standard language for GPU computing. The formal model that we present has allowed us to (1) identify several ambiguities and needless complexities in the way that structured control flow was defined in the SPIRV specification; (2) interact with the authors of the SPIRV specification to rectify these problems; (3) validate the developer tools and conformance test suites that support the SPIRV language by cross-checking them against our formal model, improving the tools, test suites, and our models in the process; and (4) develop a novel method for fuzzing SPIRV compilers to detect miscompilation bugs that leverages our formal model. The latest release of the SPIRV specification incorporates the revised set of control-flow definitions that have arisen from our work. Furthermore, our novel compiler-fuzzing technique has led to the discovery of twenty distinct, previously unknown bugs in SPIRV compilers from Google, the Khronos Group, Intel, and Mozilla. Our work showcases the practical impact that formal modelling and analysis techniques can have on the design and implementation of industry-standard programming languages.

References

  1. Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. 2007. Compilers, Principles, Techniques & Tools, Second Edition. Pearson. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Jade Alglave, Mark Batty, Alastair F. Donaldson, Ganesh Gopalakrishnan, Jeroen Ketema, Daniel Poetzl, Tyler Sorensen, and John Wickerson. 2015. GPU Concurrency: Weak Behaviours and Programming Assumptions. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2015, Istanbul, Turkey, March 14-18, 2015, Özcan Özturk, Kemal Ebcioglu, and Sandhya Dwarkadas (Eds.). ACM, 577–591. https://doi.org/10.1145/2694344.2694391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Said Amellal and Bozena Kaminska. 1993. Scheduling of a Control and Data Flow Graph. In 1993 IEEE International Symposium on Circuits and Systems, ISCAS 1993, Chicago, Illinois, USA, May 3-6, 1993. IEEE, 1666–1669. Google ScholarGoogle ScholarCross RefCross Ref
  4. Apple. 2022. Metal Shading Language. https://developer.apple.com/metal/ Google ScholarGoogle Scholar
  5. Thomas Ball and Susan Horwitz. 1993. Slicing Programs with Arbitrary Control-flow. In Automated and Algorithmic Debugging, First International Workshop, AADEBUG’93, Linköping, Sweden, May 3-5, 1993, Proceedings, Peter Fritszon (Ed.) (Lecture Notes in Computer Science, Vol. 749). Springer, 206–222. https://doi.org/10.1007/BFb0019410 Google ScholarGoogle ScholarCross RefCross Ref
  6. Mark Batty, Alastair F. Donaldson, and John Wickerson. 2016. Overhauling SC atomics in C11 and OpenCL. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016, Rastislav Bodík and Rupak Majumdar (Eds.). ACM, 634–648. https://doi.org/10.1145/2837614.2837637 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Sean Baxter. 2020. Khronos Group forum post: Clarify meaning of merge block. https://community.khronos.org/t/clarify-meaning-of-merge-block/106006 Google ScholarGoogle Scholar
  8. Sean Baxter. 2021. Tweet about SPIR-V control flow. https://twitter.com/seanbax/status/1348780718797807622 Google ScholarGoogle Scholar
  9. Adam Betts, Nathan Chong, Alastair F. Donaldson, Shaz Qadeer, and Paul Thomson. 2012. GPUVerify: a verifier for GPU kernels. In Proceedings of the 27th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2012, part of SPLASH 2012, Tucson, AZ, USA, October 21-25, 2012, Gary T. Leavens and Matthew B. Dwyer (Eds.). ACM, 113–132. https://doi.org/10.1145/2384616.2384625 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Junjie Chen, Jibesh Patra, Michael Pradel, Yingfei Xiong, Hongyu Zhang, Dan Hao, and Lu Zhang. 2020. A Survey of Compiler Testing. ACM Comput. Surv., 53, 1 (2020), 4:1–4:36. https://doi.org/10.1145/3363562 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Peter Collingbourne, Alastair F. Donaldson, Jeroen Ketema, and Shaz Qadeer. 2013. Interleaving and Lock-Step Semantics for Analysis and Verification of GPU Kernels. In Programming Languages and Systems - 22nd European Symposium on Programming, ESOP 2013, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2013, Rome, Italy, March 16-24, 2013. Proceedings, Matthias Felleisen and Philippa Gardner (Eds.) (Lecture Notes in Computer Science, Vol. 7792). Springer, 270–289. https://doi.org/10.1007/978-3-642-37036-6_16 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. David Neto. 2022. SPIR-V samples for WebGPU. https://github.com/dneto0/spirv-samples Google ScholarGoogle Scholar
  13. Alastair F. Donaldson, Hugues Evrard, Andrei Lascu, and Paul Thomson. 2017. Automated testing of graphics shader compilers. PACMPL, 1, OOPSLA (2017), 93:1–93:29. https://doi.org/10.1145/3133917 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Alastair F. Donaldson, Paul Thomson, Vasyl Teliman, Stefano Milizia, André Perez Maselco, and Antoni Karpinski. 2021. Test-case reduction and deduplication almost for free with transformation-based compiler testing. In PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 2021, Stephen N. Freund and Eran Yahav (Eds.). ACM, 1017–1032. https://doi.org/10.1145/3453483.3454092 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jason Ekstrand. 2022. In defense of NIR. https://www.jlekstrand.net/jason/blog/2022/01/in-defense-of-nir/ Google ScholarGoogle Scholar
  16. Benedict R. Gaster, Derek Hower, and Lee W. Howes. 2015. HRF-Relaxed: Adapting HRF to the Complexities of Industrial Heterogeneous Memory Models. ACM Trans. Archit. Code Optim., 12, 1 (2015), 7:1–7:26. https://doi.org/10.1145/2701618 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Davide Di Gennaro. 2015. Advanced Metaprogramming in Classic C++. Apress. Google ScholarGoogle Scholar
  18. Google. 2022. Amber Repository. https://github.com/google/amber Google ScholarGoogle Scholar
  19. Google. 2022. ANGLE - Almost Native Graphics Layer Engine. https://chromium.googlesource.com/angle/angle Google ScholarGoogle Scholar
  20. Google. 2022. The clspv project. https://github.com/google/clspv Google ScholarGoogle Scholar
  21. Google. 2022. Dawn, a WebGPU implementation. https://dawn.googlesource.com/dawn Google ScholarGoogle Scholar
  22. Google. 2022. SwiftShader, CPU-based Vulkan Implementation. https://swiftshader.googlesource.com/SwiftShader Google ScholarGoogle Scholar
  23. Google. 2022. The Tint project. https://dawn.googlesource.com/tint Google ScholarGoogle Scholar
  24. Axel Habermaier and Alexander Knapp. 2012. On the Correctness of the SIMT Execution Model of GPUs. In Programming Languages and Systems - 21st European Symposium on Programming, ESOP 2012, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2012, Tallinn, Estonia, March 24 - April 1, 2012. Proceedings, Helmut Seidl (Ed.) (Lecture Notes in Computer Science, Vol. 7211). Springer, 316–335. https://doi.org/10.1007/978-3-642-28869-2_16 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Intel. 2020. Intel Iris Plus Graphics and UHD Graphics Open Source, Programmer’s Reference Manual For the 2019 10th Generation Intel Core Processors based on the “Ice Lake” Platform, Volume 2a. https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-icllp-vol02a-commandreference-instructions_2.pdf Google ScholarGoogle Scholar
  26. Intel. 2022. Mesa 3D Graphics Stack Repository. https://gitlab.freedesktop.org/mesa/mesa Google ScholarGoogle Scholar
  27. Dan Iorga, Alastair F. Donaldson, Tyler Sorensen, and John Wickerson. 2021. The semantics of shared memory in Intel CPU/FPGA systems. Proc. ACM Program. Lang., 5, OOPSLA (2021), 1–28. https://doi.org/10.1145/3485497 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Daniel Jackson. 2019. Alloy: a language and tool for exploring software designs. Commun. ACM, 62, 9 (2019), 66–76. https://doi.org/10.1145/3338843 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Yiming Jing, Gail-Joon Ahn, and Hongxin Hu. 2012. Model-Based Conformance Testing for Android. In Advances in Information and Computer Security - 7th International Workshop on Security, IWSEC 2012, Fukuoka, Japan, November 7-9, 2012. Proceedings, Goichiro Hanaoka and Toshihiro Yamauchi (Eds.) (Lecture Notes in Computer Science, Vol. 7631). Springer, 1–18. https://doi.org/10.1007/978-3-642-34117-5_1 Google ScholarGoogle ScholarCross RefCross Ref
  30. Khronos Group. 2014. SPIR Specification, Version 1.2. https://www.khronos.org/registry/SPIR/specs/spir_spec-1.2.pdf Google ScholarGoogle Scholar
  31. Khronos Group. 2017. SPIR-V Specification, Version 1.0, Revision 12. https://www.khronos.org/registry/SPIR-V/specs/1.0/SPIRV.html Google ScholarGoogle Scholar
  32. Khronos Group. 2019. The OpenGL Shading Language Version 4.60.7. https://www.khronos.org/registry/OpenGL/specs/gl/GLSLangSpec.4.60.pdf Google ScholarGoogle Scholar
  33. Khronos Group. 2022. A complete registry of all official SPIR-V specifications. https://www.khronos.org/registry/SPIR-V/ Google ScholarGoogle Scholar
  34. Khronos Group. 2022. glslang GitHub repository. https://github.com/KhronosGroup/glslang Google ScholarGoogle Scholar
  35. Khronos Group. 2022. Khronos Vulkan, OpenGL, and OpenGL ES conformance tests. https://github.com/KhronosGroup/VK-GL-CTS Google ScholarGoogle Scholar
  36. Khronos Group. 2022. MoltenVK, a Vulkan Portability Implementation. https://github.com/KhronosGroup/MoltenVK Google ScholarGoogle Scholar
  37. Khronos Group. 2022. OpenCL-Docs. https://github.com/KhronosGroup/OpenCL-Docs Google ScholarGoogle Scholar
  38. Khronos Group. 2022. SPIR-V Specification, Version 1.6, Revision 1, Unified. https://web.archive.org/web/20220613184046/https://www.khronos.org/registry/SPIR-V/specs/unified1/SPIRV.pdf Also cited as . Google ScholarGoogle Scholar
  39. Khronos Group. 2022. SPIR-V Specification, Version 1.6, Revision 2, Unified. https://www.khronos.org/registry/SPIR-V/specs/unified1/SPIRV.html Also cited as . Google ScholarGoogle Scholar
  40. Khronos Group. 2022. SPIRV-Cross Repository. https://github.com/KhronosGroup/SPIRV-Crosss Google ScholarGoogle Scholar
  41. Khronos Group. 2022. SPIRV-Tools Repository, including spirv-opt and spirv-val. https://github.com/KhronosGroup/SPIRV-Tools Google ScholarGoogle Scholar
  42. Khronos Group. 2022. Vulkan 1.3 - A Specification (with all registered Vulkan extensions). https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html Google ScholarGoogle Scholar
  43. Vasileos Klimis, Jack Clark, Alan Baker, David Neto, John Wickerson, and Alastair F. Donaldson. 2022. Artifact for “Taking Back Control in an Intermediate Representation for GPU Computing”, POPL 2023. https://doi.org/10.5281/zenodo.7152484 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Vasileios Klimis, Jack Clark, John Wickerson, and Alastair F. Donaldson. 2022. Repository containing SPIR-V control flow Alloy model and fuzzer.. https://github.com/mc-imperial/spirv-control-flow Google ScholarGoogle Scholar
  45. Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom - June 09 - 11, 2014, Michael F. P. O’Boyle and Keshav Pingali (Eds.). ACM, 216–226. https://doi.org/10.1145/2594291.2594334 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Christopher Lidbury, Andrei Lascu, Nathan Chong, and Alastair F. Donaldson. 2015. Many-core compiler fuzzing. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, June 15-17, 2015, David Grove and Steve Blackburn (Eds.). ACM, 65–76. isbn:978-1-4503-3468-6 https://doi.org/10.1145/2737924.2737986 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Vsevolod Livinskii, Dmitry Babokin, and John Regehr. 2020. Random testing for C and C++ compilers with YARPGen. Proc. ACM Program. Lang., 4, OOPSLA (2020), 196:1–196:25. https://doi.org/10.1145/3428264 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. LLVM Compiler Infrastructure. 2022. LLVM Language Reference Manual. https://llvm.org/docs/LangRef.html Google ScholarGoogle Scholar
  49. Daniel Lustig, Sameer Sahasrabuddhe, and Olivier Giroux. 2019. A Formal Analysis of the NVIDIA PTX Memory Consistency Model. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2019, Providence, RI, USA, April 13-17, 2019, Iris Bahar, Maurice Herlihy, Emmett Witchel, and Alvin R. Lebeck (Eds.). ACM, 257–270. https://doi.org/10.1145/3297858.3304043 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Daniel Lustig, Andrew Wright, Alexandros Papakonstantinou, and Olivier Giroux. 2017. Automated Synthesis of Comprehensive Memory Model Litmus Test Suites. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2017, Xi’an, China, April 8-12, 2017, Yunji Chen, Olivier Temam, and John Carter (Eds.). ACM, 661–675. https://doi.org/10.1145/3037697.3037723 Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Nuno Macedo and Alcino Cunha. 2012. Automatic Unbounded Verification of Alloy Specifications with Prover9. CoRR, abs/1209.5773 (2012), 17 pages. arXiv:1209.5773. arxiv:1209.5773 Google ScholarGoogle Scholar
  52. Dzmitry Malyshau. 2021. Horrors of SPIR-V. http://kvark.github.io/spirv/2021/05/01/spirv-horrors.html Google ScholarGoogle Scholar
  53. Microsoft. 2019. Reference for HLSL. https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-reference Google ScholarGoogle Scholar
  54. Microsoft. 2022. DirectX Shader Compiler Repository. https://github.com/microsoft/DirectXShaderCompiler Google ScholarGoogle Scholar
  55. Eriko Nagai, Atsushi Hashimoto, and Nagisa Ishiura. 2014. Reinforcing Random Testing of Arithmetic Optimization of C Compilers by Scaling up Size and Number of Expressions. IPSJ Trans. Syst. LSI Des. Methodol., 7 (2014), 91–100. https://doi.org/10.2197/ipsjtsldm.7.91 Google ScholarGoogle ScholarCross RefCross Ref
  56. Oded Padon, Kenneth L. McMillan, Aurojit Panda, Mooly Sagiv, and Sharon Shoham. 2016. Ivy: safety verification by interactive generalization. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2016, Santa Barbara, CA, USA, June 13-17, 2016, Chandra Krintz and Emery D. Berger (Eds.). ACM, 614–630. https://doi.org/10.1145/2908080.2908118 Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Azalea Raad, John Wickerson, Gil Neiger, and Viktor Vafeiadis. 2020. Persistency semantics of the Intel-x86 architecture. Proc. ACM Program. Lang., 4, POPL (2020), 11:1–11:31. https://doi.org/10.1145/3371079 Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Azalea Raad, John Wickerson, and Viktor Vafeiadis. 2019. Weak persistency semantics from the ground up: formalising the persistency semantics of ARMv8 and transactional models. Proc. ACM Program. Lang., 3, OOPSLA (2019), 135:1–135:27. https://doi.org/10.1145/3360561 Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Philip Rebohle. 2022. DXVK. https://github.com/doitsujin/dxvk/ Google ScholarGoogle Scholar
  60. Rust Graphics Mages. 2022. The Naga project. https://github.com/gfx-rs/naga Google ScholarGoogle Scholar
  61. Mark Segal and Kurt Akeley. 2022. OpenGL 4.6 Core Profile. https://www.khronos.org/registry/OpenGL/specs/gl/glspec46.core.pdf Google ScholarGoogle Scholar
  62. Tyler Sorensen, Lucas F. Salvador, Harmit Raval, Hugues Evrard, John Wickerson, Margaret Martonosi, and Alastair F. Donaldson. 2021. Specifying and testing GPU workgroup progress models. Proc. ACM Program. Lang., 5, OOPSLA (2021), 1–30. https://doi.org/10.1145/3485508 Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Alfred Tarski. 1941. On the Calculus of Relations. J. Symb. Log., 6, 3 (1941), 73–89. https://doi.org/10.2307/2268577 Google ScholarGoogle ScholarCross RefCross Ref
  64. W3C. 2022. WebGPU Shading Language W3C Working Draft. https://www.w3.org/TR/WGSL/ Google ScholarGoogle Scholar
  65. Chunhui Wang, Fabrizio Pastore, Arda Goknil, and Lionel C. Briand. 2022. Automatic Generation of Acceptance Test Cases From Use Case Specifications: An NLP-Based Approach. IEEE Trans. Software Eng., 48, 2 (2022), 585–616. https://doi.org/10.1109/TSE.2020.2998503 Google ScholarGoogle ScholarCross RefCross Ref
  66. John Wickerson, Mark Batty, Tyler Sorensen, and George A. Constantinides. 2017. Automatically comparing memory consistency models. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18-20, 2017, Giuseppe Castagna and Andrew D. Gordon (Eds.). ACM, 190–204. https://doi.org/10.1145/3009837.3009838 Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and understanding bugs in C compilers. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, San Jose, CA, USA, June 4-8, 2011, Mary W. Hall and David A. Padua (Eds.). ACM, 283–294. https://doi.org/10.1145/1993498.1993532 Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Qirun Zhang, Chengnian Sun, and Zhendong Su. 2017. Skeletal program enumeration for rigorous compiler testing. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2017, Barcelona, Spain, June 18-23, 2017, Albert Cohen and Martin T. Vechev (Eds.). ACM, 347–361. https://doi.org/10.1145/3062341.3062379 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Taking Back Control in an Intermediate Representation for GPU Computing

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Article Metrics

      • Downloads (Last 12 months)216
      • Downloads (Last 6 weeks)18

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!