Abstract
The interest in quantum computing is growing, and with it, the importance of software platforms to develop quantum programs. Ensuring the correctness of such platforms is important, and it requires a thorough understanding of the bugs they typically suffer from. To address this need, this paper presents the first in-depth study of bugs in quantum computing platforms. We gather and inspect a set of 223 real-world bugs from 18 open-source quantum computing platforms. Our study shows that a significant fraction of these bugs (39.9%) are quantum-specific, calling for dedicated approaches to prevent and find them. The bugs are spread across various components, but quantum-specific bugs occur particularly often in components that represent, compile, and optimize quantum programming abstractions. Many quantum-specific bugs manifest through unexpected outputs, rather than more obvious signs of misbehavior, such as crashes. Finally, we present a hierarchy of recurrent bug patterns, including ten novel, quantum-specific patterns. Our findings not only show the importance and prevalence bugs in quantum computing platforms, but they help developers to avoid common mistakes and tool builders to tackle the challenge of preventing, finding, and fixing these bugs.
- 2021. Overview on Quantum Initiatives Worldwide - Update Mid 2021. https://www.qureca.com/overview-on-quantum-initiatives-worldwide-update-mid-2021/.Google Scholar
- 2021. Qiskit/Qiskit. https://github.com/Qiskit/qiskit.Google Scholar
- Edward Aftandilian, Raluca Sauciuc, Siddharth Priya, and Sundaresan Krishnan. 2012. Building Useful Program Analysis Tools Using an Extensible Java Compiler. In 2012 IEEE 12th International Working Conference on Source Code Analysis and Manipulation. 14–23. https://doi.org/10.1109/SCAM.2012.28 Google Scholar
Digital Library
- Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.Google Scholar
- Johannes Bader, Andrew Scott, Michael Pradel, and Satish Chandra. 2019. Getafix: Learning to Fix Bugs Automatically. Proceedings of the ACM on Programming Languages, 3, OOPSLA (2019), Oct., 159:1–159:27. https://doi.org/10.1145/3360585 Google Scholar
Digital Library
- Gergö Barany. 2018. Finding Missed Compiler Optimizations by Differential Testing. In Proceedings of the 27th International Conference on Compiler Construction (CC 2018). Association for Computing Machinery, New York, NY, USA. 82–92. isbn:978-1-4503-5644-2 https://doi.org/10.1145/3178372.3179521 Google Scholar
Digital Library
- Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin Vechev. 2021. TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer. In Proceedings of the 38th International Conference on Machine Learning. PMLR, 780–791. issn:2640-3498Google Scholar
- Ville Bergholm, Josh Izaac, Maria Schuld, Christian Gogolin, M. Sohaib Alam, Shahnawaz Ahmed, Juan Miguel Arrazola, Carsten Blank, Alain Delgado, Soran Jahangiri, Keri McKiernan, Johannes Jakob Meyer, Zeyue Niu, Antal Száva, and Nathan Killoran. 2020. PennyLane: Automatic Differentiation of Hybrid Quantum-Classical Computations. arXiv:1811.04968 [physics, physics:quant-ph], Feb., arxiv:1811.04968.Google Scholar
- Benjamin Bichsel, Maximilian Baader, Timon Gehr, and Martin T. Vechev. 2020. Silq: a high-level quantum language with safe uncomputation and intuitive semantics. In Proceedings of the 41st ACM SIGPLAN International Conference on Programming Language Design and Implementation, PLDI 2020, London, UK, June 15-20, 2020, Alastair F. Donaldson and Emina Torlak (Eds.). ACM, 286–300. https://doi.org/10.1145/3385412.3386007 Google Scholar
Digital Library
- José Campos and André Souto. 2021. QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments. arXiv:2103.16968 [cs], March, arxiv:2103.16968.Google Scholar
- Junjie Chen, Jibesh Patra, Michael Pradel, Yingfei Xiong, Hongyu Zhang, Dan Hao, and Lu Zhang. 2020. A Survey of Compiler Testing. ACM Comput. Surv., 53, 1 (2020), 4:1–4:36. https://doi.org/10.1145/3363562 Google Scholar
Digital Library
- Andy Chou, Junfeng Yang, Benjamin Chelf, Seth Hallem, and Dawson Engler. 2001. An Empirical Study of Operating Systems Errors. In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (SOSP ’01). Association for Computing Machinery, New York, NY, USA. 73–88. isbn:978-1-58113-389-9 https://doi.org/10.1145/502034.502042 Google Scholar
Digital Library
- Andrew W. Cross, Lev S. Bishop, John A. Smolin, and Jay M. Gambetta. 2017. Open Quantum Assembly Language. arXiv:1707.03429 [quant-ph], July, arxiv:1707.03429.Google Scholar
- Cirq Developers. 2021. Cirq. Zenodo. https://doi.org/10.5281/zenodo.5182845 Google Scholar
Cross Ref
- Elizabeth Dinella, Hanjun Dai, Ziyang Li, Mayur Naik, Le Song, and Ke Wang. 2020. Hoppity: Learning Graph Transformations to Detect and Fix Bugs in Programs. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=SJeqs6EFvBGoogle Scholar
- Aryaz Eghbali and Michael Pradel. 2020. No Strings Attached: An Empirical Study of String-Related Software Bugs. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (ASE ’20). Association for Computing Machinery, New York, NY, USA. 956–967. isbn:978-1-4503-6768-4 https://doi.org/10.1145/3324884.3416576 Google Scholar
Digital Library
- Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. 2014. A Quantum Approximate Optimization Algorithm. arXiv:1411.4028 [quant-ph], Nov., arxiv:1411.4028.Google Scholar
- Mark Fingerhuth, Tomáš Babej, and Peter Wittek. 2018. Open Source Software in Quantum Computing. PLOS ONE, 13, 12 (2018), Dec., e0208561. issn:1932-6203 https://doi.org/10.1371/journal.pone.0208561 Google Scholar
Cross Ref
- Doug Finke. 2021. Relative Popularity of Different Quantum Programming Platforms - Quantum Computing Report. https://web.archive.org/web/20210619213740/https://quantumcomputingreport.com/relative-popularity-of-different-quantum-programming-platforms/.Google Scholar
- Xiang Gao, Shraddha Barke, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani, Alan Leung, Nachiappan Nagappan, and Ashish Tiwari. 2020. Feedback-driven semi-supervised synthesis of program transformations. Proc. ACM Program. Lang., 4, OOPSLA (2020), 219:1–219:30. https://doi.org/10.1145/3428287 Google Scholar
Digital Library
- Constantin Gonzalez. 2021. Cloud Based QC with Amazon Braket. Digitale Welt, 5, 2 (2021), April, 14–17. issn:2569-1996 https://doi.org/10.1007/s42354-021-0330-z Google Scholar
Cross Ref
- Alexander S. Green, Peter LeFanu Lumsdaine, Neil J. Ross, Peter Selinger, and Benoît Valiron. 2013. Quipper: a scalable quantum programming language. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, Seattle, WA, USA, June 16-19, 2013, Hans-Juergen Boehm and Cormac Flanagan (Eds.). ACM, 333–342. https://doi.org/10.1145/2491956.2462177 Google Scholar
Digital Library
- Alexander S. Green, Peter LeFanu Lumsdaine, Neil J. Ross, Peter Selinger, and Benoît Valiron. 2013. Quipper: A Scalable Quantum Programming Language. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’13). Association for Computing Machinery, New York, NY, USA. 333–342. isbn:978-1-4503-2014-6 https://doi.org/10.1145/2491956.2462177 Google Scholar
Digital Library
- Lov K. Grover. 1996. A Fast Quantum Mechanical Algorithm for Database Search. In Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing (STOC ’96). Association for Computing Machinery, New York, NY, USA. 212–219. isbn:978-0-89791-785-8 https://doi.org/10.1145/237814.237866 Google Scholar
Digital Library
- Xue Han and Tingting Yu. 2016. An Empirical Study on Performance Bugs for Highly Configurable Software Systems. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM ’16). Association for Computing Machinery, New York, NY, USA. 1–10. isbn:978-1-4503-4427-2 https://doi.org/10.1145/2961111.2962602 Google Scholar
Digital Library
- Thomas Häner, Torsten Hoefler, and Matthias Troyer. 2020. Assertion-based optimization of Quantum programs. Proc. ACM Program. Lang., 4, OOPSLA (2020), 133:1–133:20. https://doi.org/10.1145/3428201 Google Scholar
Digital Library
- Aram W. Harrow, Avinatan Hassidim, and Seth Lloyd. 2009. Quantum Algorithm for Solving Linear Systems of Equations. Physical Review Letters, 103, 15 (2009), Oct., 150502. issn:0031-9007, 1079-7114 https://doi.org/10.1103/PhysRevLett.103.150502 arxiv:0811.3171. Google Scholar
Cross Ref
- Vincent J. Hellendoorn, Charles Sutton, Rishabh Singh, Petros Maniatis, and David Bieber. 2020. Global Relational Models of Source Code. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=B1lnbRNtwrGoogle Scholar
- Kesha Hietala, Robert Rand, Shih-Han Hung, Xiaodi Wu, and Michael Hicks. 2021. A Verified Optimizer for Quantum Circuits. Proceedings of the ACM on Programming Languages, 5, POPL (2021), Jan., 37:1–37:29. https://doi.org/10.1145/3434318 Google Scholar
Digital Library
- Yipeng Huang and Margaret Martonosi. 2019. QDB: From Quantum Algorithms Towards Correct Quantum Programs. arXiv:1811.05447 [quant-ph], 14 pages. https://doi.org/10.4230/OASIcs.PLATEAU.2018.4 arxiv:1811.05447. Google Scholar
Cross Ref
- Yipeng Huang and Margaret Martonosi. 2019. Statistical Assertions for Validating Patterns and Finding Bugs in Quantum Programs. In Proceedings of the 46th International Symposium on Computer Architecture (ISCA ’19). Association for Computing Machinery, New York, NY, USA. 541–553. isbn:978-1-4503-6669-4 https://doi.org/10.1145/3307650.3322213 Google Scholar
Digital Library
- Md Johirul Islam, Giang Nguyen, Rangeet Pan, and Hridesh Rajan. 2019. A Comprehensive Study on Deep Learning Bug Characteristics. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2019). Association for Computing Machinery, New York, NY, USA. 510–520. isbn:978-1-4503-5572-8 https://doi.org/10.1145/3338906.3338955 Google Scholar
Digital Library
- Guoliang Jin, Linhai Song, Xiaoming Shi, Joel Scherpelz, and Shan Lu. 2012. Understanding and Detecting Real-World Performance Bugs. ACM SIGPLAN Notices, 47, 6 (2012), June, 77–88. issn:0362-1340 https://doi.org/10.1145/2345156.2254075 Google Scholar
Digital Library
- Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020. Learning and Evaluating Contextual Embedding of Source Code. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 5110–5121. https://doi.org/10.48550/ARXIV.2001.00059Google Scholar
- Rafael-Michael Karampatsis and Charles Sutton. 2020. How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset. In Proceedings of the 17th International Conference on Mining Software Repositories (MSR ’20). Association for Computing Machinery, New York, NY, USA. 573–577. isbn:978-1-4503-7517-7 https://doi.org/10.1145/3379597.3387491 Google Scholar
Digital Library
- Jakob S. Kottmann, Sumner Alperin-Lea, Teresa Tamayo-Mendoza, Alba Cervera-Lierta, Cyrille Lavigne, Tzu-Ching Yen, Vladyslav Verteletskyi, Philipp Schleich, Abhinav Anand, Matthias Degroote, Skylar Chaney, Maha Kesibi, Naomi Grace Curnow, Brandon Solo, Georgios Tsilimigkounakis, Claudia Zendejas-Morales, Artur F. Izmaylov, and Alán Aspuru-Guzik. 2021. Tequila: A Platform for Rapid Development of Quantum Algorithms. Quantum Science and Technology, 6, 2 (2021), April, 024009. issn:2058-9565 https://doi.org/10.1088/2058-9565/abe567 arxiv:2011.03057. Google Scholar
Cross Ref
- Ryan LaRose, Andrea Mari, Sarah Kaiser, Peter J. Karalekas, Andre A. Alves, Piotr Czarnik, Mohamed El Mandouh, Max H. Gordon, Yousef Hindy, Aaron Robertson, Purva Thakre, Nathan Shammah, and William J. Zeng. 2021. Mitiq: A Software Package for Error Mitigation on Noisy Quantum Computers. arXiv:2009.04417 [quant-ph], Aug., arxiv:2009.04417.Google Scholar
- Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom - June 09 - 11, 2014, Michael F. P. O’Boyle and Keshav Pingali (Eds.). ACM, 216–226. https://doi.org/10.1145/2594291.2594334 Google Scholar
Digital Library
- Vu Le, Chengnian Sun, and Zhendong Su. 2015. Finding Deep Compiler Bugs via Guided Stochastic Program Mutation. ACM SIGPLAN Notices, 50, 10 (2015), Oct., 386–399. issn:0362-1340 https://doi.org/10.1145/2858965.2814319 Google Scholar
Digital Library
- Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019. Automated program repair. Commun. ACM, 62, 12 (2019), 56–65. https://doi.org/10.1145/3318162 Google Scholar
Digital Library
- Xavier Leroy. 2009. Formal Verification of a Realistic Compiler. Commun. ACM, 52, 7 (2009), July, 107–115. issn:0001-0782 https://doi.org/10.1145/1538788.1538814 Google Scholar
Digital Library
- Gushu Li, Li Zhou, Nengkun Yu, Yufei Ding, Mingsheng Ying, and Yuan Xie. 2020. Projection-based runtime assertions for testing and debugging Quantum programs. Proc. ACM Program. Lang., 4, OOPSLA (2020), 150:1–150:29. https://doi.org/10.1145/3428218 Google Scholar
Digital Library
- Yi Li, Shaohua Wang, and Tien N. Nguyen. 2020. DLFix: Context-Based Code Transformation Learning for Automated Program Repair. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (ICSE ’20). Association for Computing Machinery, New York, NY, USA. 602–614. isbn:978-1-4503-7121-6 https://doi.org/10.1145/3377811.3380345 Google Scholar
Digital Library
- Z. Li, S. Lu, S. Myagmar, and Y. Zhou. 2006. CP-Miner: Finding Copy-Paste and Related Bugs in Large-Scale Software Code. IEEE Transactions on Software Engineering, 32, 3 (2006), March, 176–192. issn:1939-3520 https://doi.org/10.1109/TSE.2006.28 Google Scholar
Digital Library
- Shan Lu, Soyeon Park, Eunsoo Seo, and Yuanyuan Zhou. 2008. Learning from Mistakes: A Comprehensive Study on Real World Concurrency Bug Characteristics. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIII). Association for Computing Machinery, New York, NY, USA. 329–339. isbn:978-1-59593-958-6 https://doi.org/10.1145/1346281.1346323 Google Scholar
Digital Library
- Tyler McDonnell, Baishakhi Ray, and Miryung Kim. 2013. An Empirical Study of API Stability and Adoption in the Android Ecosystem. In 2013 IEEE International Conference on Software Maintenance, Eindhoven, The Netherlands, September 22-28, 2013. IEEE Computer Society, 70–79. https://doi.org/10.1109/ICSM.2013.18 Google Scholar
Digital Library
- William M. McKeeman. 1998. Differential Testing for Software. Digit. Tech. J., 10, 1 (1998), 100–107.Google Scholar
- Giulia Meuli, Mathias Soeken, Martin Roetteler, and Thomas Häner. 2020. Enabling accuracy-aware Quantum compilers using symbolic resource estimation. Proc. ACM Program. Lang., 4, OOPSLA (2020), 130:1–130:26. https://doi.org/10.1145/3428198 Google Scholar
Digital Library
- Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, and Abhishek Udupa. 2019. On the fly synthesis of edit suggestions. PACMPL, 3, OOPSLA (2019), 143:1–143:29. https://doi.org/10.1145/3360569 Google Scholar
Digital Library
- Prakash Murali, David C. Mckay, Margaret Martonosi, and Ali Javadi-Abhari. 2020. Software Mitigation of Crosstalk on Noisy Intermediate-Scale Quantum Computers. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’20). Association for Computing Machinery, New York, NY, USA. 1001–1016. isbn:978-1-4503-7102-5 https://doi.org/10.1145/3373376.3378477 Google Scholar
Digital Library
- Anouk Paradis, Benjamin Bichsel, Samuel Steffen, and Martin T. Vechev. 2021. Unqomp: synthesizing uncomputation in Quantum circuits. In PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 20211, Stephen N. Freund and Eran Yahav (Eds.). ACM, 222–236. https://doi.org/10.1145/3453483.3454040 Google Scholar
Digital Library
- Hung Viet Pham, Thibaud Lutellier, Weizhen Qi, and Lin Tan. 2019. CRADLE: cross-backend validation to detect and localize bugs in deep learning libraries. In Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019, Joanne M. Atlee, Tevfik Bultan, and Jon Whittle (Eds.). IEEE / ACM, 1027–1038. https://doi.org/10.1109/ICSE.2019.00107 Google Scholar
Digital Library
- Md Rafiqul Islam Rabin, Vincent J. Hellendoorn, and Mohammad Amin Alipour. 2021. Understanding Neural Code Intelligence through Program Simplification. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021). Association for Computing Machinery, New York, NY, USA. 441–452. isbn:978-1-4503-8562-6 https://doi.org/10.1145/3468264.3468539 Google Scholar
Digital Library
- Baishakhi Ray, Vincent Hellendoorn, Saheel Godhane, Zhaopeng Tu, Alberto Bacchelli, and Premkumar Devanbu. 2016. On the "Naturalness" of Buggy Code. In Proceedings of the 38th International Conference on Software Engineering (ICSE ’16). Association for Computing Machinery, New York, NY, USA. 428–439. isbn:978-1-4503-3900-1 https://doi.org/10.1145/2884781.2884848 Google Scholar
Digital Library
- Andrew Rice, Edward Aftandilian, Ciera Jaspan, Emily Johnston, Michael Pradel, and Yulissa Arroyo-Paredes. 2017. Detecting Argument Selection Defects. Proceedings of the ACM on Programming Languages, 1, OOPSLA (2017), Oct., 104:1–104:22. https://doi.org/10.1145/3133928 Google Scholar
Digital Library
- Marija Selakovic and Michael Pradel. 2016. Performance Issues and Optimizations in JavaScript: An Empirical Study. In Proceedings of the 38th International Conference on Software Engineering (ICSE ’16). Association for Computing Machinery, New York, NY, USA. 61–72. isbn:978-1-4503-3900-1 https://doi.org/10.1145/2884781.2884829 Google Scholar
Digital Library
- Qingchao Shen, Haoyang Ma, Junjie Chen, Yongqiang Tian, Shing-Chi Cheung, and Xiang Chen. 2021. A Comprehensive Study of Deep Learning Compiler Bugs. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021). Association for Computing Machinery, New York, NY, USA. 968–980. isbn:978-1-4503-8562-6 https://doi.org/10.1145/3468264.3468591 Google Scholar
Digital Library
- Yunong Shi, Runzhou Tao, Xupeng Li, Ali Javadi-Abhari, Andrew W. Cross, Frederic T. Chong, and Ronghui Gu. 2020. CertiQ: A Mostly-automated Verification of a Realistic Quantum Compiler. arXiv:1908.08963 [quant-ph], Nov., arxiv:1908.08963.Google Scholar
- Peter W. Shor. 1999. Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer. SIAM Rev., 41, 2 (1999), Jan., 303–332. issn:0036-1445 https://doi.org/10.1137/S0036144598347011 Google Scholar
Digital Library
- Chengnian Sun, Vu Le, Qirun Zhang, and Zhendong Su. 2016. Toward understanding compiler bugs in GCC and LLVM. In Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18-20, 2016, Andreas Zeller and Abhik Roychoudhury (Eds.). ACM, 294–305. https://doi.org/10.1145/2931037.2931074 Google Scholar
Digital Library
- Yasunari Suzuki, Yoshiaki Kawase, Yuya Masumura, Yuria Hiraga, Masahiro Nakadai, Jiabao Chen, Ken M. Nakanishi, Kosuke Mitarai, Ryosuke Imai, Shiro Tamiya, Takahiro Yamamoto, Tennin Yan, Toru Kawakubo, Yuya O. Nakagawa, Yohei Ibe, Youyuan Zhang, Hirotsugu Yamashita, Hikaru Yoshimura, Akihiro Hayashi, and Keisuke Fujii. 2021. Qulacs: A Fast and Versatile Quantum Circuit Simulator for Research Purpose. Quantum, 5 (2021), Oct., 559. https://doi.org/10.22331/q-2021-10-06-559 Google Scholar
Cross Ref
- Krysta Svore, Alan Geller, Matthias Troyer, John Azariah, Christopher Granade, Bettina Heim, Vadym Kliuchnikov, Mariia Mykhailova, Andres Paz, and Martin Roetteler. 2018. Q#: Enabling Scalable Quantum Computing and Development with a High-level DSL. In Proceedings of the Real World Domain Specific Languages Workshop 2018 (RWDSL2018). Association for Computing Machinery, New York, NY, USA. 1–10. isbn:978-1-4503-6355-6 https://doi.org/10.1145/3183895.3183901 Google Scholar
Digital Library
- Runzhou Tao, Yunong Shi, Jianan Yao, John Hui, Frederic T. Chong, and Ronghui Gu. 2021. Gleipnir: toward practical error analysis for Quantum programs. In PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 20211, Stephen N. Freund and Eran Yahav (Eds.). ACM, 48–64. https://doi.org/10.1145/3453483.3454029 Google Scholar
Digital Library
- Viktor Vafeiadis, Thibaut Balabonski, Soham Chakraborty, Robin Morisset, and Francesco Zappa Nardelli. 2015. Common Compiler Optimisations Are Invalid in the C11 Memory Model and What We Can Do about It. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’15). Association for Computing Machinery, New York, NY, USA. 209–220. isbn:978-1-4503-3300-9 https://doi.org/10.1145/2676726.2676995 Google Scholar
Digital Library
- Marko Vasic, Aditya Kanade, Petros Maniatis, David Bieber, and Rishabh Singh. 2018. Neural Program Repair by Jointly Learning to Localize and Repair. In International Conference on Learning Representations.Google Scholar
- Jiyuan Wang, Qian Zhang, Guoqing Harry Xu, and Miryung Kim. 2021. QDiff: Differential Testing of Quantum Software Stacks. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 692–704. issn:2643-1572 https://doi.org/10.1109/ASE51524.2021.9678792 Google Scholar
Digital Library
- Song Wang, Nishtha Shrestha, Abarna Kucheri Subburaman, Junjie Wang, Moshi Wei, and Nachiappan Nagappan. 2021. Automatic Unit Test Generation for Machine Learning Libraries: How Far Are We? In 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, 22-30 May 2021. IEEE, 1548–1560. https://doi.org/10.1109/ICSE43902.2021.00138 Google Scholar
Digital Library
- W. K. Wootters and W. H. Zurek. 1982. A Single Quantum Cannot Be Cloned. Nature, 299, 5886 (1982), Oct., 802–803. issn:1476-4687 https://doi.org/10.1038/299802a0 Google Scholar
Cross Ref
- Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and Understanding Bugs in C Compilers. ACM SIGPLAN Notices, 46, 6 (2011), June, 283–294. issn:0362-1340 https://doi.org/10.1145/1993316.1993532 Google Scholar
Digital Library
- Nengkun Yu and Jens Palsberg. 2021. Quantum abstract interpretation. In PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, Virtual Event, Canada, June 20-25, 20211, Stephen N. Freund and Eran Yahav (Eds.). ACM, 542–558. https://doi.org/10.1145/3453483.3454061 Google Scholar
Digital Library
- Qirun Zhang, Chengnian Sun, and Zhendong Su. 2017. Skeletal Program Enumeration for Rigorous Compiler Testing. ACM SIGPLAN Notices, 52, 6 (2017), June, 347–361. issn:0362-1340 https://doi.org/10.1145/3140587.3062379 Google Scholar
Digital Library
- Jianjun Zhao. 2021. Quantum Software Engineering: Landscapes and Horizons. arXiv:2007.07047 [quant-ph], Dec., arxiv:2007.07047.Google Scholar
- Pengzhan Zhao, Jianjun Zhao, and Lei Ma. 2021. Identifying Bug Patterns in Quantum Programs. In 2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE). IEEE Computer Society, 16–21. isbn:978-1-66544-462-0 https://doi.org/10.1109/Q-SE52541.2021.00011 Google Scholar
Cross Ref
- Pengzhan Zhao, Jianjun Zhao, Zhongtao Miao, and Shuhan Lan. 2021. Bugs4Q: A Benchmark of Real Bugs for Quantum Programs. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1373–1376. issn:2643-1572 https://doi.org/10.1109/ASE51524.2021.9678908 Google Scholar
Digital Library
- Hao Zhong, Na Meng, Zexuan Li, and Li Jia. 2020. An Empirical Study on API Parameter Rules. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (ICSE ’20). Association for Computing Machinery, New York, NY, USA. 899–911. isbn:978-1-4503-7121-6 https://doi.org/10.1145/3377811.3380922 Google Scholar
Digital Library
Index Terms
Bugs in Quantum computing platforms: an empirical study
Recommendations
An empirical study of dormant bugs
MSR 2014: Proceedings of the 11th Working Conference on Mining Software RepositoriesOver the past decade, several research efforts have studied the quality of software systems by looking at post-release bugs. However, these studies do not account for bugs that remain dormant (i.e., introduced in a version of the software system, but ...
An Empirical Study of Bug Fixing Rate
COMPSAC '15: Proceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 02Bug fixing is one of the most important activities in software development and maintenance. A software project often employs an issue tracking system such as Bugzilla to store and manage their bugs. In the issue tracking system, many bugs are invalid ...
An empirical study on bugs in JavaScript engines
Abstract Context:JavaScript is a prototype-based dynamic type scripting language. The correct running of a JavaScript program depends on the correctness of both the program and the JavaScript engine.
Objective: ...






Comments