skip to main content
research-article

Log-related Coding Patterns to Conduct Postmortems of Attacks in Supervised Learning-based Projects

Published:12 April 2023Publication History
Skip Abstract Section

Abstract

Adversarial attacks against supervised learninga algorithms, which necessitates the application of logging while using supervised learning algorithms in software projects. Logging enables practitioners to conduct postmortem analysis, which can be helpful to diagnose any conducted attacks. We conduct an empirical study to identify and characterize log-related coding patterns, i.e., recurring coding patterns that can be leveraged to conduct adversarial attacks and needs to be logged. A list of log-related coding patterns can guide practitioners on what to log while using supervised learning algorithms in software projects.

We apply qualitative analysis on 3,004 Python files used to implement 103 supervised learning-based software projects. We identify a list of 54 log-related coding patterns that map to six attacks related to supervised learning algorithms. Using Log Assistant to conduct Postmortems for Supervised Learning (LOPSUL), we quantify the frequency of the identified log-related coding patterns with 278 open-source software projects that use supervised learning. We observe log-related coding patterns to appear for 22% of the analyzed files, where training data forensics is the most frequently occurring category.

REFERENCES

  1. [1] ast — Abstract Syntax Trees. (n.d.). Retrieved from https://docs.python.org/3/library/ast.html.Google ScholarGoogle Scholar
  2. [2] Model Zoo: Discover open source deep learning code and pretrained models. (n.d.). Retrieved from https://modelzoo.co.Google ScholarGoogle Scholar
  3. [3] Agrawal Amritanshu, Rahman Akond, Krishna Rahul, Sobran Alexander, and Menzies Tim. 2018. We don’t need another hero? The impact of “heroes” on software development. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice. 245253.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Albert Kendra, Penney Jon, Schneier Bruce, and Kumar Ram Shankar Siva. 2020. Politics of adversarial machine learning. In Proceedings of the Towards Trustworthy ML: Rethinking Security and Privacy for ML Workshop, 8th International Conference on Learning Representations.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Authors Anonymous. 2020. Verifiability Package for Paper. Retrieved February 10, 2021 from https://figshare.com/s/689c268c1de59dc7c2bf.Google ScholarGoogle Scholar
  6. [6] Barik Titus, DeLine Robert, Drucker Steven, and Fisher Danyel. 2016. The bones of the system: A case study of logging and telemetry at microsoft. In Proceedings of the 2016 IEEE/ACM 38th International Conference on Software Engineering Companion. IEEE, 92101.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Barreno Marco, Nelson Blaine, Joseph Anthony D., and Tygar J. Doug. 2010. The security of machine learning. Machine Learning 81, 2 (2010), 121148.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Bhuyian Farzana Ahamed and Rahman Akond. 2022. Source Code of LOPSUL. (2022). Retrieved from https://github.com/paser-group/MLForensics.Google ScholarGoogle Scholar
  9. [9] Carlini Nicholas and Wagner David. 2018. Audio adversarial examples: Targeted attacks on speech-to-text. In Proceedings of the 2018 IEEE Security and Privacy Workshops. IEEE, 17.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Chen Bryant, Carvalho Wilka, Baracaldo Nathalie, Ludwig Heiko, Edwards Benjamin, Lee Taesung, Molloy Ian, and Srivastava Biplav. 2018. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv:1811.03728. Retrieved from https://arxiv.org/abs/1811.03728.Google ScholarGoogle Scholar
  11. [11] Chen Boyuan and Jiang Zhen Ming. 2017. Characterizing and detecting anti-patterns in the logging code. In Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering. IEEE, 7181.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Chen Boyuan and Jiang Zhen Ming. 2020. Studying the use of Java logging utilities in the wild. In Proceedings of the 2020 IEEE/ACM 42nd International Conference on Software Engineering. IEEE, 397408.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Chen Boyuan and Jiang Zhen Ming Jack. 2019. Extracting and studying the Logging-Code-Issue-Introducing changes in Java-based large-scale open source software systems. Empirical Software Engineering 24, 4 (2019), 22852322.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Cohen Jacob. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 1 (1960), 3746. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Crabtree Benjamin F. and Miller William L.. 1999. Doing Qualitative Research. Sage Publications.Google ScholarGoogle Scholar
  16. [16] Ding Rui, Zhou Hucheng, Lou Jian-Guang, Zhang Hongyu, Lin Qingwei, Fu Qiang, Zhang Dongmei, and Xie Tao. 2015. Log2: A cost-aware logging mechanism for performance diagnosis. In Proceedings of the 2015 Annual Technical Conference. 139150.Google ScholarGoogle Scholar
  17. [17] Ebrahimi Javid, Rao Anyi, Lowd Daniel, and Dou Dejing. 2018. HotFlip: White-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 3136. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Evtimov Ivan, Eykholt Kevin, Fernandes Earlence, Kohno Tadayoshi, Li Bo, Prakash Atul, Rahmati Amir, and Song Dawn. 2017. Robust physical-world attacks on machine learning models. arXiv preprint arXiv:1707.08945 2, 3 (2017), 4.Google ScholarGoogle Scholar
  19. [19] Eykholt Kevin, Evtimov Ivan, Fernandes Earlence, Li Bo, Rahmati Amir, Xiao Chaowei, Prakash Atul, Kohno Tadayoshi, and Song Dawn. 2018. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 16251634.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Finlayson Samuel G., Bowers John D., Ito Joichi, Zittrain Jonathan L., Beam Andrew L., and Kohane Isaac S.. 2019. Adversarial attacks on medical machine learning. Science 363, 6433 (2019), 12871289.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Fredrikson Matthew, Lantz Eric, Jha Somesh, Lin Simon, Page David, and Ristenpart Thomas. 2014. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In Proceedings of the 23rd {USENIX} Security Symposium. 1732.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Fu Qiang, Zhu Jieming, Hu Wenlu, Lou Jian-Guang, Ding Rui, Lin Qingwei, Zhang Dongmei, and Xie Tao. 2014. Where do developers log? An empirical study on logging practices in industry. In Companion Proceedings of the 36th International Conference on Software Engineering.Association for Computing Machinery, 2433. DOI:DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Fu Qiang, Zhu Jieming, Hu Wenlu, Lou Jian-Guang, Ding Rui, Lin Qingwei, Zhang Dongmei, and Xie Tao. 2014. Where do developers log? An empirical study on logging practices in industry. In Companion Proceedings of the 36th International Conference on Software Engineering. 2433.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Garfinkel Simson, Matthews Jeanna, Shapiro Stuart S., and Smith Jonathan M.. 2017. Toward algorithmic transparency and accountability. Communications of the ACM 60, 9 (2017), 5. DOI:DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ghahramani Zoubin. 2003. Unsupervised learning. In Proceedings of the Summer School on Machine Learning. Springer, 72112.Google ScholarGoogle Scholar
  26. [26] Goodfellow Ian, Shlens Jonathon, and Szegedy Christian. 2015. Explaining and harnessing adversarial examples. In Proceedings of the International Conference on Learning Representations.Google ScholarGoogle Scholar
  27. [27] Hart Robert David. When artificial intelligence botches your medical diagnosis, who’s to blame? (n.d.). Retrieved from https://qz.com/989137/when-a-robot-ai-doctor-misdiagnoses-you-whos-to-blame/Google ScholarGoogle Scholar
  28. [28] He Pinjia, Chen Zhuangbin, He Shilin, and Lyu Michael R.. 2018. Characterizing the natural language descriptions in software logging statements. In Proceedings of the 2018 33rd IEEE/ACM International Conference on Automated Software Engineering. IEEE, 178189.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] He Yingzhe, Meng Guozhu, Chen Kai, Hu Xingbo, and He Jinwen. 2020. Towards security threats of deep learning systems: A survey. IEEE Transactions on Software Engineering (2020).Google ScholarGoogle Scholar
  30. [30] He Yingzhe, Meng Guozhu, Chen Kai, Hu Xingbo, and He Jinwen. 2020. Towards security threats of deep learning systems: A survey. IEEE Transactions on Software Engineering (2020).Google ScholarGoogle Scholar
  31. [31] Huang Ling, Joseph Anthony D., Nelson Blaine, Rubinstein Benjamin I. P., and Tygar J. Doug. 2011. Adversarial machine learning. In Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence. 4358.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Jordan Michael I. and Mitchell Tom M.. 2015. Machine learning: Trends, perspectives, and prospects. Science 349, 6245 (2015), 255260.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Kaelbling Leslie Pack, Littman Michael L., and Moore Andrew W.. 1996. Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996), 237285.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] King Jason, Pandita Rahul, and Williams Laurie. 2015. Enabling forensics by proposing heuristics to identify mandatory log events. In Proceedings of the 2015 Symposium and Bootcamp on the Science of Security.Association for Computing Machinery, 11 pages. DOI:DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] King Jason, Stallings Jon, Riaz Maria, and Williams Laurie. 2017. To log, or not to log: Using heuristics to identify mandatory log events–a controlled experiment. Empirical Software Engineering 22, 5 (2017), 26842717.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Krishna Kalpesh, Tomar Gaurav Singh, Parikh Ankur P., Papernot Nicolas, and Iyyer Mohit. 2020. Thieves on sesame street! model extraction of BERT-based APIs. In Proceedings of the International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=Byl5NREFDr.Google ScholarGoogle Scholar
  37. [37] Kurakin Alexey, Goodfellow Ian, Bengio Samy, et al. 2016. Adversarial examples in the physical world. (2016).Google ScholarGoogle Scholar
  38. [38] Kurita Keita, Michel Paul, and Neubig Graham. 2020. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 27932806. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Landis J. Richard and Koch Gary G.. 1977. The measurement of observer agreement for categorical data. Biometrics 33, 1 (1977), 159174. Retrieved from http://www.jstor.org/stable/2529310.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Li H., Shang W., Adams B., Sayagh M., and Hassan A. E.. 2020. A qualitative study of the benefits and costs of logging from developers’ perspectives. IEEE Transactions on Software Engineering (2020), 11. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Li Heng, Shang Weiyi, Adams Bram, Sayagh Mohammed, and Hassan Ahmed E.. 2020. A qualitative study of the benefits and costs of logging from developers’ perspectives. IEEE Transactions on Software Engineering (2020).Google ScholarGoogle Scholar
  42. [42] Li Heng, Shang Weiyi, and Hassan Ahmed E.. 2017. Which log level should developers choose for a new logging statement? Empirical Software Engineering 22, 4 (2017), 16841716.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Li Zhenhao, Chen Tse-Hsun, and Shang Weiyi. 2020. Where shall we log? Studying and suggesting logging locations in code blocks. In Proceedings of the 2020 35th IEEE/ACM International Conference on Automated Software Engineering. IEEE, 361372.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Lin W., Hu Y., and Tsai C.. 2012. Machine learning in financial crisis prediction: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 4 (2012), 421436. DOI:DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Liu Q., Li P., Zhao W., Cai W., Yu S., and Leung V. C. M.. 2018. A survey on security threats and defensive techniques of machine learning: A data driven view. IEEE Access 6 (2018), 1210312117. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Liu Z., Xia X., Lo D., Xing Z., Hassan A. E., and Li S.. 2019. Which variables should I log? IEEE Transactions on Software Engineering (2019), 11. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] McDaniel Patrick, Papernot Nicolas, and Celik Z. Berkay. 2016. Machine learning in adversarial settings. IEEE Security and Privacy 14, 3 (2016), 6872.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Mozaffari-Kermani Mehran, Sur-Kolay Susmita, Raghunathan Anand, and Jha Niraj K.. 2014. Systematic poisoning attacks on and defenses for machine learning in healthcare. IEEE Journal of Biomedical and Health Informatics 19, 6 (2014), 18931905.Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Mulligan Deirdre, Kohli Nitin, and Kroll Joshua. 2018. Tutorial Session: Common Pitfalls for Studying the Human Side of Machine Learning. Retrieved February 22, 2021 from https://www.facebook.com/nipsfoundation/videos/2003393576419036/.Google ScholarGoogle Scholar
  50. [50] NIST. 2019. A Taxonomy and Terminology of Adversarial Machine Learning. Retrieved February 12, 2021 from https://csrc.nist.gov/publications/detail/nistir/8269/draft.Google ScholarGoogle Scholar
  51. [51] Oliner Adam, Ganapathi Archana, and Xu Wei. 2012. Advances and challenges in log analysis. Communications of the ACM 55, 2 (2012), 5561.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Papernot Nicolas. 2018. A marauder’s map of security and privacy in machine learning: An overview of current and future research directions for making machine learning secure and private. In Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security. 11.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Papernot Nicolas, McDaniel Patrick, Sinha Arunesh, and Wellman Michael P.. 2018. Sok: Security and privacy in machine learning. In Proceedings of the 2018 IEEE European Symposium on Security and Privacy. IEEE, 399414.Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Qayyum A., Qadir J., Bilal M., and Al-Fuqaha A.. 2021. Secure and robust machine learning for healthcare: A survey. IEEE Reviews in Biomedical Engineering 14 (2021), 156180. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Qayyum A., Usama M., Qadir J., and Al-Fuqaha A.. 2020. Securing connected autonomous vehicles: Challenges posed by adversarial machine learning and the way forward. IEEE Communications Surveys Tutorials 22, 2 (2020), 9981026. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] R. Bhagasree2018. NeurIPS 2018: Rethinking transparency and accountability in machine learning. Retrieved February 23, 2021 from https://hub.packtpub.com/neurips-2018-rethinking-transparency-and-accountability-in-machine-learning/.Google ScholarGoogle Scholar
  57. [57] Rahman Akond, Rahman Md Rayhanur, Parnin Chris, and Williams Laurie. 2021. Security smells in ansible and chef scripts: A replication study. ACM Transactions on Software Engineering and Methodology 30, 1 (2021), 131.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. [58] Rivera-Ortiz Fanny and Pasquale Liliana. 2019. Towards automated logging for forensic-ready software systems. In Proceedings of the 2019 IEEE 27th International Requirements Engineering Conference Workshops. IEEE, 157163.Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Russell Stuart and Norvig Peter. 2005. AI a modern approach. Learning 2, 3 (2005), 4.Google ScholarGoogle Scholar
  60. [60] Salem Ahmed, Wen Rui, Backes Michael, Ma Shiqing, and Zhang Yang. 2020. Dynamic backdoor attacks against machine learning models. arXiv:2003.03675. Retrieved from https://arxiv.org/abs/2003.03675.Google ScholarGoogle Scholar
  61. [61] Schmidhuber Jürgen. 2015. Deep learning in neural networks: An overview. Neural Networks 61 (2015), 85117.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. [62] Shafique Muhammad, Naseer Mahum, Theocharides Theocharis, Kyrkou Christos, Mutlu Onur, Orosa Lois, and Choi Jungwook. 2020. Robust machine learning systems: Challenges, current trends, perspectives, and the road ahead. IEEE Design and Test 37, 2 (2020), 3057.Google ScholarGoogle ScholarCross RefCross Ref
  63. [63] Sharif Mahmood, Bhagavatula Sruti, Bauer Lujo, and Reiter Michael K.. 2016. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 15281540.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. [64] Kumar R. S. Siva, Nyström M., Lambert J., Marshall A., Goertzel M., Comissoneru A., Swann M., and Xia S.. 2020. Adversarial machine learning-industry perspectives. In Proceedings of the 2020 IEEE Security and Privacy Workshops. 6975. DOI:DOI:Google ScholarGoogle ScholarCross RefCross Ref
  65. [65] Song Congzheng, Ristenpart Thomas, and Shmatikov Vitaly. 2017. Machine learning models that remember too much. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 587601.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. [66] Steinhardt Jacob, Koh Pang Wei, and Liang Percy. 2017. Certified defenses for data poisoning attacks. In Proceedings of the NIPS’17 31st International Conference on Neural Information Processing Systems.Google ScholarGoogle Scholar
  67. [67] Sutton Richard S. and Barto Andrew G.. 2018. Reinforcement Learning: An Introduction. MIT Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. [68] Tabassi Elham, Burns Kevin, Hadjimichael Michael, Molina-Markham Andres, and Sexton Julian. 2019. A taxonomy and terminology of adversarial machine learning. NIST IR (2019).Google ScholarGoogle Scholar
  69. [69] Tan Pang-Ning, Steinbach Michael, and Kumar Vipin. 2005. Introduction to Data Mining (1st. ed.). Addison-Wesley Longman Publishing Co., Inc.Google ScholarGoogle Scholar
  70. [70] Tramèr Florian, Zhang Fan, Juels Ari, Reiter Michael K., and Ristenpart Thomas. 2016. Stealing machine learning models via prediction apis. In Proceedings of the 25th {USENIX} Security Symposium. 601618.Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. [71] Vorobeychik Yevgeniy and Kantarcioglu Murat. 2018. Adversarial machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 12, 3 (2018), 1169.Google ScholarGoogle ScholarCross RefCross Ref
  72. [72] Wang Binghui and Gong Neil Zhenqiang. 2018. Stealing hyperparameters in machine learning. In Proceedings of the 2018 IEEE Symposium on Security and Privacy. IEEE, 3652.Google ScholarGoogle ScholarCross RefCross Ref
  73. [73] Wang Shuo, Nepal Surya, Rudolph Carsten, Grobler Marthie, Chen Shangyu, and Chen Tianle. 2020. Backdoor attacks against transfer learning with pre-trained deep learning models. IEEE Transactions on Services Computing (2020).Google ScholarGoogle Scholar
  74. [74] Xiao Qixue, Li Kang, Zhang Deyue, and Xu Weilin. 2018. Security risks in deep learning implementations. In Proceedings of the 2018 IEEE Security and Privacy Workshops. IEEE, 123128.Google ScholarGoogle ScholarCross RefCross Ref
  75. [75] Yuan Ding, Park Soyeon, Huang Peng, Liu Yang, Lee Michael M., Tang Xiaoming, Zhou Yuanyuan, and Savage Stefan. 2012. Be conservative: Enhancing failure diagnosis with proactive logging. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation.USENIX Association, 293306.Google ScholarGoogle Scholar
  76. [76] Yuan Ding, Park Soyeon, Huang Peng, Liu Yang, Lee Michael M., Tang Xiaoming, Zhou Yuanyuan, and Savage Stefan. 2012. Be conservative: Enhancing failure diagnosis with proactive logging. In Proceedings of the 10th {USENIX} Symposium on Operating Systems Design and Implementation. 293306.Google ScholarGoogle Scholar
  77. [77] Yuan Ding, Park Soyeon, and Zhou Yuanyuan. 2012. Characterizing logging practices in open-source software. In Proceedings of the 2012 34th International Conference on Software Engineering. IEEE, 102112.Google ScholarGoogle ScholarCross RefCross Ref
  78. [78] Yuan Ding, Zheng Jing, Park Soyeon, Zhou Yuanyuan, and Savage Stefan. 2012. Improving software diagnosability via log enhancement. ACM Transactions on Computer Systems 30, 1 (2012), 128.Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. [79] Zeng Yi, Chen Jinfu, Shang Weiyi, and Chen Tse-Hsun Peter. 2019. Studying the characteristics of logging practices in mobile apps: A case study on f-droid. Empirical Software Engineering 24, 6 (2019), 33943434.Google ScholarGoogle ScholarCross RefCross Ref
  80. [80] Zhao Mengchen, An Bo, Gao Wei, and Zhang Teng. 2017. Efficient label contamination attacks against black-box learning models.. In Proceedings of the IJCAI. 39453951.Google ScholarGoogle ScholarCross RefCross Ref
  81. [81] Zhi Chen, Yin Jianwei, Deng Shuiguang, Ye Maoxin, Fu Min, and Xie Tao. 2019. An exploratory study of logging configuration practice in Java. In Proceedings of the 2019 IEEE International Conference on Software Maintenance and Evolution. IEEE, 459469.Google ScholarGoogle ScholarCross RefCross Ref
  82. [82] Zhu Jieming, He Pinjia, Fu Qiang, Zhang Hongyu, Lyu Michael R., and Zhang Dongmei. 2015. Learning to log: Helping developers make informed logging decisions. In Proceedings of the 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. IEEE, 415425.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Log-related Coding Patterns to Conduct Postmortems of Attacks in Supervised Learning-based Projects

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Privacy and Security
      ACM Transactions on Privacy and Security  Volume 26, Issue 2
      May 2023
      335 pages
      ISSN:2471-2566
      EISSN:2471-2574
      DOI:10.1145/3572849
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 12 April 2023
      • Online AM: 14 December 2022
      • Accepted: 12 September 2022
      • Revised: 1 April 2022
      • Received: 26 August 2021
      Published in tops Volume 26, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
    • Article Metrics

      • Downloads (Last 12 months)133
      • Downloads (Last 6 weeks)26

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!