skip to main content
research-article
Public Access

Enki: A Diversity-driven Approach to Test and Train Robust Learning-enabled Systems

Published:29 May 2021Publication History
Skip Abstract Section

Abstract

Data-driven Learning-enabled Systems are limited by the quality of available training data, particularly when trained offline. For systems that must operate in real-world environments, the space of possible conditions that can occur is vast and difficult to comprehensively predict at design time. Environmental uncertainty arises when run-time conditions diverge from design-time training conditions. To address this problem, automated methods can generate synthetic data to fill in gaps for training and test data coverage. We propose an evolution-based technique to assist developers with uncovering limitations in existing data when previously unseen environmental phenomena are introduced. This technique explores unique contexts for a given environmental condition, with an emphasis on diversity. Synthetic data generated by this technique may be used for two purposes: (1) to assess the robustness of a system to uncertain environmental factors and (2) to improve the system’s robustness. This technique is demonstrated to outperform random and greedy methods for multiple adverse environmental conditions applied to image-processing Deep Neural Networks.

References

  1. Alejandro Barredo Arrieta, Natalia Diaz-Rodriguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Info. Fusion 58 (June 2020), 82–115. DOI:https://doi.org/10.1016/j.inffus.2019.12.012Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, Piscataway, NJ, 3319–3327. DOI:https://doi.org/10.1109/CVPR.2017.354Google ScholarGoogle ScholarCross RefCross Ref
  3. Yoshua Bengio. 2020. Priors For Deep Learning of Semantic Representations. In Proceedings of the ACM/IEEE International Conference on Model-driven Engineering Languages and Systems (MODELS’20).Google ScholarGoogle Scholar
  4. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13 (Feb. 2012), 281–305.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Mohamed Boussaa, Olivier Barais, Gerson Sunyé, and Benoît Baudry. 2015. A novelty search approach for automatic test data generation. In Proceedings of the 8th International Workshop on Search-based Software Testing (SBST’15). IEEE, Piscataway, NJ, 40–43. DOI:https://doi.org/10.1109/SBST.2015.17Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Mohamed Boussaa, Olivier Barais, Gerson Sunyé, and Benoît Baudry. 2015. A novelty search-based test data generator for object-oriented programs. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO’15). ACM, New York, NY, 1359–1360. DOI:https://doi.org/10.1145/2739482.2764716Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Carla E. Brodley and Mark A. Friedl. 1999. Identifying mislabeled training data. J. Artif. Int. Res. 11, 1 (July 1999), 131–167.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Simon Burton, Lydia Gauerhof, and Christian Heinzemann. 2017. Making the case for safety of machine learning in highly automated driving. In Proceedings of the International Conference on Computer Safety, Reliability, and Security. Lecture Notes in Computer Science (LNCS), Vol. 10489. Springer, Cham.Google ScholarGoogle Scholar
  9. Gul Calikli and Ayse Bener. 2010. Empirical analyses of the factors affecting confirmation bias and the effects of confirmation bias on software developer/tester performance. In Proceedings of the 6th International Conference on Predictive Models in Software Engineering (PROMISE’10). ACM, New York, NY, Article 10, 11 pages. DOI:https://doi.org/10.1145/1868328.1868344Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Anthony Clark, Byron DeVries, Jared Moore, Betty H. C. Cheng, and Philip McKinley. 2016. An evolutionary approach to discovering execution mode boundaries for adaptive controllers. In Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI’16). IEEE, Piscataway, NJ, 1–8. DOI:https://doi.org/10.1109/SSCI.2016.7850178Google ScholarGoogle ScholarCross RefCross Ref
  11. Fabiano Dalpiaz, Alexander Borgida, Jennifer Horkoff, and John Mylopoulos. 2013. Runtime goal models: Keynote. In Proceedings of 7th IEEE International Conference on Research Challenges in Information Science (RCIS’13). IEEE, Piscataway, NJ, 1–11. DOI:https://doi.org/10.1109/RCIS.2013.6577674Google ScholarGoogle ScholarCross RefCross Ref
  12. Mirko D’Angelo, Simos Gerasimou, Sona Ghahremani, Johannes Grohmann, Ingrid Nunes, Evangelos Pournaras, and Sven Tomforde. 2019. On learning in collective self-adaptive systems: State of practice and a 3D framework. In Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS’19). IEEE, Piscataway, NJ, 13–24. DOI:https://doi.org/10.1109/SEAMS.2019.00012Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Junhua Ding, Xiaojun Kang, and Xin-Hua Hu. 2017. Validating a deep learning framework by metamorphic testing. In Proceedings of the 2nd International Workshop on Metamorphic Testing (MET’17). IEEE, Piscataway, NJ, 28–34. DOI:https://doi.org/10.1109/MET.2017.2Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Agoston E. Eiben and James E. Smith. 2015. Introduction to Evolutionary Computing (2nd ed.). Springer-Verlag, Berlin, Heidelberg, DEU.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. 2018. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, Piscataway, NJ, 1625–1634. DOI:https://doi.org/10.1109/CVPR.2018.00175Google ScholarGoogle ScholarCross RefCross Ref
  16. Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE’11). ACM, New York, NY, 416–419. DOI:https://doi.org/10.1145/2025113.2025179Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Hans Gaiser, Maarten de Vries, Valeriu Lacatusu, V. Carpani, Ashley Williamson, Enrico Liscio, Andreas, Yann Henon, jjiun, Cristian Gratie, Mihai Morariu, Charles Ye, Martin Zlocha, Ben Weinstein, Rodrigo Meira de Andrade, Pedro Conceição, Alexander Pacha, hannesedvartsen, Daniyal Shahrokhian, Wudi Fang, Mike Clark, meagerYak, Iver Jordal, Max Van Sande, Jin, Etienne-Meunier, Andrew Grigorev, Guillaume Erhard, Eduardo Ramos, and Denis Dowling. 2019. fizyr/keras-retinanet 0.5.1. Retrieved from https://github.com/fizyr/keras-retinanet/releases. DOI:https://doi.org/10.5281/zenodo.3250670Google ScholarGoogle Scholar
  18. Kshitiz Garg and Shree K. Nayar. 2004. Photometric Model of a Rain Drop. Technical Report. Columbia University. Retrieved from https://www1.cs.columbia.edu/CAVE/publications/pdfs/Garg_TR04.pdf.Google ScholarGoogle Scholar
  19. Xavier Gastaldi. 2017. Shake-Shake regularization. Retrieved from https://arXiv:1705.07485.Google ScholarGoogle Scholar
  20. Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, Piscataway, NJ, 3354–3361. DOI:https://doi.org/10.1109/CVPR.2012.6248074Google ScholarGoogle ScholarCross RefCross Ref
  21. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT, Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. Vol. 27. Curran Associates, Red Hook, NY, 2672–2680.Google ScholarGoogle Scholar
  23. Ian J. Goodfellow, Jonathons Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. Retrieved from https://arXiv:1412.6572.Google ScholarGoogle Scholar
  24. Ronan Hamon, Henrik Junklewitz, and Ignacio Sanchez. 2020. Robustness and Explainability of Artificial Intelligence. Technical Report EUR 30040 EN. European Commission/Joint Research Centre (JRC).Google ScholarGoogle Scholar
  25. Karen Hao. 2019. This is how AI bias really happens—And why it’s so hard to fix. MIT Technology Review Tech policy/AI Ethics (Feb. 2019), 1 page. Retrieved from https://www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/.Google ScholarGoogle Scholar
  26. Mark Harman, S. Afshin Mansouri, and Yuanyuan Zhang. 2012. Search-based software engineering: Trends, techniques, and applications. ACM Comput. Surv. 45, 1, Article 11 (Dec. 2012), 61 pages pages. DOI:https://doi.org/10.1145/2379776.2379787Google ScholarGoogle Scholar
  27. William Grant Hatcher and Wei Yu. 2018. A survey of deep learning: Platforms, applications and emerging research trends. IEEE Access 6 (2018), 24411–24432. DOI:https://doi.org/10.1109/ACCESS.2018.2830661Google ScholarGoogle ScholarCross RefCross Ref
  28. Haibo He and Edwardo A. Garcia. 2009. Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21, 9 (Sept. 2009), 1263–1284. DOI:https://doi.org/10.1109/TKDE.2008.239Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE, Piscataway, NJ, 770–778. DOI:https://doi.org/10.1109/CVPR.2016.90Google ScholarGoogle ScholarCross RefCross Ref
  30. John F. Hughes, Andries van Dam, Morgan McGuire, David F. Sklar, James D. Foley, Steven Feiner, and Kurt Akeley. 2013. Computer Graphics: Principles and Practice. Addison-Wesley, Upper Saddle River, NJ.Google ScholarGoogle Scholar
  31. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (ICML’15), Vol. 37. Microtome Publishing, Brookline, MA, 448–456.Google ScholarGoogle Scholar
  32. Nick Jacobi, Phil Husbands, and Inman Harvey. 1995. Noise and the reality gap: The use of simulation in evolutionary robotics. In Proceedings of the 3rd European Conference on Advances in Artificial Life. Springer-Verlag, Berlin, Heidelberg, DEU, 704–720.Google ScholarGoogle ScholarCross RefCross Ref
  33. Joel Janai, Fatma Güney, Aseem Behl, and Andreas Geiger. 2017. Computer vision for autonomous vehicles: Problems, datasets and state-of-the-art. Retrieved from https://arXiv:1704.05519.Google ScholarGoogle Scholar
  34. Jason Jo and Yoshua Bengio. 2017. Measuring the tendency of CNNs to learn surface statistical regularities. Retrieved from https://arXiv:1711.11561.Google ScholarGoogle Scholar
  35. Siddhartha Kumar Khaitan and James D. McCalley. 2015. Design techniques and applications of cyber physical systems: A survey. IEEE Syst. J. 9, 2 (2015), 350–365.Google ScholarGoogle ScholarCross RefCross Ref
  36. Saeed Reza Kheradpisheh, Masoud Ghodrati, Mohammad Ganjtabesh, and Timothée Masquelier. 2016. Deep networks can resemble human feed-forward vision in invariant object recognition. Sci. Rep. 6, 32672 (2016), 24 pages. DOI:https://doi.org/10.1038/srep32672Google ScholarGoogle Scholar
  37. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. Retrieved from https://arXiv:1412.6980.Google ScholarGoogle Scholar
  38. Wilhelm Kirch (Ed.). 2008. Pearson’s Correlation Coefficient. Springer Netherlands, Dordrecht, NLD, 1090–1091. DOI:https://doi.org/10.1007/978-1-4020-5614-7_2569Google ScholarGoogle Scholar
  39. Will Knight. 2017. The dark secret at the heart of AI. MIT Technology Review Artificial intelligence/Machine learning (Apr. 2017), 1. Retrieved from https://www.technologyreview.com/2017/04/11/5113/the-dark-secret-at-the-heart-of-ai/.Google ScholarGoogle Scholar
  40. Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto.Google ScholarGoogle Scholar
  41. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 6 (May 2017), 84–90. DOI:https://doi.org/10.1145/3065386Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Christian Krupitzer, Felix Maximilian Roth, Martin Pfannemüller, and Christian Becker. 2016. Comparison of approaches for self-improvement in self-adaptive systems. In Proceedings of the IEEE International Conference on Autonomic Computing (ICAC’16). IEEE, Piscataway, NJ, 308–314. DOI:https://doi.org/10.1109/ICAC.2016.18Google ScholarGoogle ScholarCross RefCross Ref
  43. Alexey Kurakin, Ian J. Goodfellow, and Sami Bengio. 2016. Adversarial machine learning at scale. Retrieved from https://arXiv:1611.01236.Google ScholarGoogle Scholar
  44. Michael A. Langford and Betty H. C. Cheng. 2019. Enhancing learning-enabled software systems to address environmental uncertainty. In Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC’19). IEEE, Piscataway, NJ, 115–124. DOI:https://doi.org/10.1109/ICAC.2019.00023Google ScholarGoogle Scholar
  45. Michael A. Langford, Glen A. Simon, Philip K. McKinley, and Betty H. C. Cheng. 2019. Applying evolution and novelty search to enhance the resilience of autonomous systems. In Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS’19). ACM, New York, NY, 63–69. DOI:https://doi.org/10.1109/SEAMS.2019.00017Google ScholarGoogle Scholar
  46. Joel Lehman. 2012. Evolution Through the Search for Novelty. Ph.D. Dissertation. University of Central Florida.Google ScholarGoogle Scholar
  47. Joel Lehman and Kenneth O. Stanley. 2011. Abandoning objectives: Evolution through the search for novelty alone. Evolution. Comput. 19, 2 (June 2011), 189–223. DOI:https://doi.org/10.1162/EVCO_a_00025Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Zenan Li, Xiaoxing Ma, Chang Xu, and Chun Cao. 2019. Structural coverage criteria for neural networks could be misleading. In Proceedings of the 41st International Conference on Software Engineering (ICSE-NIER’19). IEEE, Piscataway, NJ, 89–92. DOI:https://doi.org/10.1109/ICSE-NIER.2019.00031Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. 2017. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). IEEE, Piscataway, NJ, 2999–3007. DOI:https://doi.org/10.1109/ICCV.2017.324Google ScholarGoogle ScholarCross RefCross Ref
  50. Fredrik Lundh, Alex Clark, and Contributors. 2015. Pillow (PIL Fork) 3.0x. Read the Docs. Retrieved from https://pillow.readthedocs.io/en/3.0.x/about.html.Google ScholarGoogle Scholar
  51. Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, and Yang Liu. 2018. DeepGauge: Multi-granularity testing criteria for deep learning systems. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE’18). ACM, New York, NY, 120–131. DOI:https://doi.org/10.1145/3238147.3238202Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective automated testing for android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA’16). ACM, New York, NY, 94–105. DOI:https://doi.org/10.1145/2931037.2931054Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Phil McMinn. 2011. Search-based software testing: Past, present and future. In Proceedings of the 4th International Conference on Software Testing, Verification and Validation Workshops (ICST’11). IEEE, Piscataway, NJ, 153–163. DOI:https://doi.org/10.1109/ICSTW.2011.100Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML’10). Omnipress, Madison, WI, 807–814.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). IEEE, Piscataway, NJ, 427–436. DOI:https://doi.org/10.1109/CVPR.2015.7298640Google ScholarGoogle ScholarCross RefCross Ref
  56. Nvidia Corporation. 2017. prepare_kitti_data.py. Retrieved from https://github.com/NVIDIA/DIGITS/blob/master/examples/object-detection/prepare_kitti_data.py.Google ScholarGoogle Scholar
  57. Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. DeepXplore: Automated whitebox testing of deep learning systems. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP’17). ACM, New York, NY, 1–18. DOI:https://doi.org/10.1145/3132747.3132785Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Andres J. Ramirez, Adam C. Jensen, Betty H. C. Cheng, and David B. Knoester. 2011. Automatically exploring how uncertainty impacts behavior of dynamically adaptive systems. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE’11). IEEE, Piscataway, NJ, 568–571. DOI:https://doi.org/10.1109/ASE.2011.6100127Google ScholarGoogle Scholar
  59. Colin R. Reeves. 2000. Fitness landscapes and evolutionary algorithms. In Proceedings of the International Conference on Computer Safety, Reliability, and Security (Artificial Evolution). Springer, Berlin, 3–20.Google ScholarGoogle ScholarCross RefCross Ref
  60. Charles Richter and Nicholas Roy. 2017. Safe visual navigation via deep learning and novelty detection. In Proceedings of Robotics: Science and Systems. MIT, Cambridge, MA, 64–73. DOI:https://doi.org/10.15607/RSS.2017.XIII.064Google ScholarGoogle ScholarCross RefCross Ref
  61. Stuart J. Russell and Peter Norvig. 2010. Artificial Intelligence: A Modern Approach (3 ed.). Pearson Education, Upper Saddle River, NJ.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Ahmad Sallab, Mohammed Abdou, Etienne Perot, and Senthil Yogamani. 2017. Deep reinforcement learning framework for autonomous driving. In Proceedings of the IS&T International Symposium on Electronic Imaging, Autonomous Vehicles and Machines (AVM’17). IS&T, Springfield, VA, 70–76. DOI:https://doi.org/10.2352/ISSN.2470-1173.2017.19.AVM-023Google ScholarGoogle ScholarCross RefCross Ref
  63. Yutaka Sasaki. 2007. The Truth of the F-measure. Technical Report. University of Manchester. Retrieved from https://www.cs.odu.edu/ mukka/cs795sum09dm/Lecturenotes/Day3/F-measure-YS-26Oct07.pdf.Google ScholarGoogle Scholar
  64. Johann Schumann, Pramod Gupta, and Yan Liu. 2010. Application of Neural Networks in High Assurance Systems: A Survey. Studies in Computational Intelligence (SCI), Vol. 268. Springer, Berlin.Google ScholarGoogle Scholar
  65. Jasmine Sekhon and Cody Fleming. 2019. Towards improved testing for deep learning. In Proceedings of the 41st International Conference on Software Engineering (ICSE-NIER’19). IEEE, Piscataway, NJ, 85–88. DOI:https://doi.org/10.1109/ICSE-NIER.2019.00030Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Valerie Sessions and Marco Valtorta. 2006. The effects of data quality on machine learning algorithms. In Proceedings of the the International Conference on Information Quality (ICIQ’06). MIT, Cambridge, MA, 485–498.Google ScholarGoogle Scholar
  67. Burr Settles. 2009. Active Learning Literature Survey. Computer Sciences Technical Report 1648. University of Wisconsin–Madison.Google ScholarGoogle Scholar
  68. James V. Stone. 2015. Information Theory: A Tutorial Introduction (1st ed.). Sebtel Press.Google ScholarGoogle Scholar
  69. Oliver Strömgren. 2018. Deep Learning for Autonomous Collision Avoidance. MSc Thesis. Linköping University.Google ScholarGoogle Scholar
  70. Youcheng Sun, Xiaowei Huang, Daniel Kroening, James Sharp, Matthew Hill, and Rob Ashmore. 2019. DeepConcolic: Testing and debugging deep neural networks. In Proceedings of the 41st International Conference on Software Engineering (ICSE’19). IEEE, Piscataway, NJ, 111–114. DOI:https://doi.org/10.1109/ICSE-Companion.2019.00051Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, and Daniel Kroening. 2018. Concolic testing for deep neural networks. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE’18). ACM, New York, NY, 109–119. DOI:https://doi.org/10.1145/3238147.3238172Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. Retrieved from https://arXiv:1312.6199.Google ScholarGoogle Scholar
  73. Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. 2018. DeepTest: Automated testing of deep-neural-network-driven autonomous cars. In Proceedings of the 40th International Conference on Software Engineering (ICSE’18). ACM, New York, NY, 303–314. DOI:https://doi.org/10.1145/3180155.3180220Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Cumhur Erkan Tuncali, James Kapinski, Hisahiro Ito, and Jyotirmoy V. Deshmukh. 2018. Reasoning about safety of learning-enabled components in autonomous cyber-physical systems. In Proceedings of the 55th Annual Design Automation Conference (DAC’18). ACM, New York, NY, Article 30, 6 pages. DOI:https://doi.org/10.1145/3195970.3199852Google ScholarGoogle Scholar
  75. U.S. Department of Defense. 2016. Report of the Defense Science Board Summer Study on Autonomy. Technical Report. DoD.Google ScholarGoogle Scholar
  76. U.S. National Transportation Safety Board. 2017. Accident Report, NTSB/HAR-17/02. Technical Report PB2017-102600. NTSB.Google ScholarGoogle Scholar
  77. U.S. National Transportation Safety Board. 2018. Preliminary Report, Highway. Technical Report HWY18FH011. NTSB.Google ScholarGoogle Scholar
  78. U.S. National Transportation Safety Board. 2019. Vehicle Automation Report, Highway. Technical Report HWY18MH010. NTSB.Google ScholarGoogle Scholar
  79. Ronald E. Walpole, Raymond H. Myers, Sharon L. Myers, and Keying Ye. 2012. Probability & Statistics For Engineers & Scientists (9th ed.). Prentice Hall, Boston, MA.Google ScholarGoogle Scholar
  80. Zongben Xu and Jian Sun. 2017. Model-driven deep-learning. Natl. Sci. Rev. 5, 1 (08 2017), 22–24. DOI:https://doi.org/10.1093/nsr/nwx099Google ScholarGoogle Scholar
  81. Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. Understanding neural networks through deep visualization. Retrieved from https://arXiv:1506.06579.Google ScholarGoogle Scholar
  82. Fuxun Yu, Zhuwei Qin, Chenchen Liu, Liang Zhao, Yanzhi Wang, and Xiang Chen. 2019. Interpreting and evaluating neural network robustness. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). ijcai.org, 4199–4205. DOI:https://doi.org/10.24963/ijcai.2019/583Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. Retrieved from https://arXiv:1605.07146.Google ScholarGoogle Scholar
  84. Mengshi Zhang, Yuqun Zhang, Lingming Zhang, Cong Liu, and Sarfraz Khurshid. 2018. DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE’18). ACM, New York, NY, 132–142. DOI:https://doi.org/10.1145/3238147.3238187Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). IEEE, Piscataway, NJ, 2242–2251.Google ScholarGoogle Scholar

Index Terms

  1. Enki: A Diversity-driven Approach to Test and Train Robust Learning-enabled Systems

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Published in

              cover image ACM Transactions on Autonomous and Adaptive Systems
              ACM Transactions on Autonomous and Adaptive Systems  Volume 15, Issue 2
              June 2020
              91 pages
              ISSN:1556-4665
              EISSN:1556-4703
              DOI:10.1145/3461693
              Issue’s Table of Contents

              Copyright © 2021 ACM

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 29 May 2021
              • Accepted: 1 April 2021
              • Revised: 1 February 2021
              • Received: 1 February 2020
              Published in taas Volume 15, Issue 2

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • research-article
              • Research
              • Refereed

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format .

            View HTML Format
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!