skip to main content
research-article
Open Access

Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs

Published:15 October 2021Publication History
Skip Abstract Section

Abstract

Android apps are GUI-based event-driven software and have become ubiquitous in recent years. Obviously, functional correctness is critical for an app’s success. However, in addition to crash bugs, non-crashing functional bugs (in short as “non-crashing bugs” in this work) like inadvertent function failures, silent user data lost and incorrect display information are prevalent, even in popular, well-tested apps. These non-crashing functional bugs are usually caused by program logic errors and manifest themselves on the graphic user interfaces (GUIs). In practice, such bugs pose significant challenges in effectively detecting them because (1) current practices heavily rely on expensive, small-scale manual validation (the lack of automation); and (2) modern fully automated testing has been limited to crash bugs (the lack of test oracles).

This paper fills this gap by introducing independent view fuzzing, a novel, fully automated approach for detecting non-crashing functional bugs in Android apps. Inspired by metamorphic testing, our key insight is to leverage the commonly-held independent view property of Android apps to manufacture property-preserving mutant tests from a set of seed tests that validate certain app properties. The mutated tests help exercise the tested apps under additional, adverse conditions. Any property violations indicate likely functional bugs for further manual confirmation. We have realized our approach as an automated, end-to-end functional fuzzing tool, Genie. Given an app, (1) Genie automatically detects non-crashing bugs without requiring human-provided tests and oracles (thus fully automated); and (2) the detected non-crashing bugs are diverse (thus general and not limited to specific functional properties), which set Genie apart from prior work.

We have evaluated Genie on 12 real-world Android apps and successfully uncovered 34 previously unknown non-crashing bugs in their latest releases — all have been confirmed, and 22 have already been fixed. Most of the detected bugs are nontrivial and have escaped developer (and user) testing for at least one year and affected many app releases, thus clearly demonstrating Genie’s effectiveness. According to our analysis, Genie achieves a reasonable true positive rate of 40.9%, while these 34 non-crashing bugs could not be detected by prior fully automated GUI testing tools (as our evaluation confirms). Thus, our work complements and enhances existing manual testing and fully automated testing for crash bugs.

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

the video talk and the subtitles

References

  1. Accessibility. 2020. Accessibility overview. https://developer.android.com/guide/topics/ui/accessibilityGoogle ScholarGoogle Scholar
  2. ActivityDiary. 2020. Activity Diary. https://play.google.com/store/apps/details?id=de.rampro.activitydiaryGoogle ScholarGoogle Scholar
  3. ActivityDiary. 2020. Activity Diary’s issue #118. https://github.com/ramack/ActivityDiary/issues/118Google ScholarGoogle Scholar
  4. Christoffer Quist Adamsen, Gianluca Mezzetti, and Anders Møller. 2015. Systematic execution of Android test suites in adverse conditions. In Proceedings of the 2015 International Symposium on Software Testing and Analysis (ISSTA). 83–93. https://doi.org/10.1145/2771783.2771786 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. ADB. 2020. Android adb. https://developer.android.com/studio/command-line/adbGoogle ScholarGoogle Scholar
  6. Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Bryan Dzung Ta, and Atif M. Memon. 2015. MobiGUITAR: Automated Model-Based Testing of Mobile Apps. IEEE Software, 32, 5 (2015), 53–59. https://doi.org/10.1109/MS.2014.55 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Domenico Amalfitano, Vincenzo Riccio, Ana C. R. Paiva, and Anna Rita Fasolino. 2018. Why does the orientation change mess up my Android application? From GUI failures to code faults. Softw. Test., Verif. Reliab., 28, 1 (2018), https://doi.org/10.1002/stvr.1654 Google ScholarGoogle ScholarCross RefCross Ref
  8. AmazeFileManager Team. 2020. AmazeFileManager. https://github.com/TeamAmaze/AmazeFileManagerGoogle ScholarGoogle Scholar
  9. Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In 20th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE). 59. https://doi.org/10.1145/2393596.2393666 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. AnkiDroid Team. 2020. AnkiDroid. https://github.com/ankidroid/Anki-AndroidGoogle ScholarGoogle Scholar
  11. AntennaPod Team. 2020. AntennaPod. https://github.com/AntennaPod/AntennaPodGoogle ScholarGoogle Scholar
  12. Appium. 2020. Appium: Mobile App Automation Made Awesome.. http://appium.io/Google ScholarGoogle Scholar
  13. Tanzirul Azim and Iulian Neamtiu. 2013. Targeted and depth-first exploration for systematic testing of android apps. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications (OOPSLA). 641–660. https://doi.org/10.1145/2509136.2509549 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Young Min Baek and Doo-Hwan Bae. 2016. Automated model-based Android GUI testing using multi-level GUI comparison criteria. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE). 238–249. https://doi.org/10.1145/2970276.2970313 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2015. The Oracle Problem in Software Testing: A Survey. IEEE Trans. Software Eng., 41, 5 (2015), 507–525. https://doi.org/10.1109/TSE.2014.2372785 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Farnaz Behrang and Alessandro Orso. 2019. Test Migration Between Mobile Apps with Similar Functionality. In 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 54–65. https://doi.org/10.1109/ASE.2019.00016 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Philip Bille. 2005. A survey on tree edit distance and related problems. Theor. Comput. Sci., 337, 1-3 (2005), 217–239. https://doi.org/10.1016/j.tcs.2004.12.030 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Tsong Y. Chen, Shing C. Cheung, and Shiu Ming Yiu. 1998. Metamorphic testing: a new approach for generating next test cases. HKUST-CS98-01, Hong Kong University of Science and Technology.Google ScholarGoogle Scholar
  19. Tsong Yueh Chen, Fei-Ching Kuo, Huai Liu, Pak-Lok Poon, Dave Towey, T. H. Tse, and Zhi Quan Zhou. 2018. Metamorphic Testing: A Review of Challenges and Opportunities. ACM Comput. Surv., 51, 1 (2018), 4:1–4:27. https://doi.org/10.1145/3143561 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Wontae Choi, George C. Necula, and Koushik Sen. 2013. Guided GUI testing of Android apps with minimal restart and approximate learning. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications (OOPSLA). 623–640. https://doi.org/10.1145/2509136.2509552 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. 2015. Automated Test Input Generation for Android: Are We There Yet? In IEEE/ACM International Conference on Automated Software Engineering (ASE). 429–440. https://doi.org/10.1109/ASE.2015.89 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Compuware. 2013. Users Have Low Tolerance For Buggy Apps - Only 16% Will Try A Failing App More Than Twice. https://techcrunch.com/2013/03/12/users-have-low-tolerance-for-buggy-apps-only-16-will-try-a-failing-app-more-than-twice/Google ScholarGoogle Scholar
  23. Zhen Dong, Marcel Böhme, Lucia Cojocaru, and Abhik Roychoudhury. 2020. Time-travel Testing of Android Apps. In Proceedings of the 42nd International Conference on Software Engineering (ICSE). 481–492. https://doi.org/10.1145/3377811.3380402 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Dawson Engler, David Yu Chen, Seth Hallem, Andy Chou, and Benjamin Chelf. 2001. Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code. In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (SOSP). 57–72.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Espresso. 2020. Espresso. https://developer.android.com/training/testing/espresso/Google ScholarGoogle Scholar
  26. F-Droid. 2020. F-Droid. https://f-droid.org/en/Google ScholarGoogle Scholar
  27. Lingling Fan, Ting Su, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, and Geguang Pu. 2018. Efficiently manifesting asynchronous programming errors in Android apps. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE). 486–497. https://doi.org/10.1145/3238147.3238170 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Lingling Fan, Ting Su, Sen Chen, Guozhu Meng, Yang Liu, Lihua Xu, Geguang Pu, and Zhendong Su. 2018. Large-scale analysis of framework-specific exceptions in Android apps. In Proceedings of the 40th International Conference on Software Engineering (ICSE). 408–419. https://doi.org/10.1145/3180155.3180222 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Amin Milani Fard, Mehdi MirzaAghaei, and Ali Mesbah. 2014. Leveraging existing tests in automated test generation for web applications. In International Conference on Automated Software Engineering (ASE). 67–78. https://doi.org/10.1145/2642937.2642991 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Mattia Fazzini and Alessandro Orso. 2017. Automated cross-platform inconsistency detection for mobile apps. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 308–318. https://doi.org/10.1109/ASE.2017.8115644 Google ScholarGoogle ScholarCross RefCross Ref
  31. Mattia Fazzini, Martin Prammer, Marcelo d’Amorim, and Alessandro Orso. 2018. Automatically translating bug reports into test cases for mobile apps. In 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 141–152. https://doi.org/10.1145/3213846.3213869 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. FindBugs. 2020. FindBugs. http://findbugs.sourceforge.net/Google ScholarGoogle Scholar
  33. Google Play. 2020. Google Play Store. https://play.google.com/store/appsGoogle ScholarGoogle Scholar
  34. Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Chun Cao, Chang Xu, Yuan Yao, Qirun Zhang, Jian Lu, and Zhendong Su. 2019. Practical GUI testing of Android applications via model abstraction and refinement. In Proceedings of the 41st International Conference on Software Engineering (ICSE). 269–280. https://doi.org/10.1109/ICSE.2019.00042 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Gang Hu, Xinhao Yuan, Yang Tang, and Junfeng Yang. 2014. Efficiently, effectively detecting mobile app bugs with AppDoctor. In Ninth Eurosys Conference (EuroSys). 18:1–18:15. https://doi.org/10.1145/2592798.2592813 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Gang Hu, Linjie Zhu, and Junfeng Yang. 2018. AppFlow: using machine learning to synthesize robust, reusable UI tests. In Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 269–282. https://doi.org/10.1145/3236024.3236055 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Infer. 2020. Infer. https://fbinfer.com/Google ScholarGoogle Scholar
  38. Reyhaneh Jabbarvand, Forough Mehralian, and Sam Malek. 2020. Automated construction of energy test oracles for Android. In 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 927–938. https://doi.org/10.1145/3368089.3409677 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Jacoco Team. 2020. Jacoco. https://www.eclemma.org/jacoco/Google ScholarGoogle Scholar
  40. Casper Svenning Jensen, Mukul R. Prasad, and Anders Møller. 2013. Automated testing with targeted event sequence generation. In International Symposium on Software Testing and Analysis (ISSTA). 67–77. https://doi.org/10.1145/2483760.2483777 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Wang Jue, Jiang Yanyan, Xu Chang, Cao Chun, Ma Xiaoxing, and Lu Jian. 2020. ComboDroid: Generating High-Quality Test Inputs for Android Apps via Use Case Combinations. In Proceedings of the 42nd International Conference on Software Engineering (ICSE). 469–480. https://doi.org/10.1145/3377811.3380382 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. K-9 Mail Team. 2020. K-9 Mail. https://github.com/k9mail/k-9/Google ScholarGoogle Scholar
  43. Pavneet Singh Kochhar, Ferdian Thung, Nachiappan Nagappan, Thomas Zimmermann, and David Lo. 2015. Understanding the test automation culture of app developers. In 8th International Conference on Software Testing, Verification and Validation (ICST). 1–10. https://doi.org/10.1109/ICST.2015.7102609 Google ScholarGoogle ScholarCross RefCross Ref
  44. Pingfan Kong, Li Li, Jun Gao, Kui Liu, Tegawendé F. Bissyandé, and Jacques Klein. 2019. Automated Testing of Android Apps: A Systematic Literature Review. IEEE Trans. Reliability, 68, 1 (2019), 45–66. https://doi.org/10.1109/TR.2018.2865733 Google ScholarGoogle ScholarCross RefCross Ref
  45. Yavuz Köroglu and Alper Sen. 2018. TCM: Test Case Mutation to Improve Crash Detection in Android. In 21st International Conference on Fundamental Approaches to Software Engineering (FASE). 264–280. https://doi.org/10.1007/978-3-319-89363-1_15 Google ScholarGoogle ScholarCross RefCross Ref
  46. Yavuz Köroglu and Alper Sen. 2021. Functional test generation from UI test scenarios using reinforcement learning for android applications. Softw. Test. Verification Reliab., 31, 3 (2021), https://doi.org/10.1002/stvr.1752 Google ScholarGoogle Scholar
  47. Emily Kowalczyk, Myra B. Cohen, and Atif M. Memon. 2018. Configurations in Android testing: they matter. In 1st International Workshop on Advances in Mobile App Analysis (A-Mobile). 1–6. https://doi.org/10.1145/3243218.3243219 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Edmund S. L. Lam, Peilun Zhang, and Bor-Yuh Evan Chang. 2017. ChimpCheck: property-based randomized test generation for interactive apps. In Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!). 58–77. https://doi.org/10.1145/3133850.3133853 Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). 216–226. https://doi.org/10.1145/2594291.2594334 Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. DroidBot: a lightweight UI-guided test input generator for Android. In Proceedings of the 39th International Conference on Software Engineering (ICSE). 23–26. https://doi.org/10.1109/ICSE-C.2017.8 Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Jun-Wei Lin, Reyhaneh Jabbarvand, and Sam Malek. 2019. Test Transfer Across Mobile Apps Through Semantic Mapping. In 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 42–53. https://doi.org/10.1109/ASE.2019.00015 Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Ying-Dar Lin, José F. Rojas, Edward T.-H. Chu, and Yuan-Cheng Lai. 2014. On the Accuracy, Efficiency, and Reusability of Automated Test Oracles for Android Devices. IEEE Trans. Software Eng., 40, 10 (2014), 957–970. https://doi.org/10.1109/TSE.2014.2331982 Google ScholarGoogle Scholar
  53. Lint. 2020. Improve your code with lint checks. https://developer.android.com/studio/write/lint.htmlGoogle ScholarGoogle Scholar
  54. Localytics. 2019. 25% of Users Abandon Apps After One Use. http://info.localytics.com/blog/25-of-users-abandon-apps-after-one-useGoogle ScholarGoogle Scholar
  55. Yifei Lu, Minxue Pan, Juan Zhai, Tian Zhang, and Xuandong Li. 2019. Preference-wise testing for Android applications. In Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 268–278. https://doi.org/10.1145/3338906.3338980 Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: an input generation system for Android apps. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). 224–234. https://doi.org/10.1145/2491411.2491450 Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Riyadh Mahmood, Nariman Mirzaei, and Sam Malek. 2014. EvoDroid: segmented evolutionary testing of Android apps. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE). 599–609. https://doi.org/10.1145/2635868.2635896 Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Rupak Majumdar and Koushik Sen. 2007. Hybrid Concolic Testing. In 29th International Conference on Software Engineering (ICSE). 416–426. https://doi.org/10.1109/ICSE.2007.41 Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Maryam Imtiaz Malik, Muddassar Azam Sindhu, Akmal Saeed Khattak, Rabeeh Ayaz Abbasi, and Khalid Saleem. 2021. Automating test oracles from restricted natural language agile requirements. Expert Syst. J. Knowl. Eng., 38, 1 (2021), https://doi.org/10.1111/exsy.12608 Google ScholarGoogle ScholarCross RefCross Ref
  60. Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective Automated Testing for Android Applications. In International Symposium on Software Testing and Analysis (ISSTA). 94–105. https://doi.org/10.1145/2931037.2931054 Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Leonardo Mariani, Mauro Pezzè, and Daniele Zuddas. 2018. Augusto: exploiting popular functionalities for the generation of semantic GUI tests with Oracles. In Proceedings of the 40th International Conference on Software Engineering (ICSE). 280–290. https://doi.org/10.1145/3180155.3180162 Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Material Design. 2020. Material Design. https://material.io/designGoogle ScholarGoogle Scholar
  63. Atif M. Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. What Test Oracle Should I Use for Effective GUI Testing? In 18th IEEE International Conference on Automated Software Engineering (ASE 2003). 164–173. https://doi.org/10.1109/ASE.2003.1240304 Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Atif M. Memon, Martha E. Pollack, and Mary Lou Soffa. 2000. Automated test oracles for GUIs. In ACM SIGSOFT Symposium on Foundations of Software Engineering (FSE). 30–39. https://doi.org/10.1145/355045.355050 Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Nariman Mirzaei, Joshua Garcia, Hamid Bagheri, Alireza Sadeghi, and Sam Malek. 2016. Reducing Combinatorics in GUI Testing of Android Applications. In Proceedings of the 38th International Conference on Software Engineering (ICSE). 559–570. https://doi.org/10.1145/2884781.2884853 Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Nariman Mirzaei, Sam Malek, Corina S. Pasareanu, Naeem Esfahani, and Riyadh Mahmood. 2012. Testing Android apps through symbolic execution. ACM SIGSOFT Software Engineering Notes, 37, 6 (2012), 1–5.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Monkey. 2020. Monkey. http://developer.android.com/tools/help/monkey.htmlGoogle ScholarGoogle Scholar
  68. Motherboard. 2020. Here’s the Shadow Inc. App That Failed in Iowa Last Night. https://www.vice.com/en_us/article/y3m33x/heres-the-shadow-inc-app-that-failed-in-iowa-last-nightGoogle ScholarGoogle Scholar
  69. NewPipe Team. 2020. NewPipe. https://github.com/TeamNewPipe/NewPipeGoogle ScholarGoogle Scholar
  70. Mateusz Pawlik and Nikolaus Augsten. 2016. Tree edit distance: Robust and memory-efficient. Inf. Syst., 56 (2016), 157–173. https://doi.org/10.1016/j.is.2015.08.004 Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Joao Felipe Pimentel. 2020. Python APTED algorithm for the Edit Distance. https://github.com/JoaoFelipe/aptedGoogle ScholarGoogle Scholar
  72. Robotium. 2020. Robotium: User scenario testing for Android. http://www.robotium.orgGoogle ScholarGoogle Scholar
  73. Ariel Rosenfeld, Odaya Kardashov, and Orel Zang. 2018. Automation of Android applications functional testing using machine learning activities classification. In Proceedings of the 5th International Conference on Mobile Software Engineering and Systems (MOBILESoft). 122–132. https://doi.org/10.1145/3197231.3197241 Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Konstantin Rubinov and Luciano Baresi. 2018. What Are We Missing When Testing Our Android Apps? Computer, 51, 4 (2018), 60–68. https://doi.org/10.1109/MC.2018.2141024 Google ScholarGoogle ScholarCross RefCross Ref
  75. Sergio Segura, Gordon Fraser, Ana B. Sánchez, and Antonio Ruiz Cortés. 2016. A Survey on Metamorphic Testing. IEEE Trans. Software Eng., 42, 9 (2016), 805–824. https://doi.org/10.1109/TSE.2016.2532875 Google ScholarGoogle ScholarCross RefCross Ref
  76. Zhiyong Shan, Tanzirul Azim, and Iulian Neamtiu. 2016. Finding Resume and Restart Errors in Android Applications. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). 864–880. isbn:978-1-4503-4444-9 https://doi.org/10.1145/2983990.2984011 Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Sixth Tone. 2019. E-Commerce App Loses ‘Tens of Millions’ From Coupon Glitches. https://www.sixthtone.com/news/1003483/e-commerce-app-loses-tens-of-millions-from-coupon-glitchesGoogle ScholarGoogle Scholar
  78. Wei Song, Xiangxing Qian, and Jeff Huang. 2017. EHBDroid: Beyond GUI Testing for Android Applications. In Proceedings of the 32Nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 27–37. isbn:978-1-5386-2684-9 https://doi.org/10.1109/ASE.2017.8115615 Google ScholarGoogle ScholarCross RefCross Ref
  79. Ting Su, Lingling Fan, Sen Chen, Yang Liu, Lihua Xu, Geguang Pu, and Zhendong Su. 2020. Why My App Crashes Understanding and Benchmarking Framework-specific Exceptions of Android apps. IEEE Transactions on Software Engineering, https://doi.org/10.1109/TSE.2020.3013438 Google ScholarGoogle ScholarCross RefCross Ref
  80. Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, Stochastic Model-based GUI Testing of Android Apps. In The joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). 245–256. https://doi.org/10.1145/3106237.3106298 Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Ting Su, Jue Wang, and Zhendong Su. 2021. Benchmarking automated GUI testing for Android against real-world bugs. In 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 119–130. https://doi.org/10.1145/3468264.3468620 Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Chengnian Sun, Vu Le, and Zhendong Su. 2016. Finding compiler bugs via live code mutation. In ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). 849–863. https://doi.org/10.1145/2983990.2984038 Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Jingling Sun, Ting Su, Junxin Li, Zhen Dong, Geguang Pu, Tao Xie, and Zhendong Su. 2021. Understanding and finding system setting-related defects in Android apps. In 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 204–215. https://doi.org/10.1145/3460319.3464806 Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. 2018. DeepTest: automated testing of deep-neural-network-driven autonomous cars. In Proceedings of the 40th International Conference on Software Engineering (ICSE). 303–314.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Sandro Tolksdorf, Daniel Lehmann, and Michael Pradel. 2019. Interactive metamorphic testing of debuggers. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 273–283. https://doi.org/10.1145/3293882.3330567 Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. Porfirio Tramontana, Domenico Amalfitano, Nicola Amatucci, and Anna Rita Fasolino. 2019. Automated functional testing of mobile applications: a systematic mapping study. Software Quality Journal, 27, 1 (2019), 149–201. https://doi.org/10.1007/s11219-018-9418-6 Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. Heila van der Merwe, Brink van der Merwe, and Willem Visser. 2012. Verifying Android Applications Using Java PathFinder. SIGSOFT Softw. Eng. Notes, 37, 6 (2012), Nov., 1–5. issn:0163-5948 https://doi.org/10.1145/2382756.2382797 Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. Mario Linares Vásquez, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2017. How do Developers Test Android Applications? In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). 613–622. https://doi.org/10.1109/ICSME.2017.47 Google ScholarGoogle ScholarCross RefCross Ref
  89. Mario Linares Vásquez, Kevin Moran, and Denys Poshyvanyk. 2017. Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mobile App Testing. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). 399–410. https://doi.org/10.1109/ICSME.2017.27 Google ScholarGoogle Scholar
  90. Wenyu Wang, Dengfeng Li, Wei Yang, Yurui Cao, Zhenwen Zhang, Yuetang Deng, and Tao Xie. 2018. An empirical study of Android test generation tools in industrial cases. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE). 738–748. https://doi.org/10.1145/3238147.3240465 Google ScholarGoogle ScholarDigital LibraryDigital Library
  91. Yu Wang, Ke Wang, Fengjuan Gao, and Linzhang Wang. 2020. Learning semantic program embeddings with graph interval neural network. Proc. ACM Program. Lang., 4, OOPSLA (2020), 137:1–137:27. https://doi.org/10.1145/3428205 Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Qing Xie and Atif M. Memon. 2007. Designing and comparing automated test oracles for GUI-based software applications. ACM Trans. Softw. Eng. Methodol., 16, 1 (2007), 4. https://doi.org/10.1145/1189748.1189752 Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Xiaoyuan Xie, Joshua W. K. Ho, Christian Murphy, Gail E. Kaiser, Baowen Xu, and Tsong Yueh Chen. 2011. Testing and validating machine learning classifiers by metamorphic testing. Journal of Systems and Software, 84, 4 (2011), 544–558. https://doi.org/10.1016/j.jss.2010.11.920 Google ScholarGoogle ScholarDigital LibraryDigital Library
  94. Wei Yang, Mukul R. Prasad, and Tao Xie. 2013. A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications. In 16th International Conference on Fundamental Approaches to Software Engineering (FASE). 250–265. https://doi.org/10.1007/978-3-642-37057-1_19 Google ScholarGoogle ScholarDigital LibraryDigital Library
  95. Xun Yuan and Atif M. Memon. 2007. Using GUI Run-Time State as Feedback to Generate Test Cases. In 29th International Conference on Software Engineering (ICSE). 396–405. https://doi.org/10.1109/ICSE.2007.94 Google ScholarGoogle ScholarDigital LibraryDigital Library
  96. Razieh Nokhbeh Zaeem, Mukul R. Prasad, and Sarfraz Khurshid. 2014. Automated Generation of Oracles for Testing User-Interaction Features of Mobile Apps. In International Conference on Software Testing, Verification and Validation (ICST). 183–192. https://doi.org/10.1109/ICST.2014.31 Google ScholarGoogle ScholarDigital LibraryDigital Library
  97. Kaizhong Zhang and Dennis E. Shasha. 1989. Simple Fast Algorithms for the Editing Distance Between Trees and Related Problems. SIAM J. Comput., 18, 6 (1989), 1245–1262. https://doi.org/10.1137/0218082 Google ScholarGoogle ScholarDigital LibraryDigital Library
  98. Haibing Zheng, Dengfeng Li, Beihai Liang, Xia Zeng, Wujie Zheng, Yuetang Deng, Wing Lam, Wei Yang, and Tao Xie. 2017. Automated Test Input Generation for Android: Towards Getting There in an Industrial Case. In 39th IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP). 253–262. https://doi.org/10.1145/3238147.3240465 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!