skip to main content
research-article

Targeted and depth-first exploration for systematic testing of android apps

Published:29 October 2013Publication History
Skip Abstract Section

Abstract

Systematic exploration of Android apps is an enabler for a variety of app analysis and testing tasks. Performing the exploration while apps run on actual phones is essential for exploring the full range of app capabilities. However, exploring real-world apps on real phones is challenging due to non-determinism, non-standard control flow, scalability and overhead constraints. Relying on end-users to conduct the exploration might not be very effective: we performed a 7-use study on popular Android apps, and found that the combined 7-use coverage was 30.08% of the app screens and 6.46% of the app methods. Prior approaches for automated exploration of Android apps have run apps in an emulator or focused on small apps whose source code was available. To address these problems, we present A3E, an approach and tool that allows substantial Android apps to be explored systematically while running on actual phones, yet without requiring access to the app's source code. The key insight of our approach is to use a static, taint-style, dataflow analysis on the app bytecode in a novel way, to construct a high-level control flow graph that captures legal transitions among activities (app screens). We then use this graph to develop an exploration strategy named Targeted Exploration that permits fast, direct exploration of activities, including activities that would be difficult to reach during normal use. We also developed a strategy named Depth-first Exploration that mimics user actions for exploring activities and their constituents in a slower, but more systematic way. To measure the effectiveness of our techniques, we use two metrics: activity coverage (number of screens explored) and method coverage. Experiments with using our approach on 25 popular Android apps including BBC News, Gas Buddy, Amazon Mobile, YouTube, Shazam Encore, and CNN, show that our exploration techniques achieve 59.39--64.11% activity coverage and 29.53--36.46% method coverage.

References

  1. Gartner. Inc. Gartner Says Worldwide PC Shipment Growth Was Flat in Second Quarter of 2012. July, 2012. URL http://www.gartner.com/it/page.jsp?id=2079015.Google ScholarGoogle Scholar
  2. Gartner. Inc. Gartner Highlights Key Predictions for IT Organizations and Users in 2010 and Beyond. January, 2010. URL http://www.gartner.com/it/page.jsp?id=1278413.Google ScholarGoogle Scholar
  3. IDC. Android and iOS Surge to New Smartphone OS Record in Second Quarter, According to IDC. August, 2012. URL http://www.idc.com/getdoc.jsp?containerId=prUS23638712.Google ScholarGoogle Scholar
  4. CNET. Android reclaims 61 percent of all U.S. smartphone sales. May, 2012. URL http://news.cnet.com/8301-1023_3-57429192-93/android-reclaims-61-percent-of-all-u.s-smartphone-sales/.Google ScholarGoogle Scholar
  5. Casper S Jensen, M. R. Prasad, and A. Møller. Automated testing with targeted event sequence generation. In Proceedings of the 2013 International Symposium on Software Testing and Analysis, pages 67--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. Hu and I. Neamtiu. Automating GUI testing for Android applications. In AST '11, pages 77--83. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. L. Gomez, I. Neamtiu, T.Azim, and T. Millstein. RERAN: Timing- and Touch-Sensitive Record and Replay for Android. In ICSE '13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Jessica Guyn. Facebook users give iPhone app thumbs down. In Los Angeles Times. July, 2011. URL http://latimesblogs.latimes.com/technology/2011/07/facebook-users-give-iphone-app-thumbs-down.html, July 21.Google ScholarGoogle Scholar
  9. Domenico Amalfitano, Anna Rita Fasolino and Salvatore De. Using GUI Ripping for Automated Testing of Android Applications. In ASE'2012, pages 258--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Vaibhav Rastogi, Yan Chen and William Enck. AppsPlay- ground: automatic security analysis of smartphone applications. In CODASPY'2013, pages 209--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Wei Yang, Mukul Prasad and Tao Xie. A Grey-box Approach for Automated GUI-Model Generation of Mobile Applications. In FASE'13, pages 250--265. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Xun Yuan and Atif M. Memon. Generating Event Sequence- Based Test Cases Using GUI Run-Time State Feedback. In IEEE Transactions on Software Engineering, 2010, pages 81--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Xun Yuan and Atif M. Memon. Using GUI Run-Time State as Feedback to Generate Test Cases. In ICSE '07, pages 396--405. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Atif M. Memon. An event-flow model of GUI-based applications for testing. In Software Testing, Verification and Reliability, 2007, pages 137--157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Android Developers. UI/Application Exerciser Monkey. August, 2012. URL http://developer.android.com/tools/help/monkey.html.Google ScholarGoogle Scholar
  16. Atif Memon. GUITAR. August, 2012. URL guitar.sourceforge.net.Google ScholarGoogle Scholar
  17. Atif Memon. Android GUITAR. August, 2012. URL http://sourceforge.net/apps/mediawiki/guitar/index.php?title=Android_GUITAR.Google ScholarGoogle Scholar
  18. Google Code. Robotium. August, 2012. URL http://code.google.com/p/robotium/.Google ScholarGoogle Scholar
  19. SONY. APK Analyzer. January, 2013. URL http://developer.sonymobile.com/knowledge-base/tool-guides/analyse-your-apks-with-apkanalyser/.Google ScholarGoogle Scholar
  20. Jinseong Jeon and Jeffrey S. Foster. Troyd. January, 2013. URL https://github.com/plum-umd/troyd.Google ScholarGoogle Scholar
  21. Google Code. Androguard. January, 2013. URL http://code.google.com/p/androguard/.Google ScholarGoogle Scholar
  22. Google Code. Droidbox. January, 2013. URL http://code.google.com/p/droidbox/.Google ScholarGoogle Scholar
  23. Google Code. Android Assault. January, 2013. URL http://code.google.com/p/android-assault/.Google ScholarGoogle Scholar
  24. Android Developers. MonkeyRunner. August, 2012. URL http://developer.android.com/guide/developing/tools/monkeyrunner_concepts.html.Google ScholarGoogle Scholar
  25. Various Authors. SCanDroid. January, 2013. URL https://github.com/scandroid/scandroid.Google ScholarGoogle Scholar
  26. Bitbar. Automated Testing Tool for Android - Testdroid. January, 2013. URL http://testdroid.com/.Google ScholarGoogle Scholar
  27. Ranonex. Android Test Automation - Automate your App Testing. January, 2013. URL http://www.ranorex.com/mobile-automation-testing/android-test-automation.html.Google ScholarGoogle Scholar
  28. W. River. Wind River Framework for Automated Software Testing. January, 2013. URL http://www.windriver.com/announces/fast/.Google ScholarGoogle Scholar
  29. TestPlant. eggPlant for mobile testing.. January, 2013. URL http://www.testplant.com/products/eggplant/mobile/.Google ScholarGoogle Scholar
  30. Pamela Bhattacharya, Liudmila Ulanova, Iulian Neamtiu and Sai Charan Koduru. An Empirical Analysis of the Bug-fixing Process in Open Source Android Apps. In CSMR'13.Google ScholarGoogle Scholar
  31. W. Enck, P. Gilbert, B. G. Chun, L. P. Cox, J. Jung, P. McDaniel, and A. N. Sheth. Taintdroid: An information-flow tracking system for realtime privacy monitoring on smartphones. In OSDI, 2010, pages 393--407. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Android Developers. Android Emulator Limitations. March, 2013. URL http://developer.android.com/tools/devices/emulator.html#limitations.Google ScholarGoogle Scholar
  33. Android Developers. Android Intents. March, 2013. URL http://developer.android.com/reference/android/content/Intent.html.Google ScholarGoogle Scholar
  34. S. Anand, M. Naik, M. J. Harrold, and H. Yang. Automated concolic testing of smartphone apps. In FSE '12, pages 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. M. Böhmer, B. Hecht, J. Schöning, A. Krüger, and G. Bauer. Falling asleep with Angry Birds, Facebook and Kindle: a large scale study on mobile application usage. In MobileHCI '11, pages 47--56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. B. Cornelissen, A. Zaidman, A. van Deursen, L. Moonen, and R. Koschke. A Systematic Survey of Program Comprehension through Dynamic Analysis. In Software Engineering, IEEE Transactions on, 2009, pages 684--702. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Michael D. Ernst. Static and dynamic analysis: Synergy and duality. In WODA 2003: Workshop on Dynamic Analysis, May 9, pages 24--27.Google ScholarGoogle Scholar
  38. S. Hao, D. Li, W. Halfond, and R. Govindan. Estimating Android applications' CPU energy usage via bytecode profiling. In Green and Sustainable Software (GREENS), 2012 First International Workshop on, 2012, pages 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  39. M. Dong and L. Zhong. Self-constructive high-rate system energy modeling for battery-powered mobile systems. In MobiSys '11, pages 335--348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. X. Wei, L. Gomez, I. Neamtiu, and M. Faloutsos. Profile- Droid: multi-layer profiling of android applications. In Mobicom'12, pages 137--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. T. Takala, M. Katara, and J. Harty. Experiences of system-level model-based GUI testing of an Android application. In ICST '11, pages 377--386. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Targeted and depth-first exploration for systematic testing of android apps

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader
              About Cookies On This Site

              We use cookies to ensure that we give you the best experience on our website.

              Learn more

              Got it!