skip to main content
research-article

Replacement attacks against VM-protected applications

Published:03 March 2012Publication History
Skip Abstract Section

Abstract

Process-level virtualization is increasingly being used to enhance the security of software applications from reverse engineering and unauthorized modification (called software protection). Process-level virtual machines (PVMs) can safeguard the application code at run time and hamper the adversary's ability to launch dynamic attacks on the application. This dynamic protection, combined with its flexibility, ease in handling legacy systems and low performance overhead, has made process-level virtualization a popular approach for providing software protection. While there has been much research on using process-level virtualization to provide such protection, there has been less research on attacks against PVM-protected software. In this paper, we describe an attack on applications protected using process-level virtualization, called a replacement attack. In a replacement attack, the adversary replaces the protecting PVM with an attack VM thereby rendering the application vulnerable to analysis and modification. We present a general description of the replacement attack methodology and two attack implementations against a protected application using freely available tools. The generality and simplicity of replacement attacks demonstrates that there is a strong need to develop techniques that meld applications more tightly to the protecting PVM to prevent such attacks.

References

  1. Anckaert, B., Jakubowski, M., and Venkatesan, R. Proteus: virtualization for diversified tamper-resistance. In DRM '06: Proceedings of the ACM Workshop on Digital Rights Management (New York, NY, USA, 2006), ACM Press, pp. 47--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Apple. Mac OS X ABI Mach-o file format reference, 2009.Google ScholarGoogle Scholar
  3. Bellard, F. QEMU, a fast and portable dynamic translator. In ATEC'05: Proceedings of the USENIX Annual Technical Conference (Berkeley, CA, USA, 2005), USENIX Association, pp. 41--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Billet, O., Gilbert, H., and Ech-Chatbi, C. Cryptanalysis of a white box AES implementation. In Selected Areas in Cryptography (Hiedelberg, 2004), Springer-Verlag, pp. 227--240. Google ScholarGoogle Scholar
  5. Biondi, P., and Fabrice, D. Silver needle in the skype. In Black Hat Europe (Amsterdam, the Netherlands, 2006).Google ScholarGoogle Scholar
  6. Borello, J.-M., and Mè, L. Code obfuscation techniques for metamorphic viruses. Journal in Computer Virology 4 (2008), 211--220. 10.1007/s11416-008-0084--2.Google ScholarGoogle ScholarCross RefCross Ref
  7. Bruening, D., Garnett, T., and Amarasinghe, S. An infrastructure for adaptive dynamic optimization. In CGO '03: Proceedings of the IEEE/ACM International Symposium on Code Generation and Optimization (Los Alamitos, CA, USA, 2003), IEEE Computer Society, pp. 265--275. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Caballero, J., Johnson, N. M., McCamant, S., and Song, D. Binary code extraction and interface identification for security applications. In NDSS '10: Proceedings of the Network and Distributed System Security Symposium (2010), The Internet Society.Google ScholarGoogle Scholar
  9. Cappaert, J., Preneel, B., Anckaert, B., Madou, M., and De Bosschere, K. Towards tamper resistant code encryption: practice and experience. In ISPEC'08: Proceedings of the 4th International Conference on Information Security Practice and Experience (Berlin, Heidelberg, 2008), Springer-Verlag, pp. 86--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Chang, H., and Atallah, M. Protecting software code by guards. In Proceedings of the ACM Workshop on Security and Privacy in Digital Rights Management (2000), pp. 160--175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chen, X., Garfinkel, T., Lewis, E. C., Subrahmanyam, P., Waldspurger, C. A., Boneh, D., Dwoskin, J., and Ports, D. R. Overshadow: a virtualization-based approach to retrofitting protection in commodity operating systems. In ASPLOS XIII: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems (New York, NY, USA, 2008), ACM Press, pp. 2--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chow, S., Eisen, P. A., Johnson, H., and Oorschot, P. C. v. White-box cryptography and an AES implementation. In SAC '02: Revised Papers from the 9th Annual International Workshop on Selected Areas in Cryptography (London, UK, 2003), Springer-Verlag, pp. 250--270. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Collberg, C., Thomborson, C., and Low, D. A taxonomy of obfuscating transformations. University of Auckland Technical Report (1997), 170.Google ScholarGoogle Scholar
  14. Collberg, C., Thomborson, C., and Low, D. Manufacturing cheap, resilient and stealthy opaque constructs. In POPL'98:Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (New York, NY, USA, 1998), ACM Press, pp. 184--196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Coogan, K., Lu, G., and Debray, S. Deobfuscating virtualization-obfuscated software: A semantics-based approach. CCS '11: Proceedings of the ACM Conference on Computer and Communications Security (October 2011). To appear. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. De Bus, B., De Sutter, B., Van Put, L., Chanet, D., and De Bosschere, K. Link-time optimization of ARM binaries. In LCTES'04: Proceedings of the 2004 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (Washington D.C., U.S.A, 7 2004), ACM Press, pp. 211--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Dehnert, J. C., Grant, B. K., Banning, J. P., Johnson, R., Kistler, T., Klaiber, A., and Mattson, J. The Transmeta code morphing software: using speculation, recovery, and adaptive retranslation to address real-life challenges. In CGO'03: Proceedings of the International Symposium on Code Generation and Optimization (Washington, DC, USA, 2003), IEEE Computer Society, pp. 15--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Eagle, C. The IDA Pro Book: The Unofficial Guide to the World's Most Popular Disassembler. No Starch Press, San Francisco, CA, USA, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Garfinkel, T., Pfaff, B., Chow, J., Rosenblum, M., and Boneh, D. Terra: a virtual machine-based platform for trusted computing. In SOSP'03: Proceedings of the 19th ACM Symposium on Operating Systems Principles (New York, NY, USA, 2003), ACM Press, pp. 193--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Ghosh, S., Hiser, J. D., and Davidson, J. W. A secure and robust approach to software tamper resistance. In IH '10: Proceedings of the 12th International Conference on Information Hiding (Berlin, Heidelberg, 2010), Springer-Verlag, pp. 33--47. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Gröbert, F., Willems, C., and Holz, T. Automatic identification of cryptographic primitives in binary programs. In RAID '11: Proceedings of the 14th International Symposium on Recent Advances in Intrusion Detection (London, UK, 2011), Springer-Verlag, pp. 45--65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Halderman, J. A., Schoen, S. D., Heninger, N., Clarkson, W., Paul, W., Calandrino, J. A., Feldman, A. J., Appelbaum, J., and Felten, E. W. Lest we remember: cold-boot attacks on encryption keys, May 2009.Google ScholarGoogle Scholar
  23. Hiser, J. D., Coleman, C. L., Co, M., and Davidson, J. W. Meds: The memory error detection system. In ESSoS '09: Proceedings of the 1st International Symposium on Engineering Secure Software and Systems (Berlin, Heidelberg, 2009), Springer-Verlag, pp. 164--179. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Hiser, J. D., Williams, D., Filipi, A., Davidson, J. W., and Childers, B. R. Evaluating fragment construction policies for SDT systems. In VEE '06: Proceedings of the 2nd International Conference on Virtual Execution Environments (New York, NY, USA, 2006), ACM Press, pp. 122--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Hiser, J. D., Williams, D., Hu, W., Davidson, J. W., Mars, J., and Childers, B. R. Evaluating indirect branch handling mechanisms in software dynamic translation systems. In CGO'07: Proceedings of the International Symposium on Code Generation and Optimization (Washington, DC, USA, 2007), IEEE Computer Society, pp. 61--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Horspool, R. N., and Marovac, N. An approach to the problem of detranslation of computer programs. Computer Journal 23, 3 (1980), 223--229.Google ScholarGoogle ScholarCross RefCross Ref
  27. Hu, W., Hiser, J. D., Williams, D., Filipi, A., Davidson, J. W., Evans, D., Knight, J. C., Nguyen-Tuong, A., and Rowanhill, J. Secure and practical defense against code-injection attacks using software dynamic translation. In Proceedings of the 2nd International Conference on Virtual Execution Environments (New York, NY, USA, 2006), ACM Press, pp. 2--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Kanzaki, Y., Monden, A., Nakamura, M., and Matsumoto, K.-i. Exploiting self-modification mechanism for program protection. In COMPSAC'03: Proceedings of the 27th Annual International Conference on Computer Software and Applications (Washington, DC, USA, 2003), IEEE Computer Society, pp. 170--176. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kc, G. S., Keromytis, A. D., and Prevelakis, V. Countering code-injection attacks with instruction-set randomization. In CCS '03: Proceedings of the 10th ACM Conference on Computer and Communications Security (New York, NY, USA, 2003), ACM Press, pp. 272--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Kiriansky, V., Bruening, D., and Amarasinghe, S. P. Secure execution via program shepherding. In USENIX'02: Proceedings of the 11th USENIX Security Symposium (Berkeley, CA, USA, 2002), USENIX Association, pp. 191--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Leder, F., Martini, P., and Wichmann, A. Finding and extracting crypto routines from malware. In Proceedings of the IEEE 28th International Performance Computing and Communications Conference (IPCCC) (Washington, DC,USA, December 2009), IEEE, pp. 394 --401.Google ScholarGoogle ScholarCross RefCross Ref
  32. Linn, C., and Debray, S. Obfuscation of executable code to improve resistance to static disassembly. In CCS'03: Proceedings of the 10th ACM Conference on Computer and Communications Security (CCS) (Washington D.C., U.S.A, 2003), ACM Press, pp. 290--299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Luk, C.-K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V. J., and Hazelwood, K. Pin: building customized program analysis tools with dynamic instrumentation. In PLDI '05: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (New York, NY, USA, 2005), ACM Press, pp. 190--200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Madou, M., Anckaert, B., De Sutter, B., and De Bosschere, K. Hybrid static-dynamic attacks against software protection mechanisms. In DRM '05: Proceedings of the 5th ACM workshop on Digital Rights Management (New York, NY, USA, 2005), ACM Press, pp. 75--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Madou, M., Anckaert, B., Moseley, P., Debray, S., De Sutter, B., and De Bosschere, K. Software protection through dynamic code mutation. In The 6th International Workshop on Information Security Applications (WISA 2005) (August 2005), vol. LNCS, Springer Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Oreans Technologies. Themida. http://oreans.com/themida.php, 2009.Google ScholarGoogle Scholar
  37. Oreons Technology. Codevirtualizer. http://oreans.com/codevirtualizer.php, 2009.Google ScholarGoogle Scholar
  38. Payer, M., and Gross, T. R. Fine-grained user-space security through virtualization. In VEE'11: Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (New York, NY, USA, 2011), ACM Press, pp. 157--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Popek, G. J., and Goldberg, R. P. Formal requirements for virtualizable third generation architectures. Communications of the ACM 17 (July 1974), 412--421. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Portokalidis, G., and Keromytis, A. D. Fast and practical instruction-set randomization for commodity systems. In ACSAC'10: Proceedings of the 26th Annual Computer Security Applications Conference (New York, NY, USA, 2010), ACM Press, pp. 41--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Rolles, R. Unpacking virtualization obfuscators. In WOOT'09: Proceedings of the 3rd USENIX Conference on Offensive Technologies (Berkeley, CA, USA, 2009), USENIX Association, pp. 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Scott, K., and Davidson, J. Safe virtual execution using software dynamic translation. In ACSAC '02: Proceedings of the 18th Annual Computer Security Applications Conference (Los Alamitos, CA, USA, 2002), IEEE Computer Society, p. 209. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Scott, K., Kumar, N., Velusamy, S., Childers, B., Davidson, J. W., and Soffa, M. L. Retargetable and reconfigurable software dynamic translation. In CGO '03: Proceedings of the International Symposium on Code Generation and Optimization (Washington D.C., U.S.A, 2003), IEEE Computer Society, pp. 36--47. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Sharif, M., Lanzi, A., Giffin, J., and Lee, W. Automatic reverse engineering of malware emulators. In SP'07: Proceedings of the 2009 30th IEEE Symposium on Security and Privacy (Washington, DC, USA, 2009), IEEE Computer Society, pp. 94--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Sites, R. L., Chernoff, A., Kirk, M. B., Marks, M. P., and Robinson, S. G. Binary translation. Communcations of the ACM 36 (February 1993), 69--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Smith, J., and Nair, R. Virtual Machines: Versatile Platforms for Systems and Processes (The Morgan Kaufmann Series in Computer Architecture and Design). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Sridhar, S., Shapiro, J. S., Northup, E., and Bungale, P. P. HDTrans: an open source, low-level dynamic instrumentation system. In VEE'06: Proceedings of the 2nd International Conference on Virtual Execution Environments (New York, NY, USA, 2006), ACM, pp. 175--185. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. StarForce. Starforce crypto. http://www.star-force.com/, 2008.Google ScholarGoogle Scholar
  49. Szor, P. The Art of Computer Virus Research and Defense. Addison-Wesley Professional, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Udupa, S., Debray, S., and Madou, M. Deobfuscation: reverse engineering obfuscated code. In WCRE '05: Proceedings of the International Working Conference on Reverse Engineering (Los Alamitos, CA, USA, Nov. 2005), vol. 0, IEEE Computer Society, pp. 45--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. VMProtect Software. VMProtect. http://vmpsoft.com/, 2008.Google ScholarGoogle Scholar
  52. Wang, C., Davidson, J., Hill, J., and Knight, J. Protection of software-based survivability mechanisms. In DSN'01: Proceedings of the International Conference on Dependable Systems and Networks (Goteborg, Sweden, 2001), IEEE Computer Society, pp. 193--202. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Wang, C., Hill, J., Knight, J., and Davidson, J. Software tamper resistance: Obstructing static analysis of programs. Tech. rep., Charlottesville, VA, USA, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Youngdale, E. Kernel korner: The ELF object file format: Introduction. Linux Journal 1995 (April 1995). Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Yourst, M. PTLsim: A cycle accurate full system x86--64 microarchitectural simulator. In ISPASS'07: Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (2007), IEEE, pp. 23--34.Google ScholarGoogle ScholarCross RefCross Ref
  56. Zambreno, J., Choudhary, A., Simha, R., Narahari, B., and Memon, N. SAFE-OPS: An approach to embedded software security. Transactions on Embedded Computing Systems 4, 1 (2005), 189--210. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Replacement attacks against VM-protected applications

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!