skip to main content
10.1145/3371307.3371309acmotherconferencesArticle/Chapter ViewAbstractPublication PagesssprewConference Proceedingsconference-collections
research-article

Obfuscation: where are we in anti-DSE protections? (a first attempt)

Published: 09 December 2019 Publication History

Abstract

Obfuscation is widely used to protect software against man-at-the-end attacks. Recent attacks based on semantic methods, especially dynamic symbolic execution (DSE), have proven extremely powerful against standard obfuscation techniques, leading several teams to investigate anti-DSE protections. Yet, the domain is in its infancy, and the current state of research on the topic is quite unclear. We propose a systematic review of anti-DSE techniques. In particular, we propose a classification and identify strengths and weaknesses of the current lines of research, as well as promising future directions.

References

[1]
Tigress challenge. http://tigress.cs.arizona.edu/challenges.html.
[2]
S. Anand, E. K. Burke, T. Y. Chen, J. Clark, M. B. Cohen, W. Grieskamp, M. Harman, M. J. Harrold, and P. McMinn. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software, 2013.
[3]
Domagoj Babic, Lorenzo Martignoni, Stephen McCamant, and Dawn Song. Statically-directed dynamic automated test generation. In Proceedings of the 20th International Symposium on Software Testing and Analysis, ISSTA 2011, Toronto, ON, Canada, July 17--21, 2011, pages 12--22, 2011.
[4]
Sebastian Banescu, Christian S. Collberg, Vijay Ganesh, Zack Newsham, and Alexander Pretschner. Code obfuscation against symbolic execution attacks. In Annual Conference on Computer Security Applications, ACSAC 2016, 2016.
[5]
Sebastian Banescu, Christian S. Collberg, and Alexander Pretschner. Predicting the resilience of obfuscated code against symbolic execution attacks via machine learning. In USENIX Security Symposium, 2017.
[6]
Boaz Barak. Hopes, fears, and software obfuscation. Commun. ACM, 59(3):88--96, 2016.
[7]
Boaz Barak, Oded Goldreich, Russell Impagliazzo, Steven Rudich, Amit Sahai, Salil P. Vadhan, and Ke Yang. On the (im)possibility of obfuscating programs. In Advances in Cryptology - CRYPTO, 2001.
[8]
Sébastien Bardin, Robin David, and Jean-Yves Marion. Backward-bounded DSE: targeting infeasibility questions on obfuscated codes. In 2017 IEEE Symposium on Security and Privacy, SP, 2017.
[9]
Sébastien Bardin and Philippe Herrmann. OSMOSE: automatic structural testing of executables. Softw. Test., Verif. Reliab., 21(1), 2011.
[10]
M. Beunardeau, A. Connolly, R. GÃl'raud, and D. Naccache. White-box cryptography: Security in an insecure environment. IEEE Security Privacy, 14(5):88--92, Sep. 2016.
[11]
Guillaume Bonfante, José M. Fernandez, Jean-Yves Marion, Benjamin Rouxel, Fabrice Sabatier, and Aurélien Thierry. Codisasm: Medium scale concatic disassembly of self-modifying binaries with overlapping instructions. In Conference on Computer and Communications Security, 2015.
[12]
Ella Bounimova, Patrice Godefroid, and David A. Molnar. Billions and billions of constraints: whitebox fuzz testing in production. In 35th International Conference on Software Engineering, ICSE '13, San Francisco, CA, USA, May 18--26, 2013. IEEE Computer Society, 2013.
[13]
David Brumley, Cody Hartwig, Zhenkai Liang, James Newsome, Dawn Xiaodong Song, and Heng Yin. Automatically identifying trigger-based behavior in malware. In Wenke Lee, Cliff Wang, and David Dagon, editors, Botnet Detection: Countering the Largest Security Threat, volume 36 of Advances in Information Security, pages 65--88. Springer, 2008.
[14]
Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In 8th USENIX Symposium on Operating Systems Design and Implementation, OSDI, 2008.
[15]
Cristian Cadar and Koushik Sen. Symbolic execution for software testing: three decades later. Commun. ACM, 56(2), 2013.
[16]
Sang Kil Cha, Thanassis Avgerinos, Alexandre Rebert, and David Brumley. Unleashing mayhem on binary code. In Symposium on Security and Privacy, SP, 2012.
[17]
Vitaly Chipounov, Volodymyr Kuznetsov, and George Candea. The S2E platform: Design, implementation, and applications. ACM Trans. Comput. Syst., 30(1):2:1--2:49, 2012.
[18]
Christian Collberg and Jasvir Nagra. Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection. Addison-Wesley Professional, 1st edition, 2009.
[19]
Christian Collberg, Clark Thomborson, and Douglas Low. A taxonomy of obfuscating transformations, 1997.
[20]
Christian S. Collberg, Clark D. Thomborson, and Douglas Low. Manufacturing cheap, resilient, and stealthy opaque constructs. In, Symposium on Principles of Programming Languages, POPL, 1998.
[21]
Kevin Coogan, Gen Lu, and Saumya K. Debray. Deobfuscation of virtualization-obfuscated software: a semantics-based approach. In Conference on Computer and Communications Security, CCS, 2011.
[22]
Robin David, Sébastien Bardin, Josselin Feist, Laurent Mounier, Marie-Laure Potet, Thanh Dinh Ta, and Jean-Yves Marion. Specification of concretization and symbolization policies in symbolic execution. In International Symposium on Software Testing and Analysis, ISSTA 2016, 2016.
[23]
Robin David, Sébastien Bardin, Thanh Dinh Ta, Laurent Mounier, Josselin Feist, Marie-Laure Potet, and Jean-Yves Marion. BINSEC/SE: A dynamic symbolic execution toolkit for binary-level analysis. In IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, SANER, 2016.
[24]
Saumya K. Debray and Jay Patel. Reverse engineering self-modifying code: Unpacker extraction. In Working Conference on Reverse Engineering, WCRE, 2010.
[25]
Ninon Eyrolles, Louis Goubin, and Marion Videau. Defeating mba-based obfuscation. In Proceedings of the 2016 ACM Workshop on Software PROtection, SPRO@CCS 2016, 2016.
[26]
Patrice Godefroid, Michael Y. Levin, and David A. Molnar. SAGE: whitebox fuzzing for security testing. Commun. ACM, 55(3), 2012.
[27]
Nguyen Minh Hai, Mizuhito Ogawa, and Quan Thanh Tho. Obfuscation code localization based on CFG generation of malware. In Foundations and Practice of Security - 8th International Symposium, FPS 2015, Clermont-Ferrand, France, October 26--28, 2015, Revised Selected Papers. Springer, 2015.
[28]
Min Gyung Kang, Pongsin Poosankam, and Heng Yin. Renovo: a hidden code extractor for packed executables. In ACM Workshop Recurring Malcode (WORM). ACM, 2007.
[29]
Johannes Kinder. Towards static analysis of virtualization-obfuscated binaries. In 19th Working Conference on Reverse Engineering, WCRE, 2012.
[30]
Andreas Moser, Christopher Kruegel, and Engin Kirda. Limits of static analysis for malware detection. In 23rd Annual Computer Security Applications Conference (ACSAC 2007), December 10--14, 2007, Miami Beach, Florida, USA. IEEE Computer Society, 2007.
[31]
Mathilde Ollivier, Sébastien Bardin, Richard Bonichon, and Jean-Yves Marion. How to kill symbolic deobfuscation for free (or: Unleashing the potential of path-oriented protections). In Annual Conference on Computer Security Applications, ACSAC 2019. ACM, 2019.
[32]
Jonathan Salwan, Sébastien Bardin, and Marie-Laure Potet. Symbolic deobfuscation: from virtualized code back to the original. In 5th Conference on Detection of Intrusions and malware & Vulnerability Assessment (DIMVA), 2018.
[33]
Florent Saudel and Jonathan Salwan. Triton : Framework d'exÃl'cution concolique. In SSTIC, 2015.
[34]
Sebastian Schrittwieser, Stefan Katzenbeisser, Johannes Kinder, Georg Merzdovnik, and Edgar Weippl. Protecting software through obfuscation: Can it keep pace with progress in code analysis? ACM Comput. Surv., 49(1), 2016.
[35]
Edward J. Schwartz, Thanassis Avgerinos, and David Brumley. All you ever wanted to know about dynamic taint analysis and forward symbolic execution (but might have been afraid to ask). In Symposium on Security and Privacy, S&P, 2010.
[36]
Monirul I. Sharif, Andrea Lanzi, Jonathon T. Giffin, and Wenke Lee. Impeding malware analysis using conditional code obfuscation. In Network and Distributed System Security Symposium, NDSS, 2008.
[37]
Yan Shoshitaishvili, Ruoyu Wang, Christopher Salls, Nick Stephens, Mario Polino, Andrew Dutcher, John Grosen, Siji Feng, Christophe Hauser, Christopher Krügel, and Giovanni Vigna. SOK: (state of) the art of war: Offensive techniques in binary analysis. In IEEE Symposium on Security and Privacy, SP, 2016.
[38]
Venkatesh Srinivasan and Thomas W. Reps. An improved algorithm for slicing machine code. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2016. ACM.
[39]
Jon Stephens, Babak n, Christian S. Collberg, Saumya Debray, and Carlos Scheidegger. Probabilistic obfuscation through covert channels. In European Symposium on Security and Privacy, EuroS&P, 2018.
[40]
Ramtine Tofighi-Shirazi, Maria Christofi, Philippe Elbaz-Vincent, and Thanh Ha Le. Dose: Deobfuscation based on semantic equivalence. In Proceedings of the 8th Software Security, Protection, and Reverse Engineering Workshop, San Juan, PR, USA, December 3--4, 2018. ACM, 2018.
[41]
Julien Vanegue and Sean Heelan. SMT solvers in software security. In 6th USENIX Workshop on Offensive Technologies, WOOT'12, 2012.
[42]
Chenxi Wang, Jonathan Hill, John Knight, and Jack Davidson. Software tamper resistance: Obstructing static analysis of programs. Technical report, Charlottesville, VA, USA, 2000.
[43]
Zhi Wang, Jiang Ming, Chunfu Jia, and Debin Gao. Linear obfuscation to combat symbolic execution. In European Symposium on Research in Computer Security, ESORICS, 2011.
[44]
Babak Yadegari and Saumya Debray. Symbolic execution of obfuscated code. In Conference on Computer and Communications Security (CCS), 2015.
[45]
Babak Yadegari, Brian Johannesmeyer, Ben Whitely, and Saumya Debray. A generic approach to automatic deobfuscation of executable code. In Symposium on Security and Privacy, SP, 2015.
[46]
Yongxin Zhou, Alec Main, Yuan Xiang Gu, and Harold Johnson. Information hiding in software with mixed boolean-arithmetic transforms. In Information Security Applications, WISA, 2007.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SSPREW9 '19: Proceedings of the 9th Workshop on Software Security, Protection, and Reverse Engineering
December 2019
56 pages
ISBN:9781450377461
DOI:10.1145/3371307
© 2019 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 December 2019

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

SSPREW9

Acceptance Rates

Overall Acceptance Rate 6 of 13 submissions, 46%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)23
  • Downloads (Last 6 weeks)5
Reflects downloads up to 23 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Defeating Data Plane Attacks With Program ObfuscationIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.327793921:3(1317-1330)Online publication date: May-2024
  • (2024)WASMixer: Binary Obfuscation for WebAssemblyComputer Security – ESORICS 202410.1007/978-3-031-70896-1_5(88-109)Online publication date: 6-Sep-2024
  • (2023)Deep Learning for Zero-day Malware Detection and Classification: A SurveyACM Computing Surveys10.1145/360577556:2(1-37)Online publication date: 15-Sep-2023
  • (2023)Evaluating Defensive Countermeasures for Software-Based Hardware AbstractionE-Business and Telecommunications10.1007/978-3-031-36840-0_13(281-304)Online publication date: 22-Jul-2023
  • (2021)Search-Based Local Black-Box DeobfuscationProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3485337(2384-2386)Online publication date: 12-Nov-2021
  • (2021)Search-Based Local Black-Box Deobfuscation: Understand, Improve and MitigateProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3485250(2513-2525)Online publication date: 12-Nov-2021
  • (2021)Boosting SMT solver performance on mixed-bitwise-arithmetic expressionsProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3453483.3454068(651-664)Online publication date: 19-Jun-2021
  • (2020) SCORE : Source Code Optimization & REconstruction IEEE Access10.1109/ACCESS.2020.30089058(129478-129496)Online publication date: 2020

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media