skip to main content
research-article

Dispersing proprietary applications as benchmarks through code mutation

Published:01 March 2008Publication History
Skip Abstract Section

Abstract

Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not representative of the real-life applications of interest.

This paper proposes code mutation, a novel technique that mutates a proprietary application to complicate reverse engineering so that it can be distributed as a benchmark. The benchmark mutant then serves as a proxy for the proprietary application. The key idea in the proposed code mutation approach is to preserve the proprietary application's dynamic memory access and/or control flow behavior in the benchmark mutant while mutating the rest of the application code. To this end, we compute program slices for memory access operations and/or control flow operationstrimmed through constant value and branch profiles; and subsequently mutate the instructions not appearing in these slices through binary rewriting.

Our experimental results using SPEC CPU2000 and MiBench benchmarks show that code mutation is a promising technique that mutates up to 90% of the static binary, up to 50% of the dynamically executed instructions, and up to 35% of the at run time exposed inter-operation data dependencies. The performance characteristics of the mutant are very similar to those of the proprietary application across a wide range of microarchitectures and hardware implementations.

Skip Supplemental Material Section

Supplemental Material

Video

References

  1. H. Agrawal and J. Horgan. Dynamic program slicing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 246--256, June 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. Bell, Jr. and L. K. John. Improved automatic testcase synthesis for performance model validation. In Proceedings of the ACM International Conference on Supercomputing (ICS), pages 111--120, June 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. B. Calder, P. Feller, and A. Eustace. Value profiling. In Proceedings of the Annual ACM/IEEE International Symposium on Microarchitecture (MICRO), pages 259--269, Dec. 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. C. Collberg, C. Thomborson, and D. Low. A taxonomy of obfuscating transformations. Technical Report 148, The University of Auckland, July 1997.Google ScholarGoogle Scholar
  5. T. M. Conte, M. A. Hirsch, and K. N. Menezes. Reducing state loss for effective trace sampling of superscalar processors. In Proceedings of the International Conference on Computer Design (ICCD), pages 468--477, Oct. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. H. J. Curnow and B. A. Wichmann. A synthetic benchmark. The Computer Journal, 19(1):43--49, 1976.Google ScholarGoogle ScholarCross RefCross Ref
  7. B. De Bus, D. Kaestner, D. Chanet, L. Van Put, and B. De Sutter. Post-pass compaction techniques. Communications of the ACM, 46(8):41--46, Aug. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. L. Eeckhout, R. H. Bell Jr., B. Stougie, K. De Bosschere, and L. K. John. Control flow modeling in statistical simulation for accurate and efficient processor design studies. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 350--361, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J. Ge, S. Chaudhuri, and A. Tyagi. Control flow based obfuscation. In Proceedings of the ACM Workshop on Digital Rights Management (DRM), pages 83--92, Nov. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. R. Guthaus, J. S. Ringenberg, D. Ernst, T. M. Austin, T. Mudge, and R. B. Brown. MiBench: A free, commercially representative embedded benchmark suite. In Proceedings of the IEEE Annual Workshop on Workload Characterization (WWC), Dec. 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. L. Henning. SPEC CPU2000: Measuring CPU performance in the new millennium. IEEE Computer, 33(7):28--35, July 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. K. Hoste, A. Phansalkar, L. Eeckhout, A. Georges, L. K. John, and K. De Bosschere. Performance prediction based on inherent program similarity. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 114--122, Sept. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. C. Hsieh and M. Pedram. Micro-processor power estimation using profile-driven program synthesis. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 17(11):1080--1089, Nov. 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. V. S. Iyengar, L. H. Trevillyan, and P. Bose. Representative traces for processor models with infinite cache. In Proceedings of the Second International Symposium on High-Performance Computer Architecture (HPCA), pages 62--73, Feb. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. A. M. Joshi, L. Eeckhout, R. H. Bell Jr., and L. K. John. Performance cloning: A technique for disseminating proprietary applications as benchmarks. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), pages 105--115, Oct. 2006.Google ScholarGoogle ScholarCross RefCross Ref
  16. T. S. Karkhanis and J. E. Smith. A first-order superscalar processor model. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 338--349, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. M. H. Lipasti, C. B. Wilkerson, and J. P. Shen. Value locality and load value prediction. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 138--147, Oct. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. C.-K. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S.Wallace, V. J. Reddi, and K. Hazelwood. Pin: Building customized program analysis tools with dynamic instrumentation. In Proceedings of the ACM SIGPLAN Conference on Programming Languages Design and Implementation (PLDI), pages 190--200, June 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. S. Narayanasamy, C. Pereira, H. Patil, R. Cohn, and B. Calder. Automatic logging of operating system effects to guide application level architecture simulation. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pages 216--227, June 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. S. Nussbaum and J. E. Smith. Modeling superscalar processors via statistical simulation. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 15--24, Sept. 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. Oskin, F. T. Chong, and M. Farrens. HLS: Combining statistical and symbolic simulation to guide microprocessor design. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 71--82, June 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Ringenberg, C. Pelosi, D. Oehmke, and T. Mudge. Intrinsic checkpointing: A methodology for decreasing simulation time through binary modification. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 78--88, Mar. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. T. Sherwood, E. Perelman, G. Hamerly, and B. Calder. Automatically characterizing large scale program behavior. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 45--57, Oct. 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. K. Skadron, M. Martonosi, D. I. August, M. D. Hill, D. J. Lilja, and V. S. Pai. Challenges in computer architecture evaluation. IEEE Computer, 36(8):30--36, Aug. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Srivastava and A. Eustace. ATOM: A system for building customized program analysis tools. Technical Report 94/2, Western Research Lab, Compaq, Mar. 1994.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. F. Tip. A survey of program slicing techniques. Journal of Programming Languages, 3(3):121--189, 1995.Google ScholarGoogle Scholar
  27. R. P. Weicker. Dhrystone: A synthetic systems programming benchmark. Communications of the ACM, 27(10):1013--1030, Oct. 1984. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. M. Weiser. Program slicing. IEEE Transaction on Software Engineering, 10(4):352--357, July 1984.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. R. E.Wunderlich, T. F.Wenisch, B. Falsafi, and J. C. Hoe. SMARTS: Accelerating microarchitecture simulation via rigorous statistical sampling. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 84--95, June 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. T. Yourst. PTLsim: A cycle accurate full system x86-64 microarchitectural simulator. In Proceedings of the International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 23--34, Apr. 2007.Google ScholarGoogle ScholarCross RefCross Ref
  31. X. Zhang, R. Gupta, and Y. Zhang. Cost and precision tradeoffs of dynamic data slicing algorithms. ACM Transactions on Programming Languages and Systems, 27(4):631--661, July 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. C. B. Zilles and G. S. Sohi. Understanding the backward slices of performance degrading instructions. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 172--181, June 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Dispersing proprietary applications as benchmarks through code mutation

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGARCH Computer Architecture News
          ACM SIGARCH Computer Architecture News  Volume 36, Issue 1
          ASPLOS '08
          March 2008
          339 pages
          ISSN:0163-5964
          DOI:10.1145/1353534
          Issue’s Table of Contents
          • cover image ACM Conferences
            ASPLOS XIII: Proceedings of the 13th international conference on Architectural support for programming languages and operating systems
            March 2008
            352 pages
            ISBN:9781595939586
            DOI:10.1145/1346281

          Copyright © 2008 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 1 March 2008

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!