Abstract
Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not representative of the real-life applications of interest.
This paper proposes code mutation, a novel technique that mutates a proprietary application to complicate reverse engineering so that it can be distributed as a benchmark. The benchmark mutant then serves as a proxy for the proprietary application. The key idea in the proposed code mutation approach is to preserve the proprietary application's dynamic memory access and/or control flow behavior in the benchmark mutant while mutating the rest of the application code. To this end, we compute program slices for memory access operations and/or control flow operationstrimmed through constant value and branch profiles; and subsequently mutate the instructions not appearing in these slices through binary rewriting.
Our experimental results using SPEC CPU2000 and MiBench benchmarks show that code mutation is a promising technique that mutates up to 90% of the static binary, up to 50% of the dynamically executed instructions, and up to 35% of the at run time exposed inter-operation data dependencies. The performance characteristics of the mutant are very similar to those of the proprietary application across a wide range of microarchitectures and hardware implementations.
Supplemental Material
Available for Download
Supplemental material for Dispersing proprietary applications as benchmarks through code mutation
- H. Agrawal and J. Horgan. Dynamic program slicing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 246--256, June 1990. Google Scholar
Digital Library
- R. Bell, Jr. and L. K. John. Improved automatic testcase synthesis for performance model validation. In Proceedings of the ACM International Conference on Supercomputing (ICS), pages 111--120, June 2005. Google Scholar
Digital Library
- B. Calder, P. Feller, and A. Eustace. Value profiling. In Proceedings of the Annual ACM/IEEE International Symposium on Microarchitecture (MICRO), pages 259--269, Dec. 1997. Google Scholar
Digital Library
- C. Collberg, C. Thomborson, and D. Low. A taxonomy of obfuscating transformations. Technical Report 148, The University of Auckland, July 1997.Google Scholar
- T. M. Conte, M. A. Hirsch, and K. N. Menezes. Reducing state loss for effective trace sampling of superscalar processors. In Proceedings of the International Conference on Computer Design (ICCD), pages 468--477, Oct. 1996. Google Scholar
Digital Library
- H. J. Curnow and B. A. Wichmann. A synthetic benchmark. The Computer Journal, 19(1):43--49, 1976.Google Scholar
Cross Ref
- B. De Bus, D. Kaestner, D. Chanet, L. Van Put, and B. De Sutter. Post-pass compaction techniques. Communications of the ACM, 46(8):41--46, Aug. 2003. Google Scholar
Digital Library
- L. Eeckhout, R. H. Bell Jr., B. Stougie, K. De Bosschere, and L. K. John. Control flow modeling in statistical simulation for accurate and efficient processor design studies. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 350--361, June 2004. Google Scholar
Digital Library
- J. Ge, S. Chaudhuri, and A. Tyagi. Control flow based obfuscation. In Proceedings of the ACM Workshop on Digital Rights Management (DRM), pages 83--92, Nov. 2005. Google Scholar
Digital Library
- M. R. Guthaus, J. S. Ringenberg, D. Ernst, T. M. Austin, T. Mudge, and R. B. Brown. MiBench: A free, commercially representative embedded benchmark suite. In Proceedings of the IEEE Annual Workshop on Workload Characterization (WWC), Dec. 2001. Google Scholar
Digital Library
- J. L. Henning. SPEC CPU2000: Measuring CPU performance in the new millennium. IEEE Computer, 33(7):28--35, July 2000. Google Scholar
Digital Library
- K. Hoste, A. Phansalkar, L. Eeckhout, A. Georges, L. K. John, and K. De Bosschere. Performance prediction based on inherent program similarity. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 114--122, Sept. 2006. Google Scholar
Digital Library
- C. Hsieh and M. Pedram. Micro-processor power estimation using profile-driven program synthesis. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 17(11):1080--1089, Nov. 1998. Google Scholar
Digital Library
- V. S. Iyengar, L. H. Trevillyan, and P. Bose. Representative traces for processor models with infinite cache. In Proceedings of the Second International Symposium on High-Performance Computer Architecture (HPCA), pages 62--73, Feb. 1996. Google Scholar
Digital Library
- A. M. Joshi, L. Eeckhout, R. H. Bell Jr., and L. K. John. Performance cloning: A technique for disseminating proprietary applications as benchmarks. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), pages 105--115, Oct. 2006.Google Scholar
Cross Ref
- T. S. Karkhanis and J. E. Smith. A first-order superscalar processor model. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 338--349, June 2004. Google Scholar
Digital Library
- M. H. Lipasti, C. B. Wilkerson, and J. P. Shen. Value locality and load value prediction. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 138--147, Oct. 1996. Google Scholar
Digital Library
- C.-K. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S.Wallace, V. J. Reddi, and K. Hazelwood. Pin: Building customized program analysis tools with dynamic instrumentation. In Proceedings of the ACM SIGPLAN Conference on Programming Languages Design and Implementation (PLDI), pages 190--200, June 2005. Google Scholar
Digital Library
- S. Narayanasamy, C. Pereira, H. Patil, R. Cohn, and B. Calder. Automatic logging of operating system effects to guide application level architecture simulation. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pages 216--227, June 2006. Google Scholar
Digital Library
- S. Nussbaum and J. E. Smith. Modeling superscalar processors via statistical simulation. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 15--24, Sept. 2001. Google Scholar
Digital Library
- M. Oskin, F. T. Chong, and M. Farrens. HLS: Combining statistical and symbolic simulation to guide microprocessor design. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 71--82, June 2000. Google Scholar
Digital Library
- J. Ringenberg, C. Pelosi, D. Oehmke, and T. Mudge. Intrinsic checkpointing: A methodology for decreasing simulation time through binary modification. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 78--88, Mar. 2005. Google Scholar
Digital Library
- T. Sherwood, E. Perelman, G. Hamerly, and B. Calder. Automatically characterizing large scale program behavior. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 45--57, Oct. 2002. Google Scholar
Digital Library
- K. Skadron, M. Martonosi, D. I. August, M. D. Hill, D. J. Lilja, and V. S. Pai. Challenges in computer architecture evaluation. IEEE Computer, 36(8):30--36, Aug. 2003. Google Scholar
Digital Library
- A. Srivastava and A. Eustace. ATOM: A system for building customized program analysis tools. Technical Report 94/2, Western Research Lab, Compaq, Mar. 1994.Google Scholar
Digital Library
- F. Tip. A survey of program slicing techniques. Journal of Programming Languages, 3(3):121--189, 1995.Google Scholar
- R. P. Weicker. Dhrystone: A synthetic systems programming benchmark. Communications of the ACM, 27(10):1013--1030, Oct. 1984. Google Scholar
Digital Library
- M. Weiser. Program slicing. IEEE Transaction on Software Engineering, 10(4):352--357, July 1984.Google Scholar
Digital Library
- R. E.Wunderlich, T. F.Wenisch, B. Falsafi, and J. C. Hoe. SMARTS: Accelerating microarchitecture simulation via rigorous statistical sampling. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 84--95, June 2003. Google Scholar
Digital Library
- M. T. Yourst. PTLsim: A cycle accurate full system x86-64 microarchitectural simulator. In Proceedings of the International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 23--34, Apr. 2007.Google Scholar
Cross Ref
- X. Zhang, R. Gupta, and Y. Zhang. Cost and precision tradeoffs of dynamic data slicing algorithms. ACM Transactions on Programming Languages and Systems, 27(4):631--661, July 2005. Google Scholar
Digital Library
- C. B. Zilles and G. S. Sohi. Understanding the backward slices of performance degrading instructions. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA), pages 172--181, June 2000. Google Scholar
Digital Library
Index Terms
Dispersing proprietary applications as benchmarks through code mutation
Recommendations
Dispersing proprietary applications as benchmarks through code mutation
ASPLOS '08Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not ...
Dispersing proprietary applications as benchmarks through code mutation
ASPLOS '08Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not ...
Dispersing proprietary applications as benchmarks through code mutation
ASPLOS XIII: Proceedings of the 13th international conference on Architectural support for programming languages and operating systemsIndustry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not ...







Comments