Abstract
Deterministic replay systems record and reproduce the execution of a hardware or software system. In contrast to replaying execution on uniprocessors, deterministic replay on multiprocessors is very challenging to implement efficiently because of the need to reproduce the order or values read by shared memory operations performed by multiple threads. In this paper, we present DoublePlay, a new way to efficiently guarantee replay on commodity multiprocessors. Our key insight is that one can use the simpler and faster mechanisms of single-processor record and replay, yet still achieve the scalability offered by multiple cores, by using an additional execution to parallelize the record and replay of an application. DoublePlay timeslices multiple threads on a single processor, then runs multiple time intervals (epochs) of the program concurrently on separate processors. This strategy, which we call uniparallelism, makes logging much easier because each epoch runs on a single processor (so threads in an epoch never simultaneously access the same memory) and different epochs operate on different copies of the memory. Thus, rather than logging the order of shared-memory accesses, we need only log the order in which threads in an epoch are timesliced on the processor. DoublePlay runs an additional execution of the program on multiple processors to generate checkpoints so that epochs run in parallel. We evaluate DoublePlay on a variety of client, server, and scientific parallel benchmarks; with spare cores, DoublePlay reduces logging overhead to an average of 15% with two worker threads and 28% with four threads.
- ALTEKAR, G., AND STOICA, I. ODR: Output-deterministic replay for multicore debugging. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (October 2009), pp. 193--206. Google Scholar
Digital Library
- BACON, D. F., AND GOLDSTEIN, S. C. Hardware assisted replay of multiprocessor programs. In Proceedings of the 1991 ACM/ONR Workshop on Parallel and Distributed Debugging (1991), ACM Press, pp. 194--206. Google Scholar
Digital Library
- BERGER, E. D., YANG, T., LIU, T., AND NOVARK, G. Grace: safe multithreaded programming for c/c++. In Proceeding of the 24th ACM SIGPLAN conference on Object oriented programming systems languages and applications (Orlando, Florida, USA, 2009), pp. 81--96. Google Scholar
Digital Library
- BHANSALI, S., CHEN, W., DE JONG, S., EDWARDS, A., AND DRINIC, M. Framework for instruction-level tracing and analysis of programs. In Second International Conference on Virtual Execution Environments (June 2006), pp. 154--163. Google Scholar
Digital Library
- BRESSOUD, T. C., AND SCHNEIDER, F. B. Hypervisor-based fault tolerance. ACM Transactions on Computer Systems 14, 1 (February 1996), 80--107. Google Scholar
Digital Library
- CHOI, J. D., ALPERN, B., NGO, T., AND SRIDHARAN, M. A perturbation free replay platform for cross-optimized multithreaded applications. In Proceedings of the 15th International Parallel and Distributed Processing Symposium (April 2001). Google Scholar
Digital Library
- CHOW, J., GARFINKEL, T., AND CHEN, P. M. Decoupling dynamic program analysis from execution in virtual environments. In Proceedings of the 2008 USENIX Technical Conference (June 2008), pp. 1--14. Google Scholar
Digital Library
- DEVIETTI, J., LUCIA, B., CEZE, L., AND OSKIN, M. DMP: Deterministic shared memory multiprocessing. In Proceedings of the 2009 International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (March 2009), pp. 85--96. Google Scholar
Digital Library
- DUNLAP, G. W., KING, S. T., CINAR, S., BASRAI, M. A., AND CHEN, P. M. ReVirt: Enabling intrusion analysis through virtualmachine logging and replay. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (Boston, MA, December 2002), pp. 211--224. Google Scholar
Digital Library
- DUNLAP, G. W., LUCCHETTI, D. G., FETTERMAN, M., AND CHEN, P. M. Execution replay on multiprocessor virtual machines. In Proceedings of the 2008 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE) (March 2008), pp. 121--130. Google Scholar
Digital Library
- FELDMAN, S. I., AND BROWN, C. B. IGOR: A system for program debugging via reversible execution. In PADD '88: Proceedings of the 1988 ACM SIGPLAN and SIGOPS Workshop on Parallel and Distributed Debugging (1988), pp. 112--123. Google Scholar
Digital Library
- HOWER, D. R., AND HILL, M. D. Rerun: Exploiting episodes for lightweight memory race recording. In Proceedings of the 2008 International Symposium on Computer Architecture (June 2008), pp. 265--276. Google Scholar
Digital Library
- JR., R. L. B., ADVE, V. S., DIG, D., ADVE, S. V., HEUMANN, S., KOMURAVELLI, R., OVERBEY, J., SIMMONS, P., SUNG, H., AND VAKILIAN, M. A type and effect system for deterministic parallel java. In OOPSLA (2009), pp. 97--116. Google Scholar
Digital Library
- KELSEY, K., BAI, T., DING, C., AND ZHANG, C. Fast Track: A software system for speculative program optimization. In Proceedings of the 2009 International Symposium on Code Generation and Optimization (CGO) (March 2009), pp. 157--168. Google Scholar
Digital Library
- KING, S. T., DUNLAP, G. W., AND CHEN, P. M. Debugging operating systems with time-traveling virtual machines. In Proceedings of the 2005 USENIX Technical Conference (April 2005), pp. 1--15. Google Scholar
Digital Library
- LEBLANC, T. J., AND MELLOR-CRUMMEY, J. M. Debugging parallel programs with instant replay. IEEE Transaction on Computers 36, 4 (1987), 471--482. Google Scholar
Digital Library
- LEE, D., SAID, M., NARAYANASAMY, S., YANG, Z. J., AND PEREIRA, C. Offline symbolic analysis for multi-processor execution replay. In International Symposium on Microarchitecture (MICRO) (2009). Google Scholar
Digital Library
- LEE, D., WESTER, B., VEERARAGHAVAN, K., CHEN, P. M., FLINN, J., AND NARAYANASAMY, S. Respec: Efficient online multiprocessor replay via speculation and external determinism. In Proceedings of the 15th International Conference on Architectural Support for Programming Languages and Operating Systems (Pittsburgh, PA, March 2010), pp. 77--89. Google Scholar
Digital Library
- MELLOR-CRUMMEY, J. M., AND LEBLANC, T. J. A Software Instruction Counter. In Proceedings of the 1989 International Conference on Architectural Support for Programming Languages and Operating Systems (April 1989), pp. 78--86. Google Scholar
Digital Library
- MONTESINOS, P., CEZE, L., AND TORRELLAS, J. DeLorean: Recording and deterministically replaying shared-memory multiprocessor execution efficiently. In Proceedings of the 2008 International Symposium on Computer Architecture (June 2008), pp. 289--300. Google Scholar
Digital Library
- NARAYANASAMY, S., PEREIRA, C., AND CALDER, B. Recording shared memory dependencies using Strata. In ASPLOS-XII: Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (2006), pp. 229--240. Google Scholar
Digital Library
- NARAYANASAMY, S., PEREIRA, C., PATIL, H., COHN, R., AND CALDER, B. Automatic logging of operating system effects to guide application-level architecture simulation. In International Conference on Measurements and Modeling of Computer Systems (SIGMETRICS) (June 2006), pp. 216--227. Google Scholar
Digital Library
- NARAYANASAMY, S., POKAM, G., AND CALDER, B. BugNet: Continuously recording program execution for deterministic replay debugging. In Proceedings of the 32nd Annual International Symposium on Computer Architecture (ISCA) (June 2005), pp. 284--295. Google Scholar
Digital Library
- NETZER, R. H. B. Optimal tracing and replay for debugging sharedmemory parallel programs. In Proceedings of the ACM/ONR Workshop on Parallel and Distributed Debugging (1993), pp. 1--11. Google Scholar
Digital Library
- NIGHTINGALE, E. B., CHEN, P. M., AND FLINN, J. Speculative execution in a distributed file system. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (Brighton, United Kingdom, October 2005), pp. 191--205. Google Scholar
Digital Library
- NIGHTINGALE, E. B., PEEK, D., CHEN, P. M., AND FLINN, J. Parallelizing security checks on commodity hardware. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems (Seattle,WA,March 2008), pp. 308--318. Google Scholar
Digital Library
- NIGHTINGALE, E. B., VEERARAGHAVAN, K., CHEN, P. M., AND FLINN, J. Rethink the sync. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (Seattle, WA, October 2006), pp. 1--14. Google Scholar
Digital Library
- OLSZEWSKI, M., ANSEL, J., AND AMARASINGHE, S. Kendo: efficient deterministic multithreading in software. In Proceedings of the 2009 International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (March 2009), pp. 97--108. Google Scholar
Digital Library
- OPLINGER, J., AND LAM, M. S. Enhancing software reliability using speculative threads. In Proceedings of the 2002 InternationalConference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (October 2002), pp. 184--196. Google Scholar
Digital Library
- PARK, S., ZHOU, Y., XIONG, W., YIN, Z., KAUSHIK, R., LEE, K. H., AND LU, S. PRES: Probabilistic replay with execution sketching on multiprocessors. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (October 2009), pp. 177--191. Google Scholar
Digital Library
- PETERSON, Z. N. J., AND BURNS, R. Ext3cow: A time-shifting file system for regulatory compliance. ACM Transacations on Storage 1, 2 (2005), 190--212. Google Scholar
Digital Library
- RONSSE, M., AND BOSSCHERE, K. D. RecPlay: A full integrated practical record/replay system. ACM Transactions on Computer Systems 17, 2 (May 1999), 133--152. Google Scholar
Digital Library
- RUSSINOVICH, M., AND COGSWELL, B. Replay for concurrent nondeterministic shared-memory applications. In Proceedings of the ACM SIGPLAN 1996 Conference on Programming Language Design and Implementation (1996), pp. 258--266. Google Scholar
Digital Library
- SANTRY, D. S., FEELEY, M. J., HUTCHINSON, N. C., VEITCH, A. C., CARTON, R. W., AND OFIR, J. Deciding when to forget in the Elephant file system. SIGOPS Operating Systems Review 33, 5 (1999), 110--123. Google Scholar
Digital Library
- SOHI, G. S., BREACH, S. E., AND VIJAYKUMAR, T. N. Multiscalar processors. In Proceedings of the 1995 International Symposium on Computer Architecture (June 1995), pp. 414--425. Google Scholar
Digital Library
- SRINIVASAN, S., ANDREWS, C., KANDULA, S., AND ZHOU, Y. Flashback: A light-weight extension for rollback and deterministic replay for software debugging. In Proceedings of the 2004 USENIX Technical Conference (Boston, MA, June 2004), pp. 29--44. Google Scholar
Digital Library
- STEFFAN, J. G., AND MOWRY, T. C. The potential for using threadlevel data speculation to facilitate automatic parallelization. In Proceedings of the 1998 Symposium on High Performance Computer Architecture (February 1998), pp. 2--13. Google Scholar
Digital Library
- SUNDARAMOORTHY, K., PURSER, Z., AND ROTENBERG, E. Slipstream processors: improving both performance and fault tolerance. In Proceedings of the 9th International Conference on Architectural Support for Programming Languages and Operating Systems (November 2000), pp. 257--268. Google Scholar
Digital Library
- SÜ SSKRAUT, M., KNAUTH, T., WEIGERT, S., SCHIFFEL, U., MEINHOLD, M., FETZER, C., BAI, T., DING, C., AND ZHANG, C. Prospect: A compiler framework for speculative parallelization. In Proceedings of the 2010 International Symposium on Code Generation and Optimization (CGO) (April 2010), pp. 131--140. Google Scholar
Digital Library
- TUCEK, J., LU, S., HUANG, C., XANTHOS, S., AND ZHOU, Y. Triage: Diagnosing production run failures at the user's site. In Proceedings of the 21st ACMSymposium on Operating Systems Principles (October 2007), pp. 131--144. Google Scholar
Digital Library
- VEERARAGHAVAN, K., FLINN, J., NIGHTINGALE, E. B., AND NOBLE, B. quFiles: The right file at the right time. In Proceedings of the 8th USENIX Conference on File and Storage Technologies (San Jose, CA, February 2010), pp. 1--14. Google Scholar
Digital Library
- VLACHOS, E., GOODSTEIN, M. L., KOZUCH, M. A., CHEN, S., FALSAFI, B., GIBBONS, P. B., AND MOWRY, T. C. ParaLog: Enabling and accelerating online parallel monitoring of multithreaded applications. In Proceedings of the 15th International Conference on Architectural Support for Programming Languages and Operating Systems (Pittsburgh, PA, March 2010), pp. 271--284. Google Scholar
Digital Library
- WEERATUNGE, D., ZHANG, X., AND JAGANNATHAN, S. Analyzing multicore dumps to facilitate concurrency bug reproduction. In Proceedings of the 2010 International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (March 2010), pp. 155--166. Google Scholar
Digital Library
- WOO, S. C., OHARA, M., TORRIE, E., SINGH, J. P., AND GUPTA, A. The SPLASH-2 programs: Characterization and methodological considerations. In Proceedings of the 22nd International Symposium on Computer Architecture (June 1995), pp. 24--36. Google Scholar
Digital Library
- XU, M., BODIK, R., AND HILL, M. D. A "flight data recorder" for enabling full-system multiprocessor deterministic replay. In Proceedings of the 2003 International Symposium on Computer Architecture (June 2003), pp. 122--135. Google Scholar
Digital Library
- XU, M., MALYUGIN, V., SHELDON, J., VENKITACHALAM, G., AND WEISSMAN, B. ReTrace: Collecting execution trace with virtual machine deterministic replay. In Proceedings of the 2007 Workshop on Modeling, Benchmarking and Simulation (MoBS) (June 2007).Google Scholar
- ZAMFIR, C., AND CANDEA, G. Execution synthesis: A technique for automated software debugging. In Proceedings of the 2010 European Conference on Computer Systems (EuroSys) (April 2010), pp. 321--334. Google Scholar
Digital Library
- ZILLES, C., AND SOHI, G. Master/slave speculative parallelization. In Proceedings of the 35th Annual ACM/IEEE International Symposium on Microarchitecture (MICRO) (2002), pp. 85--96. Google Scholar
Digital Library
Index Terms
DoublePlay: parallelizing sequential logging and replay
Recommendations
DoublePlay: parallelizing sequential logging and replay
ASPLOS XVI: Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systemsDeterministic replay systems record and reproduce the execution of a hardware or software system. In contrast to replaying execution on uniprocessors, deterministic replay on multiprocessors is very challenging to implement efficiently because of the ...
DoublePlay: Parallelizing Sequential Logging and Replay
Special Issue APLOS 2011Deterministic replay systems record and reproduce the execution of a hardware or software system. In contrast to replaying execution on uniprocessors, deterministic replay on multiprocessors is very challenging to implement efficiently because of the ...
DoublePlay: parallelizing sequential logging and replay
ASPLOS '11Deterministic replay systems record and reproduce the execution of a hardware or software system. In contrast to replaying execution on uniprocessors, deterministic replay on multiprocessors is very challenging to implement efficiently because of the ...







Comments