Abstract
Remote data access latency is a significant performance bottleneck in many modern programs that use remote databases and web services. We present Sprint - a run-time system for optimizing such programs by prefetching and caching data from remote sources in parallel to the execution of the original program. Sprint separates the concerns of exposing potentially-independent data accesses from the mechanism for executing them efficiently in parallel or in a batch. In contrast to prior work, Sprint can efficiently prefetch data in the presence of irregular or input-dependent access patterns, while preserving the semantics of the original program.
We used Sprint to automatically improve the performance of several real-world Java programs that access remote databases (MySQL, DB2) and web services (Facebook, IBM's Yellow Pages). Sprint achieves speedups ranging 2.4x to 15.8x over sequential execution, which are comparable to those achieved by manually modifying the program for asynchronous and batch execution of data accesses. Sprint provides a simple interface that allows a programmer to plug in support for additional data sources without modifying the client program.
- M. Allen. RestFB: Facebook Java api. http://restfb.com.Google Scholar
- P. Bogle and B. Liskov. Reducing cross domain call overhead using batched futures. In Proc. of OOPSLA '94. Google Scholar
Digital Library
- E. Bruneton, R. Lenglet, and T. Coupaye. ASM: A code manipulation tool to implement adaptable systems. In Adaptable and Extensible Component Systems, 2002.Google Scholar
- F. Chang and G. A. Gibson. Automatic I/O hint generation through speculative execution. In Proc. of OSDI '99. Google Scholar
Digital Library
- Y. Chen, S. Byna, X.-H. Sun, R. Thakur, and W. Gropp. Hiding I/O latency with pre-execution prefetching for parallel applications. In Proc. of SC '08. Google Scholar
Digital Library
- J. D. Collins, D. M. Tullsen, H. Wang, and J. P. Shen. Dynamic speculative precomputation. In Proc. of MICRO '01. Google Scholar
Digital Library
- R. Cooksey, S. Jourdan, and D. Grunwald. A stateless, content-directed data prefetching mechanism. In Proc. of ASPLOS '02. Google Scholar
Digital Library
- K. M. Curewitz, P. Krishnan, and J. S. Vitter. Practical prefetching via data compression. In Proc. of SIGMOD '93. Google Scholar
Digital Library
- C. Ding, X. Shen, K. Kelsey, C. Tice, R. Huang, and C. Zhang. Software behavior oriented parallelization. In Proc. of PLDI '07. Google Scholar
Digital Library
- D. Duchamp. Prefetching hyperlinks. In Proc. of USENIX '99. Google Scholar
Digital Library
- A. N. Eden, B. W. Joh, and T. Mudge. Web latency reduction via client-side prefetching. In Proc. of ISPASS '00. Google Scholar
Digital Library
- C. Flanagan and S. N. Freund. The RoadRunner dynamic analysis framework for concurrent programs. In Proc. of PASTE '10. Google Scholar
Digital Library
- S. Horwitz, T. Reps, and D. Binkley. Interprocedural slicing using dependence graphs. In Proc. of PLDI '88. Google Scholar
Digital Library
- A. Ibrahim, M. F. II, W. R. Cook, and E. Tilevich. Remote batch invocation for web services: Document-oriented web services with object-oriented interfaces. In Proc. of ECOWS '09. Google Scholar
Digital Library
- A. Ibrahim, Y. Jiao, E. Tilevich, and W. R. Cook. Remote batch invocation for compositional object services. In Proc. of ECOOP '06. Google Scholar
Digital Library
- D. Kim and D. Yeung. Design and evaluation of compiler algorithms for pre-execution. In Proc. of ASPLOS '02. Google Scholar
Digital Library
- R. Koller and R. Rangaswami. I/O deduplication: Utilizing content similarity to improve I/O performance. ACM Transactions on Storage (TOS), 2010. Google Scholar
Digital Library
- D. Kotz and C. S. Ellis. Practical prefetching techniques for parallel file systems. In Proc. of PDIS '91. Google Scholar
Digital Library
- M. Kulkarni, K. Pingali, B. Walter, G. Ramanarayanan, K. Bala, and L. P. Chew. Optimistic parallelism requires abstractions. In Proc. of PLDI '07. Google Scholar
Digital Library
- J. Lee, C. Jung, D. Lim, and Y. Solihin. Prefetching with helper threads for loosely coupled multiprocessor systems. IEEE Transactions on Parallel and Distributed Systems, 2009. Google Scholar
Digital Library
- C.-K. Luk. Tolerating memory latency through software-controlled pre-execution in simultaneous multithreading processors. SIGARCH Computer Architecture News, 2001. Google Scholar
Digital Library
- J. Mickens, J. Elson, J. Howell, and J. Lorch. Crom: Faster web browsing using speculative execution. In Proc. of NSDI '10. Google Scholar
Digital Library
- T. C. Mowry, A. K. Demke, and O. Krieger. Automatic compiler-inserted I/O prefetching for out-of-core applications. In Proc. of OSDI '96. Google Scholar
Digital Library
- Y. Ni, V. S. Menon, A.-R. Adl-Tabatabai, A. L. Hosking, R. L. Hudson, J. E. B. Moss, B. Saha, and T. Shpeisman. Open nesting in software transactional memory. In Proc. of PPoPP '07. Google Scholar
Digital Library
- V. N. Padmanabhan and J. C. Mogul. Using predictive prefetching to improve world wide web latency. ACM SIGCOMM Computer Communication Review, 1996. Google Scholar
Digital Library
- R. H. Patterson and G. A. Gibson. Exposing I/O concurrency with informed prefetching. In Proc. of PDIS '94. Google Scholar
Digital Library
- E. Raman, N. Vachharajani, R. Rangan, and D. I. August. Spice: speculative parallel iteration chunk execution. In Proc. of CGO '08. Google Scholar
Digital Library
- U. Srivastava, K. Munagala, J. Widom, and R. Motwani. Query optimization over web services. In Proc. of VLDB '06. Google Scholar
Digital Library
- J. M. Tamayo. Pepe JDBC dependency tracker. http://github.com/jtamayo/pepe.Google Scholar
- F. Tip. A survey of program slicing techniques. Technical report, CWI, Amsterdam, The Netherlands, The Netherlands, 1994. Google Scholar
Digital Library
- K. S. Trivedi. On the paging performance of array algorithms. IEEE Transactions on Computers, 1977. Google Scholar
Digital Library
- P. H. Wang, J. D. Collins, H. Wang, D. Kim, B. Greene, K. ming Chan, A. B. Yunus, T. Sych, S. F. Moore, and J. P. Shen. Helper threads via virtual multithreading on an experimental Itanium 2 processor-based platform. In Proc. of ASPLOS '04. Google Scholar
Digital Library
- A. Welc, S. Jagannathan, and A. Hosking. Safe Futures for Java. In Proc. of OOPSLA '05. Google Scholar
Digital Library
- C.-K. Yang, T. Mitra, and T.-c. Chiueh. A decoupled architecture for application-specific file prefetching. In Proc. of FREENIX Track: USENIX '02. Google Scholar
Digital Library
- W. Zhang, D. M. Tullsen, and B. Calder. Accelerating and adapting precomputation threads for efficient prefetching. In Proc. of HPCA '07. Google Scholar
Digital Library
Index Terms
Sprint: speculative prefetching of remote data
Recommendations
Sprint: speculative prefetching of remote data
OOPSLA '11: Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applicationsRemote data access latency is a significant performance bottleneck in many modern programs that use remote databases and web services. We present Sprint - a run-time system for optimizing such programs by prefetching and caching data from remote sources ...
Reducing memory latency using a small software driven array cache
HICSS '95: Proceedings of the 28th Hawaii International Conference on System SciencesFrom the programming viewpoint, data references can be classified into two types: array reference and non-array references. Array references have relatively strong spatial locality while non-array references have relatively strong temporal locality. ...
Criticality aware tiered cache hierarchy: a fundamental relook at multi-level cache hierarchies
ISCA '18: Proceedings of the 45th Annual International Symposium on Computer ArchitectureOn-die caches are a popular method to help hide the main memory latency. However, it is difficult to build large caches without substantially increasing their access latency, which in turn hurts performance. To overcome this difficulty, on-die caches ...







Comments