Abstract
Parallel streaming computations have been studied extensively, and many languages, libraries, and systems have been designed to support this model of computation. In particular, we consider acyclic streaming computations in which individual nodes can choose to filter, or discard, some of their inputs in a data-dependent manner. In these applications, if the channels between nodes have finite buffers, the computation can deadlock. One method of deadlock avoidance is to augment the data streams between nodes with occasional dummy messages; however, for general DAG topologies, no polynomial time algorithm is known to compute the intervals at which dummy messages must be sent to avoid deadlock.
In this paper, we show that deadlock avoidance for streaming computations with filtering can be performed efficiently for a large class of DAG topologies. We first present a new method where each dummy message is tagged with a destination, so as to reduce the number of dummy messages sent over the network. We then give efficient algorithms for dummy interval computation in series-parallel DAGs. We finally generalize our results to a larger graph family, which we call the CS4 DAGs, in which every undirected Cycle is Single-Source and Single-Sink (CS4). Our results show that, for a large set of application topologies that are both intuitively useful and formalizable, the streaming model with filtering can be implemented safely with reasonable overhead.
- I. Buck, T. Foley, D. Horn, J. Sugerman, K. Fatahalian, M. Houston, and P. Hanrahan. Brook for GPUs: Stream computing on graphics hardware. ACM Trans. Graphics, 23 (3): 777--786, 2004. Google Scholar
Digital Library
- J. T. Buck. Static scheduling and code generation from dynamic dataflow graphs with integer-valued control streams. In Asilomar Conf. on Signals, Systems, and Computers, pages 508--513, Nov. 1994.Google Scholar
Cross Ref
- J. Buhler, J. M. Lancaster, A. C. Jacob, and R. D. Chamberlain. Mercury BLASTN: Faster DNA sequence comparison using a streaming hardware architecture. In Proc. Reconfigurable Systems Summer Institute, Urbana, IL, July 2007.Google Scholar
- R. J. Duffin. Topology of series-parallel networks. Journal of Mathematical Analysis and Applications, 10: 303--318, 1965.Google Scholar
Cross Ref
- M. A. Franklin, E. J. Tyson, J. H. Buckley, P. Crowley, and J. Maschmeyer. Auto-pipe and the X language: A pipeline design tool and description language. In IEEE Int'l Parallel and Distributed Processing Symp., Apr. 2006. Google Scholar
Digital Library
- M. M. Gaber, A. Zaslavsky, and S. Krishnaswamy. Mining data streams: a review. SIGMOD Rec., 34 (2): 18--26, 2005. Google Scholar
Digital Library
- B. Khailany, W. Dally, S. Rixner, U. Kapasi, P. Mattson, J. Namkoong, J. Owens, B. Towles, and A. Chang. Imagine: Media processing with streams. IEEE Micro, pages 35--46, March/April 2001. Google Scholar
Digital Library
- E. A. Lee. Consistency in dataflow graphs. IEEE Trans. on Parallel and Distributed Systems, 2 (2): 223--235, Apr. 1991. Google Scholar
Digital Library
- E. A. Lee and D. G. Messerschmitt. Synchronous data flow. Proceedings of the IEEE, 75 (9): 1235--1245, Sept. 1987.Google Scholar
Cross Ref
- P. Li, K. Agrawal, J. Buhler, and R. D. Chamberlain. Deadlock avoidance for streaming computations with filtering. In ACM Symp. on Parallelism in Algorithms and Architectures, 2010. Google Scholar
Digital Library
- Li, Agrawal, Buhler, Chamberlain, and Lancaster}ASAP10P. Li, K. Agrawal, J. Buhler, R. D. Chamberlain, and J. M. Lancaster. Deadlock-avoidance for streaming applications with split-join structure: Two case studies. In IEEE Int'l Conf. on Application-specific Systems, Architectures and Processors, pages 333--336, July 2010.Google Scholar
- Y. Liu, N. Vijayakumar, and B. Plale. Stream processing in data-driven computational science. In IEEE/ACM Int'l Conf. on Grid Computing, pages 160--167, 2006. Google Scholar
Digital Library
- W. R. Mark, R. S. Glanville, K. Akeley, and M. J. Kilgard. Cg: a system for programming graphics hardware in a C-like language. ACM Trans. on Graphics, 22 (3): 896--907, July 2003. Google Scholar
Digital Library
- J. Misra. Distributed discrete-event simulation. ACM Comput. Surv., 18 (1): 39--65, 1986. Google Scholar
Digital Library
- J. W. Romein, P. C. Broekema, E. van Meijeren, K. van der Schaaf, and W. H. Zwart. Astronomical real-time streaming signal processing on a Blue Gene/L supercomputer. In ACM Symp. on Parallelism in Algorithms and Architectures, pages 59--66, 2006. Google Scholar
Digital Library
- W. Thies and S. Amarasinghe. An empirical characterization of stream programs and its implications for language and compiler design. In Int'l Conf. on Parallel Architectures and Compilation Techniques, pages 365--376, 2010. Google Scholar
Digital Library
- W. Thies, M. Karczmarek, and S. Amarasinghe. StreamIt: A language for streaming applications. In Int'l Conf. on Compiler Construction, pages 179--196, 2002. Google Scholar
Digital Library
- J. Valdes, R. E. Tarjan, and E. L. Lawler. The recognition of series parallel digraphs. In ACM Symposium on Theory of Computing, 1979. Google Scholar
Digital Library
Index Terms
Efficient deadlock avoidance for streaming computation with filtering
Recommendations
Efficient deadlock avoidance for streaming computation with filtering
PPoPP '12: Proceedings of the 17th ACM SIGPLAN symposium on Principles and Practice of Parallel ProgrammingParallel streaming computations have been studied extensively, and many languages, libraries, and systems have been designed to support this model of computation. In particular, we consider acyclic streaming computations in which individual nodes can ...
Efficient distributed deadlock avoidance with liveness guarantees
EMSOFT '06: Proceedings of the 6th ACM & IEEE International conference on Embedded softwareWe present a deadlock avoidance algorithm for distributed systems that guarantees liveness. Deadlock avoidance in distributed systems is a hard problem and general solutions are considered impractical due to the high communication overhead. In previous ...
Automatic repair for multi-threaded programs with Deadlock/Livelock using maximum satisfiability
ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and AnalysisDeadlock-freedom is a major challenge in developing multi-threaded programs, as a deadlock cannot be resolved until one restarts the program (mostly by using manual intervention). To avoid the risk of blocking, a program may use the trylock operations ...







Comments