Abstract
We present the first language design to uniformly express variants of n-way joins over asynchronous event streams from different domains, e.g., stream-relational algebra, event processing, reactive and concurrent programming. We model asynchronous reactive programs and joins in direct style, on top of algebraic effects and handlers. Effect handlers act as modular interpreters of event notifications, enabling fine-grained control abstractions and customizable event matching. Join variants can be considered as cartesian product computations with ”degenerate” control flow, such that unnecessary tuples are not materialized a priori. Based on this computational interpretation, we decompose joins into a generic, naive enumeration procedure of the cartesian product, plus variant-specific extensions, represented in terms of user-supplied effect handlers. Our microbenchmarks validate that this extensible design avoids needless materialization. Alongside a formal semantics for joining and prototypes in Koka and multicore OCaml, we contribute a systematic comparison of the covered domains and features.
- Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil Immerman. 2008. Efficient Pattern Matching over Event Streams. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google Scholar
Digital Library
- Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernández-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, Frances Perry, Eric Schmidt, and Sam Whittle. 2015. The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-scale, Unbounded, Out-of-order Data Processing. Proceedings of the VLDB Endowment 12 (2015), 1792–1803. Google Scholar
Digital Library
- Arvind Arasu, Shivnath Babu, and Jennifer Widom. 2004. CQL: A Language for Continuous Queries over Streams and Relations. In Database Programming Languages. Springer Berlin Heidelberg.Google Scholar
- Arvind Arasu, Shivnath Babu, and Jennifer Widom. 2006. The CQL Continuous Query Language: Semantic Foundations and Query Execution. The VLDB Journal (2006). Google Scholar
Digital Library
- Arvind Arasu and Jennifer Widom. 2004. A Denotational Semantics for Continuous Queries over Streams and Relations. SIGMOD Record (2004). Google Scholar
Digital Library
- Engineer Bainomugisha, Andoni Lombide Carreton, Tom Van Cutsem, Stijn Mostinckx, and Wolfgang De Meuter. 2013. A survey on reactive programming. Comput. Surveys 45, 4 (2013). Google Scholar
Digital Library
- Andrej Bauer and Matija Pretnar. 2015. Programming with Algebraic Effects and Handlers. Journal of Logical and Algebraic Methods in Programming 84, 1 (2015), 108–123.Google Scholar
Cross Ref
- Nick Benton, Luca Cardelli, and Cédric Fournet. 2004. Modern Concurrency Abstractions for C#. Transactions on Programming Languages and Systems (TOPLAS) 26, 5 (2004), 769–804. Google Scholar
Digital Library
- Aggelos Biboudis, Nick Palladinos, George Fourtounis, and Yannis Smaragdakis. 2015. Streams à la carte: Extensible Pipelines with Object Algebras. In Proceedings of European Conference on Object-Oriented Programming (ECOOP).Google Scholar
- Gavin Bierman, Claudio Russo, Geoffrey Mainland, Erik Meijer, and Mads Torgersen. 2012. Pause ’n’ Play: Formalizing Asynchronous C♯. In Proceedings of European Conference on Object-Oriented Programming (ECOOP), James Noble (Ed.). Google Scholar
Digital Library
- Dariusz Biernacki, Maciej Piróg, Piotr Polesiuk, and Filip Sieczkowski. 2018. Handle with care: relational interpretation of algebraic effects and handlers. PACMPL POPL (2018). Google Scholar
Digital Library
- Irina Botan, Roozbeh Derakhshan, Nihal Dindar, Laura Haas, Renée J. Miller, and Nesime Tatbul. 2010. SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems. Proceedings of the VLDB Endowment (2010). Google Scholar
Digital Library
- Andrew Cave, Francisco Ferreira, Prakash Panangaden, and Brigitte Pientka. 2014. Fair Reactive Programming. In Proceedings of Symposium on Principles of Programming Languages (POPL) (POPL ’14). ACM, 361–372. Google Scholar
Digital Library
- Sharma Chakravarthy, V. Krishnaprasad, Eman Anwar, and S.-K. Kim. 1994. Composite Events for Active Databases: Semantics, Contexts and Detection. In Proceedings of International Conference on Very Large Data Bases (VLDB). Google Scholar
Digital Library
- S. Chakravarthy and D. Mishra. 1994. Snoop: An expressive event specification language for active databases. Data & Knowledge Engineering 14, 1 (1994), 1 – 26. Google Scholar
Digital Library
- James Cheney, Sam Lindley, and Philip Wadler. 2013. A practical theory of language-integrated query. In Proceedings of International Conference on Functional Programming (ICFP). Google Scholar
Digital Library
- Silvain Conchon and Fabrice Le Fessant. 1999. JoCaml: mobile agents for Objective-Caml. In First and Third International Symposium on Agent Systems Applications, and Mobile Agents (ASAMA). Google Scholar
Digital Library
- Gregory H. Cooper and Shriram Krishnamurthi. 2006. Embedding Dynamic Dataflow in a Call-by-Value Language. In Proceedings of the European Conference on Programming Languages and Systems (ESOP). Google Scholar
Digital Library
- Gianpaolo Cugola and Alessandro Margara. 2010. TESLA: A Formally Defined Event Specification Language. In Proceedings of the International Conference on Distributed Event-based Systems (DEBS). Google Scholar
Digital Library
- Gianpaolo Cugola and Alessandro Margara. 2012. Processing Flows of Information: From Data Stream to Complex Event Processing. Comput. Surveys 44, 3 (2012), 15:1–15:62. Google Scholar
Digital Library
- Evan Czaplicki and Stephen Chong. 2013. Asynchronous Functional Reactive Programming for GUIs. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). Google Scholar
Digital Library
- Ana Lúcia de Moura and Roberto Ierusalimschy. 2009. Revisiting coroutines. ACM Trans. Program. Lang. Syst. 31, 2 (2009). Google Scholar
Digital Library
- Alan Demers, Johannes Gehrke, Mingsheng Hong, Mirek Riedewald, and Walker White. 2006. Towards Expressive Publish/Subscribe Systems. In Proceedings of the 10th International Conference on Advances in Database Technology (EDBT’06). Google Scholar
Digital Library
- Yanlei Diao, Neil Immerman, and Daniel Gyllstrom. 2007. SASE+: An Agile Language for Kleene Closure Over Event Streams. Technical Report. UMass Technical Report 07.Google Scholar
- Nihal Dindar, Nesime Tatbul, Renée J. Miller, Laura M. Haas, and Irina Botan. 2013. Modeling the execution semantics of stream processing engines with SECRET. The VLDB Journal (2013). Google Scholar
Digital Library
- Stephen Dolan, Spiros Eliopoulos, Daniel Hillerström, Anil Madhavapeddy, KC Sivaramakrishnan, and Leo White. 2017. Concurrent System Programming with Effect Handlers. In Proceedings of the Symposium on Trends in Functional Programming.Google Scholar
- Jonathan Edwards. 2009. Coherent Reaction. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). Google Scholar
Digital Library
- Conal M. Elliott. 2009. Push-pull Functional Reactive Programming. In Proceedings of the 2nd ACM SIGPLAN Symposium on Haskell (Haskell’09). Google Scholar
Digital Library
- Esper. {n. d.}. Esper. http://www.espertech.com/esper/ .Google Scholar
- Patrick Eugster and K.R. Jayaram. 2009. EventJava: An Extension of Java for Event Correlation. In Proceedings of European Conference on Object-Oriented Programming (ECOOP). Google Scholar
Digital Library
- Matthias Felleisen and Robert Hieb. 1992. The Revised Report on the Syntactic Theories of Sequential Control and State. Theoretical Computer Science 103, 2 (1992), 235–271. Google Scholar
Digital Library
- Matthew Fluet, Mike Rainey, John H. Reppy, and Adam Shaw. 2008. Implicitly-threaded parallelism in Manticore. In Proceedings of International Conference on Functional Programming (ICFP). Google Scholar
Digital Library
- Yannick Forster, Ohad Kammar, Sam Lindley, and Matija Pretnar. 2017. On the Expressive Power of User-defined Effects: Effect Handlers, Monadic Reflection, Delimited Control. Proc. ACM Program. Lang. 1, ICFP (2017). Google Scholar
Digital Library
- Cédric Fournet and Georges Gonthier. 1996. The reflexive CHAM and the Join-calculus. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Philipp Haller and Heather Miller. 2013. RAY: Integrating Rx and Async for Direct-Style Reactive Streams. (2013).Google Scholar
- Martin Hirzel, Robert Soulé, Scott Schneider, Bugra Gedik, and Robert Grimm. 2013. A catalog of stream processing optimizations. Comput. Surveys (2013). Google Scholar
Digital Library
- John E Hopcroft, Rajeev Motwani, and Jeffrey D Ullman. 2006. Automata theory, languages, and computation. Pearson. Google Scholar
Digital Library
- Ohad Kammar, Sam Lindley, and Nicolas Oury. 2013. Handlers in action. In Proceedings of International Conference on Functional Programming (ICFP). Google Scholar
Digital Library
- Oleg Kiselyov, Aggelos Biboudis, Nick Palladinos, and Yannis Smaragdakis. 2017. Stream Fusion, to Completeness. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Oleg Kiselyov and Hiromi Ishii. 2015. Freer monads, more extensible effects. In Proceedings of Haskell Symposium. Google Scholar
Digital Library
- Jürgen Krämer and Bernhard Seeger. 2009. Semantics and implementation of continuous sliding window queries over data streams. ACM Transactions on Database Systems 34, 1 (2009), 1–49. Google Scholar
Digital Library
- Neelakantan R. Krishnaswami. 2013. Higher-order Functional Reactive Programming Without Spacetime Leaks. In Proceedings of International Conference on Functional Programming (ICFP). Google Scholar
Digital Library
- Daan Leijen. 2017a. Structured Asynchrony with Algebraic Effects. In Proceedings of the International Workshop on TypeDriven Development (TyDe). Google Scholar
Digital Library
- Daan Leijen. 2017b. Type directed compilation of row-typed algebraic effects. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Jeffrey R. Lewis, John Launchbury, Erik Meijer, and Mark B. Shields. 2000. Implicit Parameters: Dynamic Scoping with Static Types. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Sheng Liang, Paul Hudak, and Mark P. Jones. 1995. Monad Transformers and Modular Interpreters. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Sam Lindley. 2014. Algebraic effects and effect handlers for idioms and arrows. In Proceedings of the 10th ACM SIGPLAN workshop on Generic programming, WGP 2014, Gothenburg, Sweden, August 31, 2014. Google Scholar
Digital Library
- Sam Lindley, Conor McBride, and Craig McLaughlin. 2017. Do Be Do Be Do. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Hai Liu and Paul Hudak. 2007. Plugging a Space Leak with an Arrow. (2007), 29 – 45. Festschrift honoring Gary Lindstrom on his retirement from the University of Utah after 30 years of service. Google Scholar
Digital Library
- David C. Luckham. 2001. The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley Longman Publishing Co., Inc. Google Scholar
Digital Library
- D. C. Luckham, J. J. Kenney, L. M. Augustin, J. Vera, D. Bryan, and W. Mann. 1995. Specification and analysis of system architecture using Rapide. IEEE Transactions on Software Engineering 21, 4 (1995), 336–354. Google Scholar
Digital Library
- Louis Mandel and Luc Maranget. 2014. The JoCaml language, Documentation and user’s manual, Chapter on Concurrent programming. http://jocaml.inria.fr/doc/concurrent.html .Google Scholar
- Erik Meijer, Brian Beckman, and Gavin Bierman. 2006. LINQ: Reconciling Object, Relations and XML in the .NET Framework. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google Scholar
Digital Library
- Leo A. Meyerovich, Arjun Guha, Jacob Baskin, Gregory H. Cooper, Michael Greenberg, Aleks Bromfield, and Shriram Krishnamurthi. 2009. Flapjax: A Programming Language for Ajax Applications. In Proceedings of Conference on ObjectOriented Programming, Systems, Languages, and Applications (OOPSLA). Google Scholar
Digital Library
- Neil Mitchell. 2013. Leaking Space. ACM Queue 11, 9, Article 10 (Sept. 2013), 14 pages. Google Scholar
Digital Library
- Eugenio Moggi. 1991. Notions of computation and monads. Information and computation 93, 1 (1991), 55–92. Google Scholar
Digital Library
- Bruno C.d.S. Oliveira and William R. Cook. 2012. Extensibility for the Masses - Practical Extensibility with Object Algebras. In Proceedings of European Conference on Object-Oriented Programming (ECOOP). Google Scholar
Digital Library
- Bruno C.d.S. Oliveira, Tijs van der Storm, Alex Loh, and William R. Cook. 2013. Feature-Oriented Programming with Object Algebras. In Proceedings of European Conference on Object-Oriented Programming (ECOOP). Google Scholar
Digital Library
- Tomas Petricek, Alan Mycroft, and Don Syme. 2011. Extending monads with pattern matching. In Proceedings of Haskell Symposium. Google Scholar
Digital Library
- Tomas Petricek, Dominic Orchard, and Alan Mycroft. 2014. Coeffects: A calculus of context-dependent computation. In Proceedings of International Conference on Functional Programming (ICFP). Google Scholar
Digital Library
- Tomas Petricek and Don Syme. 2011. Joinads: A Retargetable Control-Flow Construct for Reactive, Parallel and Concurrent Programming. In Proceedings of the International Symposium on Practical Aspects of Declarative Languages (PADL). Google Scholar
Digital Library
- Gordon D. Plotkin and John Power. 2003. Algebraic Operations and Generic Effects. Applied Categorical Structures (2003).Google Scholar
- Gordon D. Plotkin and Matija Pretnar. 2009. Handlers of Algebraic Effects. In Proceedings of the European Conference on Programming Languages and Systems (ESOP). Google Scholar
Digital Library
- ReactiveX. {n. d.}. ReactiveX. http://reactivex.io .Google Scholar
- John H. Reppy. 1991. CML: A Higher-Order Concurrent Language. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). Google Scholar
Digital Library
- Rx.NET Example. 2013. Event Correlation. https://github.com/dotnet/reactive/blob/master/Rx.NET/Samples/ EventCorrelationSample/EventCorrelationSample/Program.cs#L55Google Scholar
- Guido Salvaneschi, Gerold Hintz, and Mira Mezini. 2014. REScala: Bridging Between Object-oriented and Functional Style in Reactive Applications. In Proceedings of the International Conference on Modularity. ACM. Google Scholar
Digital Library
- Robert Soulé, Martin Hirzel, Robert Grimm, Buğra Gedik, Henrique Andrade, Vibhore Kumar, and Kun-Lung Wu. 2010. A Universal Calculus for Stream Processing Languages. In Proceedings of the European Conference on Programming Languages and Systems (ESOP). Springer-Verlag. Google Scholar
Digital Library
- Wouter Swierstra. 2008. Data types à la carte. Functional Programming (2008).Google Scholar
- Don Syme, Tomas Petricek, and Dmitry Lomov. 2011. The F# Asynchronous Programming Model. In Proceedings of the International Symposium on Practical Aspects of Declarative Languages (PADL). Google Scholar
Digital Library
- Philip Wadler. 1990a. Comprehending monads. In LISP and Functional Programming. Google Scholar
Digital Library
- Philip Wadler. 1990b. Linear types can change the world. In Proceedings of the IFIP Working Group 2.2/2.3 Working Conference on Programming Concepts and Methods.Google Scholar
- Philip Wadler. 1992. The Essence of Functional Programming. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google Scholar
Digital Library
- Walker White, Mirek Riedewald, Johannes Gehrke, and Alan Demers. 2007. What is "Next" in Event Processing?. In Proceedings of the Symposium on Principles of Database Systems (PODS). Google Scholar
Digital Library
- Lukasz Ziarek, K. C. Sivaramakrishnan, and Suresh Jagannathan. 2011. Composable asynchronous events. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). Google Scholar
Digital Library
Index Terms
Versatile event correlation with algebraic effects
Recommendations
Efficient compilation of algebraic effect handlers
The popularity of algebraic effect handlers as a programming language feature for user-defined computational effects is steadily growing. Yet, even though efficient runtime representations have already been studied, most handler-based programs are still ...
Type directed compilation of row-typed algebraic effects
POPL '17Algebraic effect handlers, introduced by Plotkin and Power in 2002,
are recently gaining in popularity as a purely functional approach to
modeling effects. In this article, we give a full overview of
practical algebraic effects in the context of ...
Temporal order optimizations of incremental joins for composite event detection
DEBS '07: Proceedings of the 2007 inaugural international conference on Distributed event-based systemsQueries for composite events typically involve the four complementary dimensions of event data, event composition, relationships between events (esp. temporal and causal), and accumulating events over time windows for negation and aggregation. We ...






Comments