skip to main content
research-article
Open Access

Versatile event correlation with algebraic effects

Published:30 July 2018Publication History
Skip Abstract Section

Abstract

We present the first language design to uniformly express variants of n-way joins over asynchronous event streams from different domains, e.g., stream-relational algebra, event processing, reactive and concurrent programming. We model asynchronous reactive programs and joins in direct style, on top of algebraic effects and handlers. Effect handlers act as modular interpreters of event notifications, enabling fine-grained control abstractions and customizable event matching. Join variants can be considered as cartesian product computations with ”degenerate” control flow, such that unnecessary tuples are not materialized a priori. Based on this computational interpretation, we decompose joins into a generic, naive enumeration procedure of the cartesian product, plus variant-specific extensions, represented in terms of user-supplied effect handlers. Our microbenchmarks validate that this extensible design avoids needless materialization. Alongside a formal semantics for joining and prototypes in Koka and multicore OCaml, we contribute a systematic comparison of the covered domains and features.

References

  1. Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil Immerman. 2008. Efficient Pattern Matching over Event Streams. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernández-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, Frances Perry, Eric Schmidt, and Sam Whittle. 2015. The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-scale, Unbounded, Out-of-order Data Processing. Proceedings of the VLDB Endowment 12 (2015), 1792–1803. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Arvind Arasu, Shivnath Babu, and Jennifer Widom. 2004. CQL: A Language for Continuous Queries over Streams and Relations. In Database Programming Languages. Springer Berlin Heidelberg.Google ScholarGoogle Scholar
  4. Arvind Arasu, Shivnath Babu, and Jennifer Widom. 2006. The CQL Continuous Query Language: Semantic Foundations and Query Execution. The VLDB Journal (2006). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Arvind Arasu and Jennifer Widom. 2004. A Denotational Semantics for Continuous Queries over Streams and Relations. SIGMOD Record (2004). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Engineer Bainomugisha, Andoni Lombide Carreton, Tom Van Cutsem, Stijn Mostinckx, and Wolfgang De Meuter. 2013. A survey on reactive programming. Comput. Surveys 45, 4 (2013). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Andrej Bauer and Matija Pretnar. 2015. Programming with Algebraic Effects and Handlers. Journal of Logical and Algebraic Methods in Programming 84, 1 (2015), 108–123.Google ScholarGoogle ScholarCross RefCross Ref
  8. Nick Benton, Luca Cardelli, and Cédric Fournet. 2004. Modern Concurrency Abstractions for C#. Transactions on Programming Languages and Systems (TOPLAS) 26, 5 (2004), 769–804. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Aggelos Biboudis, Nick Palladinos, George Fourtounis, and Yannis Smaragdakis. 2015. Streams à la carte: Extensible Pipelines with Object Algebras. In Proceedings of European Conference on Object-Oriented Programming (ECOOP).Google ScholarGoogle Scholar
  10. Gavin Bierman, Claudio Russo, Geoffrey Mainland, Erik Meijer, and Mads Torgersen. 2012. Pause ’n’ Play: Formalizing Asynchronous C♯. In Proceedings of European Conference on Object-Oriented Programming (ECOOP), James Noble (Ed.). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Dariusz Biernacki, Maciej Piróg, Piotr Polesiuk, and Filip Sieczkowski. 2018. Handle with care: relational interpretation of algebraic effects and handlers. PACMPL POPL (2018). Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Irina Botan, Roozbeh Derakhshan, Nihal Dindar, Laura Haas, Renée J. Miller, and Nesime Tatbul. 2010. SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems. Proceedings of the VLDB Endowment (2010). Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Andrew Cave, Francisco Ferreira, Prakash Panangaden, and Brigitte Pientka. 2014. Fair Reactive Programming. In Proceedings of Symposium on Principles of Programming Languages (POPL) (POPL ’14). ACM, 361–372. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Sharma Chakravarthy, V. Krishnaprasad, Eman Anwar, and S.-K. Kim. 1994. Composite Events for Active Databases: Semantics, Contexts and Detection. In Proceedings of International Conference on Very Large Data Bases (VLDB). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. Chakravarthy and D. Mishra. 1994. Snoop: An expressive event specification language for active databases. Data & Knowledge Engineering 14, 1 (1994), 1 – 26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. James Cheney, Sam Lindley, and Philip Wadler. 2013. A practical theory of language-integrated query. In Proceedings of International Conference on Functional Programming (ICFP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Silvain Conchon and Fabrice Le Fessant. 1999. JoCaml: mobile agents for Objective-Caml. In First and Third International Symposium on Agent Systems Applications, and Mobile Agents (ASAMA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Gregory H. Cooper and Shriram Krishnamurthi. 2006. Embedding Dynamic Dataflow in a Call-by-Value Language. In Proceedings of the European Conference on Programming Languages and Systems (ESOP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Gianpaolo Cugola and Alessandro Margara. 2010. TESLA: A Formally Defined Event Specification Language. In Proceedings of the International Conference on Distributed Event-based Systems (DEBS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Gianpaolo Cugola and Alessandro Margara. 2012. Processing Flows of Information: From Data Stream to Complex Event Processing. Comput. Surveys 44, 3 (2012), 15:1–15:62. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Evan Czaplicki and Stephen Chong. 2013. Asynchronous Functional Reactive Programming for GUIs. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ana Lúcia de Moura and Roberto Ierusalimschy. 2009. Revisiting coroutines. ACM Trans. Program. Lang. Syst. 31, 2 (2009). Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Alan Demers, Johannes Gehrke, Mingsheng Hong, Mirek Riedewald, and Walker White. 2006. Towards Expressive Publish/Subscribe Systems. In Proceedings of the 10th International Conference on Advances in Database Technology (EDBT’06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Yanlei Diao, Neil Immerman, and Daniel Gyllstrom. 2007. SASE+: An Agile Language for Kleene Closure Over Event Streams. Technical Report. UMass Technical Report 07.Google ScholarGoogle Scholar
  25. Nihal Dindar, Nesime Tatbul, Renée J. Miller, Laura M. Haas, and Irina Botan. 2013. Modeling the execution semantics of stream processing engines with SECRET. The VLDB Journal (2013). Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Stephen Dolan, Spiros Eliopoulos, Daniel Hillerström, Anil Madhavapeddy, KC Sivaramakrishnan, and Leo White. 2017. Concurrent System Programming with Effect Handlers. In Proceedings of the Symposium on Trends in Functional Programming.Google ScholarGoogle Scholar
  27. Jonathan Edwards. 2009. Coherent Reaction. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Conal M. Elliott. 2009. Push-pull Functional Reactive Programming. In Proceedings of the 2nd ACM SIGPLAN Symposium on Haskell (Haskell’09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Esper. {n. d.}. Esper. http://www.espertech.com/esper/ .Google ScholarGoogle Scholar
  30. Patrick Eugster and K.R. Jayaram. 2009. EventJava: An Extension of Java for Event Correlation. In Proceedings of European Conference on Object-Oriented Programming (ECOOP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Matthias Felleisen and Robert Hieb. 1992. The Revised Report on the Syntactic Theories of Sequential Control and State. Theoretical Computer Science 103, 2 (1992), 235–271. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Matthew Fluet, Mike Rainey, John H. Reppy, and Adam Shaw. 2008. Implicitly-threaded parallelism in Manticore. In Proceedings of International Conference on Functional Programming (ICFP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Yannick Forster, Ohad Kammar, Sam Lindley, and Matija Pretnar. 2017. On the Expressive Power of User-defined Effects: Effect Handlers, Monadic Reflection, Delimited Control. Proc. ACM Program. Lang. 1, ICFP (2017). Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Cédric Fournet and Georges Gonthier. 1996. The reflexive CHAM and the Join-calculus. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Philipp Haller and Heather Miller. 2013. RAY: Integrating Rx and Async for Direct-Style Reactive Streams. (2013).Google ScholarGoogle Scholar
  36. Martin Hirzel, Robert Soulé, Scott Schneider, Bugra Gedik, and Robert Grimm. 2013. A catalog of stream processing optimizations. Comput. Surveys (2013). Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. John E Hopcroft, Rajeev Motwani, and Jeffrey D Ullman. 2006. Automata theory, languages, and computation. Pearson. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Ohad Kammar, Sam Lindley, and Nicolas Oury. 2013. Handlers in action. In Proceedings of International Conference on Functional Programming (ICFP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Oleg Kiselyov, Aggelos Biboudis, Nick Palladinos, and Yannis Smaragdakis. 2017. Stream Fusion, to Completeness. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Oleg Kiselyov and Hiromi Ishii. 2015. Freer monads, more extensible effects. In Proceedings of Haskell Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Jürgen Krämer and Bernhard Seeger. 2009. Semantics and implementation of continuous sliding window queries over data streams. ACM Transactions on Database Systems 34, 1 (2009), 1–49. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Neelakantan R. Krishnaswami. 2013. Higher-order Functional Reactive Programming Without Spacetime Leaks. In Proceedings of International Conference on Functional Programming (ICFP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Daan Leijen. 2017a. Structured Asynchrony with Algebraic Effects. In Proceedings of the International Workshop on TypeDriven Development (TyDe). Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Daan Leijen. 2017b. Type directed compilation of row-typed algebraic effects. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jeffrey R. Lewis, John Launchbury, Erik Meijer, and Mark B. Shields. 2000. Implicit Parameters: Dynamic Scoping with Static Types. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Sheng Liang, Paul Hudak, and Mark P. Jones. 1995. Monad Transformers and Modular Interpreters. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Sam Lindley. 2014. Algebraic effects and effect handlers for idioms and arrows. In Proceedings of the 10th ACM SIGPLAN workshop on Generic programming, WGP 2014, Gothenburg, Sweden, August 31, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Sam Lindley, Conor McBride, and Craig McLaughlin. 2017. Do Be Do Be Do. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Hai Liu and Paul Hudak. 2007. Plugging a Space Leak with an Arrow. (2007), 29 – 45. Festschrift honoring Gary Lindstrom on his retirement from the University of Utah after 30 years of service. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. David C. Luckham. 2001. The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley Longman Publishing Co., Inc. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. D. C. Luckham, J. J. Kenney, L. M. Augustin, J. Vera, D. Bryan, and W. Mann. 1995. Specification and analysis of system architecture using Rapide. IEEE Transactions on Software Engineering 21, 4 (1995), 336–354. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Louis Mandel and Luc Maranget. 2014. The JoCaml language, Documentation and user’s manual, Chapter on Concurrent programming. http://jocaml.inria.fr/doc/concurrent.html .Google ScholarGoogle Scholar
  53. Erik Meijer, Brian Beckman, and Gavin Bierman. 2006. LINQ: Reconciling Object, Relations and XML in the .NET Framework. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Leo A. Meyerovich, Arjun Guha, Jacob Baskin, Gregory H. Cooper, Michael Greenberg, Aleks Bromfield, and Shriram Krishnamurthi. 2009. Flapjax: A Programming Language for Ajax Applications. In Proceedings of Conference on ObjectOriented Programming, Systems, Languages, and Applications (OOPSLA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Neil Mitchell. 2013. Leaking Space. ACM Queue 11, 9, Article 10 (Sept. 2013), 14 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Eugenio Moggi. 1991. Notions of computation and monads. Information and computation 93, 1 (1991), 55–92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Bruno C.d.S. Oliveira and William R. Cook. 2012. Extensibility for the Masses - Practical Extensibility with Object Algebras. In Proceedings of European Conference on Object-Oriented Programming (ECOOP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Bruno C.d.S. Oliveira, Tijs van der Storm, Alex Loh, and William R. Cook. 2013. Feature-Oriented Programming with Object Algebras. In Proceedings of European Conference on Object-Oriented Programming (ECOOP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Tomas Petricek, Alan Mycroft, and Don Syme. 2011. Extending monads with pattern matching. In Proceedings of Haskell Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Tomas Petricek, Dominic Orchard, and Alan Mycroft. 2014. Coeffects: A calculus of context-dependent computation. In Proceedings of International Conference on Functional Programming (ICFP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Tomas Petricek and Don Syme. 2011. Joinads: A Retargetable Control-Flow Construct for Reactive, Parallel and Concurrent Programming. In Proceedings of the International Symposium on Practical Aspects of Declarative Languages (PADL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Gordon D. Plotkin and John Power. 2003. Algebraic Operations and Generic Effects. Applied Categorical Structures (2003).Google ScholarGoogle Scholar
  63. Gordon D. Plotkin and Matija Pretnar. 2009. Handlers of Algebraic Effects. In Proceedings of the European Conference on Programming Languages and Systems (ESOP). Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. ReactiveX. {n. d.}. ReactiveX. http://reactivex.io .Google ScholarGoogle Scholar
  65. John H. Reppy. 1991. CML: A Higher-Order Concurrent Language. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Rx.NET Example. 2013. Event Correlation. https://github.com/dotnet/reactive/blob/master/Rx.NET/Samples/ EventCorrelationSample/EventCorrelationSample/Program.cs#L55Google ScholarGoogle Scholar
  67. Guido Salvaneschi, Gerold Hintz, and Mira Mezini. 2014. REScala: Bridging Between Object-oriented and Functional Style in Reactive Applications. In Proceedings of the International Conference on Modularity. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Robert Soulé, Martin Hirzel, Robert Grimm, Buğra Gedik, Henrique Andrade, Vibhore Kumar, and Kun-Lung Wu. 2010. A Universal Calculus for Stream Processing Languages. In Proceedings of the European Conference on Programming Languages and Systems (ESOP). Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Wouter Swierstra. 2008. Data types à la carte. Functional Programming (2008).Google ScholarGoogle Scholar
  70. Don Syme, Tomas Petricek, and Dmitry Lomov. 2011. The F# Asynchronous Programming Model. In Proceedings of the International Symposium on Practical Aspects of Declarative Languages (PADL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Philip Wadler. 1990a. Comprehending monads. In LISP and Functional Programming. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Philip Wadler. 1990b. Linear types can change the world. In Proceedings of the IFIP Working Group 2.2/2.3 Working Conference on Programming Concepts and Methods.Google ScholarGoogle Scholar
  73. Philip Wadler. 1992. The Essence of Functional Programming. In Proceedings of Symposium on Principles of Programming Languages (POPL). Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Walker White, Mirek Riedewald, Johannes Gehrke, and Alan Demers. 2007. What is "Next" in Event Processing?. In Proceedings of the Symposium on Principles of Database Systems (PODS). Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Lukasz Ziarek, K. C. Sivaramakrishnan, and Suresh Jagannathan. 2011. Composable asynchronous events. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Versatile event correlation with algebraic effects

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!