Abstract
The vertex-centric programming model, known as “think like a vertex”, is being used more and more to support various big graph processing methods through iterative supersteps that execute in parallel a user-defined vertex program over each vertex of a graph. However, the imperative and message-passing style of existing systems makes defining a vertex program unintuitive. In this paper, we show that one can benefit more from “Thinking like a vertex” by “Behaving like a function” rather than “Acting like a procedure” with full use of side effects and explicit control of message passing, state, and termination. We propose a functional approach to vertex-centric graph processing in which the computation at every vertex is abstracted as a higher-order function and present Fregel, a new domain-specific language. Fregel has clear functional semantics, supports declarative description of vertex computation, and can be automatically translated into Pregel, an emerging imperative-style distributed graph processing framework, and thereby achieve promising performance. Experimental results for several typical examples show the promise of this functional approach.
- S.-H. Bae and B. Howe. GossipMap: A distributed community detection algorithm for billion-edge directed graphs. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’15, pages 27:1–27:12. ACM, 2015. Google Scholar
Digital Library
- B. Bahmani, R. Kumar, and S. Vassilvitskii. Densest subgraph in streaming and MapReduce. Proceedings of the VLDB Endowment, 5(5):454–465, 2012. Google Scholar
Digital Library
- P. Buneman, M. Fernandez, and D. Suciu. UnQL: A query language and algebra for semistructured data based on structural recursion. The VLDB Journal, 9(1):76–110, 2000. Google Scholar
Digital Library
- M. Erwig. Functional programming with graphs. In Proceedings of the 1997 ACM SIGPLAN International Conference on Functional Programming, ICFP ’97, pages 52–65. ACM, 1997. Google Scholar
Digital Library
- M. Erwig. Inductive graphs and functional graph algorithms. Journal of Functional Programming, 11(5):467–492, 2001. Google Scholar
Digital Library
- L. Fegaras and T. Sheard. Revisiting catamorphisms over datatypes with embedded functions (or, programs from outer space). In Proceedings of the 23rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’96, pages 284–294. ACM, 1996. Google Scholar
Digital Library
- A. F. Gates, O. Natkovich, S. Chopra, P. Kamath, S. M. Narayanamurthy, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Building a highlevel dataflow system on top of Map-Reduce: The Pig experience. Proceedings of the VLDB Endowment, 2(2):1414–1425, 2009. Google Scholar
Digital Library
- J. E. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin. PowerGraph: Distributed graph-parallel computation on natural graphs. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, pages 17–30. USENIX Association, 2012. Google Scholar
Digital Library
- M. Hamana. Initial algebra semantics for cyclic sharing tree structures. Logical Methods in Computer Science, 6(3):1–23, 2010.Google Scholar
Cross Ref
- S. Hidaka, Z. Hu, K. Inaba, H. Kato, K. Matsuda, and K. Nakano. Bidirectionalizing graph transformations. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming, ICFP ’10, pages 205–216. ACM, 2010. Google Scholar
Digital Library
- S. Hidaka, K. Asada, Z. Hu, H. Kato, and K. Nakano. Structural recursion for querying ordered graphs. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional Programming, ICFP ’13, pages 305–318. ACM, 2013. Google Scholar
Digital Library
- S. Hong, H. Chafi, E. Sedlar, and K. Olukotun. Green-Marl: A DSL for easy and efficient graph analysis. In Proceedings of the Seventeenth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVII, pages 349–362. ACM, 2012. Google Scholar
Digital Library
- S. Hong, S. Salihoglu, J. Widom, and K. Olukotun. Simplifying scalable graph processing with a domain-specific language. In Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization, CGO ’14, pages 208–218. ACM, 2014. Google Scholar
Digital Library
- J. Launchbury. Graph algorithms with a functional flavour. In Advanced Functional Programming, First International Spring School on Advanced Functional Programming Techniques-Tutorial Text, pages 308– 331. Springer-Verlag, 1995. Google Scholar
Digital Library
- M. Lesniak. Palovca: Describing and executing graph algorithms in haskell. In Proceedings of the 14th International Conference on Practical Aspects of Declarative Languages, PADL’12, pages 153–167. Springer-Verlag, 2012. Google Scholar
Digital Library
- Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. M. Hellerstein. Distributed GraphLab: A framework for machine learning and data mining in the cloud. Proceedings of the VLDB Endowment, 5(8): 716–727, 2012. Google Scholar
Digital Library
- G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, SIGMOD ’10, pages 135–146. ACM, 2010. Google Scholar
Digital Library
- R. R. McCune, T. Weninger, and G. Madey. Thinking like a vertex: A survey of vertex-centric frameworks for large-scale distributed graph processing. ACM Computing Surveys, 48(2):25:1–25:39, 2015. Google Scholar
Digital Library
- B. C. Oliveira and W. R. Cook. Functional programming with structured graphs. In Proceedings of the 17th ACM SIGPLAN International Conference on Functional Programming, ICFP ’12, pages 77–88. ACM, 2012. Google Scholar
Digital Library
- D. Sengupta, S. L. Song, K. Agarwal, and K. Schwan. GraphReduce: Processing large-scale graphs on accelerator-based systems. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’15, pages 28:1–28:12. ACM, 2015. Google Scholar
Digital Library
- A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive: A warehousing solution over a Map-Reduce framework. Proceedings of the VLDB Endowment, 2(2):1626– 1629, 2009. Google Scholar
Digital Library
- L. D. Tung and Z. Hu. Towards systematic parallelization of graph transformations over Pregel. In Proceedings of the 8th International Symposium on High-level Parallel Programming and Applications, HLPP ’15, 2015.Google Scholar
- L. G. Valiant. A bridging model for parallel computation. Communications of the ACM, 33(8):103–111, 1990. Google Scholar
Digital Library
- D. Yan, J. Cheng, K. Xing, Y. Lu, W. Ng, and Y. Bu. Pregel algorithms for graph connectivity problems with performance guarantees. Proceedings of the VLDB Endowment, 7(14):1821–1832, 2014. Google Scholar
Digital Library
Index Terms
Think like a vertex, behave like a function! a functional DSL for vertex-centric big graph processing
Recommendations
Think like a vertex, behave like a function! a functional DSL for vertex-centric big graph processing
ICFP 2016: Proceedings of the 21st ACM SIGPLAN International Conference on Functional ProgrammingThe vertex-centric programming model, known as “think like a vertex”, is being used more and more to support various big graph processing methods through iterative supersteps that execute in parallel a user-defined vertex program over each vertex of a ...
Bimagic Vertex Labelings
The notion of the equivalence of vertex labelings on a given graph is introduced. The equivalence of three bimagic labelings for regular graphs is proved. A particular solution is obtained for the problem of the existence of a 1-vertex bimagic vertex ...
Adjacent vertex-distinguishing edge and total chromatic numbers of hypercubes
An adjacent vertex-distinguishing edge coloring of a simple graph G is a proper edge coloring of G such that incident edge sets of any two adjacent vertices are assigned different sets of colors. A total coloring of a graph G is a coloring of both the ...







Comments