skip to main content
research-article

Lazygraph: lazy data coherency for replicas in distributed graph-parallel computation

Authors Info & Claims
Published:10 February 2018Publication History
Skip Abstract Section

Abstract

Replicas 1 of a vertex play an important role in existing distributed graph processing systems which make a single vertex to be parallel processed by multiple machines and access remote neighbors locally without any remote access. However, replicas of vertices introduce data coherency problem. Existing distributed graph systems treat replicas of a vertex v as an atomic and indivisible vertex, and use an eager data coherency approach to guarantee replicas atomicity. In eager data coherency approach, any changes to vertex data must be immediately communicated to all replicas of v, thus leading to frequent global synchronizations and communications.

In this paper, we propose a lazy data coherency approach, called LazyAsync, which treats replicas of a vertex as independent vertices and maintains the data coherency by computations, rather than communications in existing eager approach. Our approach automatically selects some data coherency points from the graph algorithm, and maintains all replicas to share the same global view only at such points, which means the replicas are enabled to maintain different local views between any two adjacent data coherency points. Based on PowerGraph, we develop a distributed graph processing system LazyGraph to implement the LazyAsync approach and exploit graph-aware optimizations. On a 48-node EC2-like cluster, LazyGraph outperforms PowerGraph on four widely used graph algorithms across a variety of real-world graphs, with a speedup ranging from 1.25x to 10.69x.

References

  1. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A System for Large-Scale Graph Processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data (SIGMOD 2010), ACM, pp. 135--146, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. Distributed Graphlab: a Framework for Machine Learning and Data Mining in the Cloud. In Proceedings of the VLDB Endowment, pp. 716--727, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Guestrin. PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs. In Proceedings of the 10th USENIX conference on Operating Systems Design and Implementation (OSDI, 2012), pp. 17--30, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. X. Zhu, W. Chen, W. Zheng and X. Ma. Gemini: A Computation-Centric Distributed Graph Processing System. In Proceedings of t 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 301--316, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. E. Gonzalez, R. S. Xin, A. Dave, D. Crankshaw, M. J. Franklin and I. Stoica. Graphx: Graph Processing in A Distributed Dataflow Framework. In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation (OSDI 2014), pp. 599--613, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. AVERY. Giraph: Large-scale graph processing infrastructure on hadoop. In Proceedings of the Hadoop Summit, 2011.Google ScholarGoogle Scholar
  7. B. Shao, H. Wang, and Y. Li. Trinity: A Distributed Graph Engine on A Memory Cloud. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD 2013), ACM, pp. 505--516, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. R. Chen, J. Shi, Y. Chen, and H. Chen. Powerlyra: Differentiated Graph Computation and Partitioning on Skewed Graphs. In Proceedings of the Tenth European Conference on Computer Systems (EuroSys 2015), 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Xie, R. Chen, H. Guan, B. Zang, and H. Chen. Sync or Async: Time to Fuse for Distributed Graph-Parallel Computation. In Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP 2015), 50(8), pp. 194--204, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Roy, L. Bindschaedler, J. Malicevic, and W. Zwaenepoel. Chaos: Scale-out Graph Processing from Secondary Storage. In Proceedings of the 25th Symposium on Operating Systems Principles (SOSP 2015), ACM, pp. 410--424, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Seo, E. J. Yoon, J. Kim, and S. Jin. Hama: An Efficient Matrix Computation with the Mapreduce Framework. In Proceedings of 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom 2010), IEEE, pp. 721--726, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. D. Gregor, and A. Lumsdaine. The Parallel BGL: A Generic Library for Distributed Graph Computations. In Parallel Object-Oriented Scientific Computing, 2015.Google ScholarGoogle Scholar
  13. I. Hoque, and I. Gupta. LFGraph: Simple and Fast Distributed Graph Analytics. In Proceedings of the First ACM SIGOPS Conference on Timely Results in Operating Systems (SIGOPS 2013), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. H. Teixeira, A. J. Fonseca, M. Serafini, G. Siganos, M. J. Zaki, and A. Aboulnaga. Arabesque: A System for Distributed Graph Mining. In Proceedings of the 25th Symposium on Operating Systems Principles (SOSP 2015). pp. 425--440, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. D. Nguyen, A. Lenharth, and K. Pingali. A Lightweight Infrastructure for Graph Analytics. In Proceedings of the TwentyFourth ACM Symposium on Operating Systems Principles (SOSP 2013), pp. 456--471, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. J. Shun, and G. E. Blelloch. Ligra: A Lightweight Graph Processing Framework for Shared Memory. In Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming (PPoPP 2013), 48(8), pp. 135--146, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. N. Sundaram, N. Satish, M. M. A. Patwary, S. R. Dulloor, M. J. Anderson, S. G. Vadlamudi, D. Das, and P. Dubey. Graphmat: High performance graph analytics made productive. In Proceedings of the VLDB Endowment (VLDB 2015), 8(11), pp. 1214--1225, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. K. Zhang, R. Chen, and H. Chen. NUMA-Aware Graph-Structured Analytics. In Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP 2015), pp. 183--193, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. G. Wang, W. Xie, A. J. Demers, and J. Gehrke. Asynchronous Large-Scale Graph Processing Made Easy. In CIDR. 2013.Google ScholarGoogle Scholar
  20. U. V. Catalyurek, and C. Aykanat. Decomposing Irregularly Sparse Matrices for Parallel Matrix Vector Multiplication. In Proceedings of the Third International Workshop on Parallel Algorithms for Irregularly Structured Problems (IRREGULAR 1996), pp. 75--86, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. N. Jain, G. Liao, and T. L. Willke. GraphBuilder: A Scalable Graph ETL Framework. In First International Workshop on Graph Data Management Experiences and Systems (GRADES 2013), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. G. Karypis and V. Kumar. Parallel Multilevel k-way Partitioning Scheme for Irregular Graphs. In Proceedings of the 1996 ACM/IEEE conference on Supercomputing, 41(2), pp. 278--300, 1999.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. K. Schloegel, G. Karypis, and V. Kumar. Parallel Multilevel Algorithms for Multi-constraint Graph Partitioning. In Proceedings of the 1998 ACM/IEEE conference on Supercomputing (Euro-Par 2000), pp. 296--310, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. I. Stanton and G. Kliot. Streaming Graph Partitioning for Large Distributed Graphs. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 2012), pp 1222--1230, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. C. Tsourakakis, C. Gkantsidis, B. Radunovic, and M. Vojnovic. FENNEL: Streaming Graph Partitioning for Massive Scale Graphs. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM 2014), pp.333--342, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. H. Kwak, C. Lee, H. Park, and S. Moon. What is Twitter, a Social Network or a News Media? In Proceedings of 19th International World-Wide Web Conference (WWW 2010), pp. 591--600, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. P. Boldi, B. Codenotti, M. Santini, and S. Vigna. UbiCrawler: A Scalable Fully Distributed Web Crawler. In Journal of Software: Practice and Experience, 34(8), pp. 711--726, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. H. Haselgrove. Wikipedia page-to-page link database. http://haselgrove.id.au/wikipedia.htm, 2010.Google ScholarGoogle Scholar
  29. F. Chierichetti, R. Kumar, S. Lattanzi, M. Mitzenmacher, A. Panconesi, and P. Raghavan. On Compressing Social Networks. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 2009), pp. 219--228, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. J. Leskovec, K. J. Lang, A. Dasgupta, and M. W. Mahoney. Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters. Internet Mathematics, 6(1), pp. 29--123, 2009.Google ScholarGoogle ScholarCross RefCross Ref
  31. SNAP: Stanford Network Analysis Platform. snap.stanford.edu/snap/index.htmlGoogle ScholarGoogle Scholar
  32. 9th DIMACS Implementation Challenge. http://www.dis.uniroma1.it/challenge9/download.shtml.Google ScholarGoogle Scholar
  33. F. Bourse, M. Lelarge, and M. Vojnovic. Balanced Graph Edge Partition. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining (SIGKDD 2014), pp. 1456--1465, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. Wu, F. Yang, J. Xue, W. Xiao, Y. Miao, L. Wei, H. Lin, Y. Dai, and L. Zhou. Gram: Scaling Graph Computation to the Trillions. In Proceedings of the Sixth ACM Symposium on Cloud Computing (SOCC 2015), pp. 408--421, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. S. Hong, S. Depner, T. Manhardt, J. Van Der Lugt, M. Verstraaten, and H. Chafi. Pgx.d: A Fast Distributed Graph Processing Engine. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'15), 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. A. Quamar, A. Deshpande, and J. Lin. Nscale: Neighborhood-Centric Analytics on Large graphs. In Proceedings of the VLDB Endowment, pp. 1673--1676, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. J. Shi, Y. Yao, R. Chen, H. Chen, and F. Li. Fast and Concurrent RDF Queries with RDMA-Based Distributed Graph Exploration. In Proceedings of 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. D. Zheng, D. Mhembere, R. Burns, J. Vogelstein, C. E. Priebe, and A. S. Szalay. FlashGraph: Processing Billion-Node Graphs on an Array of Commodity SSDs. In Proceedings of 13th USENIX Conference on File and Storage Technologies (FAST15), pp. 45--58, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. R. Cheng, J. Hong, A. Kyrola, Y. Miao, X. Weng, M. Wu, F. Yang, L. Zhou, F. Zhao, and E. Chen. Kineograph: Taking the Pulse of a Fast-Changing and Connected World. In Proceedings of the 7th ACM European Conference on Computer Systems (EuroSys 2012), pp. 85--98, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. W. Han, Y. Miao, K. Li, M. Wu, F. Yang, L. Zhou, V. Prabhakaran, W. Chen, and E. Chen. Chronos: A Graph Engine for Temporal Graph Analysis. In Proceedings of the Ninth European Conference on Computer Systems(EuroSys 2014), 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. U. Khurana and A. Deshpande. Efficient Snapshot Retrieval over Historical Graph Data. In Proceedings of 2013 IEEE 29th International Conference on Data Engineering (ICDE 2013), pp. 997--1008, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. P. Macko, V. J. Marathe, D. W. Margo, and M. I. Seltzer. LLAMA: Efficient Graph Analytics Using Large Multiversioned Arrays. In Proceedings of IEEE 31st International Conference on Data Engineering (ICDE 2015), 2015.Google ScholarGoogle ScholarCross RefCross Ref
  43. M. Zhang, Y. Wu, K. Chen, X. Qian, X. Li, and W. Zheng. Exploring the Hidden Dimension in Graph Processing. In Proceedings of 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI16), 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. J. Zhong, and B. He. Medusa: Simplified Graph Processing on GPUs. In IEEE Transactions on Parallel and Distributed Systems (TPDS 2013). 25(6), pp.1543--1552, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. J. Zhong, and B. He. Parallel Graph Processing on Graphics Processors Made Easy. In Proceedings of the VLDB Endowment (VLDB 2013), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. The laboratory for web algorithmic. http://law.dsi.unimi.it/datasets.php.Google ScholarGoogle Scholar
  47. D. G. Murray, F. Mcsherry, R. Isaacs, M. Isard, P. Barham, and M. Abadi. Naiad: A Timely Dataflow System. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP 2013), 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. R. Chen, X. Ding, P. Wang, H. Chen, B. Zang, and H. Guan. Computation and Communication Efficient Graph Processing with Distributed Immutable View. In Proceedings of the 23rd international symposium on High-performance parallel and distributed computing (HPDC 2014), 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. S. Brin, and L. Page. The Anatomy of A Large-Scale Hypertextual Web Search Engine. In Proceedings of Seventh International World-Wide Web Conference (WWW 1998), 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Gonzalez J. E., Low Y., Guestrin C., and O'HALLARON, D. Distributed Parallel Inference on Large Factor Graphs. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI 2009). pp. 203--212, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. M. Han, and K. Daudjee Giraph. Unchained: Barrierless Asynchronous Parallel Execution in Pregel-like Graph Processing Systems. In Proceedings of the VLDB Endowment. 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. X. Ju, H. Jamjoom, K. G. Shin. Hieroglyph: Locally-Sufficient Graph Processing via Compute-Sync-Merg. In Proceedings of the 2017 ACM SIGMETRICS / International Conference on Measurement and Modeling of Computer Systems, 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. R. R. McCune, T. Weninger, and G. Madey. Thinking Like a Vertex: a Survey of Vertex-Centric Frameworks for Large-Scale Distributed Graph Processing. In ACM Computing Surveys, 48(2), 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. X. Shi, X. Luo, J. Liang, P. Zhao, S. Di, B. He, H. Jin. Frog: Asynchronous Graph Processing on GPU with Hybrid Coloring Model. In IEEE Transactions on Knowledge and Data Engineering, 30 (1), pp 29--42, 2018.Google ScholarGoogle ScholarCross RefCross Ref
  55. L. Wang, F. Yang, L. Zhuang, H. Cui, F. Lv, X. Feng. Articulation Points Guided Redundancy Elimination for Betweenness Centrality. In Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP 2016), 51(8), 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. J. Zhao, H. Cui, J. Xue, X. Feng, Y. Yan, and W. Yang. An Empirical Model for Predicting Cross-core Performance Interference on Multicore Processors. In Proceedings of the 22nd International Conference on Parallel Architectures and Compilation Techniques (PACT 2013), pp. 201--212, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Lazygraph: lazy data coherency for replicas in distributed graph-parallel computation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 53, Issue 1
        PPoPP '18
        January 2018
        426 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/3200691
        Issue’s Table of Contents
        • cover image ACM Conferences
          PPoPP '18: Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
          February 2018
          442 pages
          ISBN:9781450349826
          DOI:10.1145/3178487

        Copyright © 2018 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 10 February 2018

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!