skip to main content
research-article

Fiedler Vector Approximation via Interacting Random Walks

Published:27 May 2020Publication History
Skip Abstract Section

Abstract

The Fiedler vector of a graph, namely the eigenvector corresponding to the second smallest eigenvalue of a graph Laplacian matrix, plays an important role in spectral graph theory with applications in problems such as graph bi-partitioning and envelope reduction. Algorithms designed to estimate this quantity usually rely on a priori knowledge of the entire graph, and employ techniques such as graph sparsification and power iterations, which have obvious shortcomings in cases where the graph is unknown, or changing dynamically. In this paper, we develop a framework in which we construct a stochastic process based on a set of interacting random walks on a graph and show that a suitably scaled version of our stochastic process converges to the Fiedler vector for a sufficiently large number of walks. Like other techniques based on exploratory random walks and on-the-fly computations, such as Markov Chain Monte Carlo (MCMC), our algorithm overcomes challenges typically faced by power iteration based approaches. But, unlike any existing random walk based method such as MCMCs where the focus is on the leading eigenvector, our framework with interacting random walks converges to the Fiedler vector (second eigenvector). We also provide numerical results to confirm our theoretical findings on different graphs, and show that our algorithm performs well over a wide range of parameters and the number of random walks. Simulations results over time varying dynamic graphs are also provided to show the efficacy of our random walk based technique in such settings. As an important contribution, we extend our results and show that our framework is applicable for approximating not just the Fiedler vector of graph Laplacians, but also the second eigenvector of any time reversible Markov Chain kernel via interacting random walks. To the best of our knowledge, our attempt to approximate the second eigenvector of any time reversible Markov Chain using random walks is the first of its kind, opening up possibilities to achieving approximations of higher level eigenvectors using random walks on graphs.

References

  1. David Aldous and James Allen Fill. 2002. Reversible Markov Chains and Random Walks on Graphs. Unfinished monograph, recompiled 2014, available at http://www.stat.berkeley.edu/ aldous/RWG/book.html.Google ScholarGoogle Scholar
  2. R. Andersen, F. Chung, and K. Lang. 2006. Local Graph Partitioning using PageRank Vectors. In 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06). 475--486.Google ScholarGoogle Scholar
  3. B. Aspvall and J. Gilbert. 1984. Graph Coloring Using Eigenvalue Decomposition. SIAM Journal on Algebraic Discrete Methods , Vol. 5, 4 (1984), 526--538.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. John Ball. 2016. Semiflows, Lyapunov functions and approach to equilibrium. University lecture. https://people.maths.ox.ac.uk/ball/Teaching/cdtsemiflows16.pdfGoogle ScholarGoogle Scholar
  5. S. T. Barnard, A. Pothen, and H. D. Simon. 1993. A spectral algorithm for envelope reduction of sparse matrices. In Proceedings of the 1993 ACM/IEEE Conference on Supercomputing (SC'93). 493--502.Google ScholarGoogle Scholar
  6. Stephen T. Barnard and Horst D. Simon. 1993. A Fast Multilevel Implementation of Recursive Spectral Bisection for Partitioning Unstructured Problems. Concurrency - Practice and Experience , Vol. 6 (1993), 101--117.Google ScholarGoogle ScholarCross RefCross Ref
  7. Michel Benaïm and Jörgen W. Weibull. 2003. Deterministic Approximation of Stochastic Evolution in Games. Econometrica , Vol. 71, 3 (2003), 873--903.Google ScholarGoogle ScholarCross RefCross Ref
  8. Alexander Bertrand and Marc Moonen. 2013. Distributed computation of the Fiedler vector with application to topology inference in ad hoc networks. Signal Processing , Vol. 93, 5 (2013), 1106--1117.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. Brémaud. 1999. Markov chains: Gibbs fields, Monte Carlo simulation, and queues. Springer-Verlag.Google ScholarGoogle Scholar
  10. John C. Urschel, Xiaozhe Hu, Jinchao Xu, and Ludmil T. Zikatanov. 2015. A Cascadic Multigrid Algorithm for Computing the Fiedler Vector of Graph Laplacians. Journal of Computational Mathematics , Vol. 33 (2015), 209--226.Google ScholarGoogle ScholarCross RefCross Ref
  11. F. R. K. Chung. 1997. Spectral Graph Theory .American Mathematical Society.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Richard Combes and Mikael Touati. 2019. Computationally Efficient Estimation of the Spectral Gap of a Markov Chain. Proc. ACM Meas. Anal. Comput. Syst. , Vol. 3, 1, Article 7 (2019), bibinfonumpages21 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. K.Ch. Das. 2004. The Laplacian spectrum of a graph. Computers & Mathematics with Applications , Vol. 48, 5 (2004), 715--724.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Inderjit S. Dhillon, Yuqiang Guan, and Brian Kulis. 2004. Kernel K-means: Spectral Clustering and Normalized Cuts. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '04). 551--556.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Chris Ding, Xiaofeng He, and Horst D. Simon. 2005. On the equivalence of nonnegative matrix factorization and spectral clustering. In SIAM International Conference on Data Mining (ICDM'05).Google ScholarGoogle Scholar
  16. C. H. Q. Ding, Xiaofeng He, Hongyuan Zha, Ming Gu, and H. D. Simon. 2001. A min-max cut algorithm for graph partitioning and data clustering. In Proceedings 2001 IEEE International Conference on Data Mining (ICDM'01). 107--114.Google ScholarGoogle Scholar
  17. Moez Draief and Laurent Massouli. 2010. Epidemics and Rumours in Complex Networks 1st ed.). Cambridge University Press, New York, NY, USA.Google ScholarGoogle Scholar
  18. Rafael Van Driessche and Dirk Roose. 1995. A Grapg Contraction Algorithm for the Fast Calculation of the Fiedler Vector of a Graph. In Proceedings of the seventh SIAM conference on Parallel Processing for Scientific Computing (PPSC'95).Google ScholarGoogle Scholar
  19. Miroslav Fiedler. 1973. Algebraic connectivity of graphs. Czechoslovak Mathematical Journal , Vol. 23(98) (1973), 298--305.Google ScholarGoogle ScholarCross RefCross Ref
  20. Miroslav Fiedler. 1975. A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory. Czechoslovak Mathematical Journal , Vol. 25(100), 4 (1975), 619--633.Google ScholarGoogle ScholarCross RefCross Ref
  21. L. Hagen and A. B. Kahng. 1992. New spectral methods for ratio cut partitioning and clustering. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , Vol. 11, 9 (1992), 1074--1085.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Insu. Han, Dmitry. Malioutov, Haim. Avron, and Jinwoo. Shin. 2017. Approximating Spectral Sums of Large-Scale Matrices using Stochastic Chebyshev Approximations. SIAM Journal on Scientific Computing , Vol. 39, 4 (2017), A1558--A1585.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. W. K. Hastings. 1970. Monte Carlo Sampling Methods Using Markov Chains and Their Applications. Biometrika , Vol. 57, 1 (1970), 97--109.Google ScholarGoogle ScholarCross RefCross Ref
  24. Morris W. Hirsch and Stephen Smale. 1974. Differential equations, dynamical systems, and linear algebra .Acad. Press, San Diego, CA, USA.Google ScholarGoogle Scholar
  25. Roger A. Horn and Charles R. Johnson. 2012. Matrix Analysis 2nd ed.). Cambridge University Press, New York, NY, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Daniel Hsu, Aryeh Kontorovich, and Csaba Szepesvári. 2015. Mixing Time Estimation in Reversible Markov Chains from a Single Sample Path. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'15). 1459--1467.Google ScholarGoogle Scholar
  27. Yehuda. Koren, Liran. Carmel, and David. Harel. 2003. Drawing Huge Graphs by Algebraic Multigrid Optimization. Multiscale Modeling & Simulation , Vol. 1, 4 (2003), 645--673.Google ScholarGoogle ScholarCross RefCross Ref
  28. Sibsankar Kundu, Dan C. Sorensen, and George N. Phillips, Jr. 2004. Automatic domain decomposition of proteins by a Gaussian Network Model. Proteins: Structure, Function, and Bioinformatics , Vol. 57, 4 (2004), 725--733.Google ScholarGoogle ScholarCross RefCross Ref
  29. Chul-Ho Lee, Min Kang, and Do Young Eun. 2019. Non-Markovian Monte Carlo on Directed Graphs. In Proceedings of the ACM SIGMETRICS/IFIP Performance 2019 (SIGMETRICS'19).Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Chul-Ho Lee, Xin Xu, and Do Young Eun. 2012. Beyond Random Walk and Metropolis-Hastings Samplers: Why You Should Not Backtrack for Unbiased Graph Sampling. In Proceedings of the ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS'12). 319--330.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jure Leskovec and Andrej Krevl. 2014. SNAP Datasets: Stanford Large Network Dataset Collection. http://snap.stanford.edu/data .Google ScholarGoogle Scholar
  32. David A. Levin, Yuval Peres, and Elizabeth L. Wilmer. 2006. Markov chains and mixing times. American Mathematical Society.Google ScholarGoogle Scholar
  33. Jun S. Liu. 2004. Monte Carlo Strategies in Scientific Computing .Springer-Verlag.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. D. Lusseau, K. Schneider, O. J. Boisseau, P. Haase, E. Slooten, and S. M. Dawson. 2003. The Bottlenose Dolphin Community of Doubtful Sound Features a Large Proportion of Long-Lasting Associations. Behavioral Ecology and Sociobiology , Vol. 54 (2003), 396--405.Google ScholarGoogle ScholarCross RefCross Ref
  35. Carl D. Meyer. 2000. Matrix Analysis and Applied Linear Algebra .SIAM.Google ScholarGoogle Scholar
  36. R. K. Miller and A. N. Michel. 1982. Ordinary Differential Equations .Academic Press, New York, NY, USA.Google ScholarGoogle Scholar
  37. A.Y. Ng, Michael Jordan, and Y Weiss. 2001. On Spectral Clustering: Analysis and an Algorithm. Adv. Neural Inf. Process. Syst. , Vol. 2 (11 2001).Google ScholarGoogle Scholar
  38. Pekka Orponen and Satu Elisa Schaeffer. 2005. Local Clustering of Large Graphs by Approximate Fiedler Vectors. In Proceedings of the 4th International Conference on Experimental and Efficient Algorithms (WEA'05). 524--533.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Lawrence Perko. 2001. Differential Equations and Dynamical Systems (Third ed.) .New York: Springer.Google ScholarGoogle Scholar
  40. P. H. Peskun. 1973. Optimum Monte-Carlo Sampling Using Markov Chains. Biometrika , Vol. 60, 3 (1973), 607--612.Google ScholarGoogle ScholarCross RefCross Ref
  41. Alex Pothen, Horst D. Simon, and Kan-Pu Liou. 1990 a. Partitioning Sparse Matrices with Eigenvectors of Graphs. SIAM J. Matrix Anal. Appl. , Vol. 11, 3 (1990), 430--452.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Alex. Pothen, Horst D. Simon, and Kang-Pu. Liou. 1990 b. Partitioning Sparse Matrices with Eigenvectors of Graphs. SIAM J. Matrix Anal. Appl. , Vol. 11, 3 (1990), 430--452.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. S. J. Shepherd, C. B. Beggs, and S. Jones. 2007. Amino acid partitioning using a Fiedler vector model. European Biophysics Journal , Vol. 37, 1 (2007), 105--109.Google ScholarGoogle ScholarCross RefCross Ref
  44. Jianbo Shi and J. Malik. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 22, 8 (2000), 888--905.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. H.D. Simon. 1991. Partitioning of unstructured problems for parallel processing. Computing Systems in Engineering , Vol. 2, 2 (1991), 135--148.Google ScholarGoogle Scholar
  46. Brian Slininger. 2013. Fiedlers Theory of Spectral Graph Partitioning.Google ScholarGoogle Scholar
  47. Jean-Jacques E. Slotine and Weiping Li. 1991. Applied Nonlinear Control .Prentice-Hall.Google ScholarGoogle Scholar
  48. Daniel A. Spielman and Nikhil. Srivastava. 2011. Graph Sparsification by Effective Resistances. SIAM J. Comput. , Vol. 40, 6 (2011), 1913--1926.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Daniel A. Spielman and Shang-Hua. Teng. 2013. A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning. SIAM J. Comput. , Vol. 42, 1 (2013), 1--26.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Daniel A. Spielman and Shang-Hua. Teng. 2014. Nearly Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems. SIAM J. Matrix Anal. Appl. , Vol. 35, 3 (2014), 835--885.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Ashok Srinivasan and Michael Mascagni. 2002. Monte Carlo Techniques for Estimating the Fiedler Vector in Graph Applications. In Proceedings of the International Conference on Computational Science-Part II (ICCS'02). 635--645.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. M. Vidyasagar. 1993. Nonlinear Systems Analysis .Prentice-Hall, Englewood Cliffs, NJ, USA.Google ScholarGoogle Scholar
  53. Ulrike von Luxburg. 2007. A Tutorial on Spectral Clustering. Statistics and computing , Vol. 17, 4 (2007), 395--416.Google ScholarGoogle Scholar
  54. Yu and Shi. 2003. Multiclass spectral clustering. In Proceedings Ninth IEEE International Conference on Computer Vision (ICCV'03). 313--319.Google ScholarGoogle Scholar

Index Terms

  1. Fiedler Vector Approximation via Interacting Random Walks

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!