skip to main content
research-article

Computationally Efficient Estimation of the Spectral Gap of a Markov Chain

Published:26 March 2019Publication History
Skip Abstract Section

Abstract

We consider the problem of estimating from sample paths the absolute spectral gap 1 - λ of a reversible, irreducible and aperiodic Markov chain (X_t)_t∈N over a finite state space Ω. We propose the UCPI (Upper Confidence Power Iteration) algorithm for this problem, a low-complexity algorithm which estimates the spectral gap in time O(n) and memory space O ((łn n)^2) given n samples. This is in stark contrast with most known methods which require at least memory space O(|Ω|), so that they cannot be applied to large state spaces. We also analyze how n should scale to reach a target estimation error. Furthermore, UCPI is amenable to parallel implementation.

References

  1. C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan. 2003. An Introduction to MCMC for Machine Learning. Machine Learning , Vol. 50, 1--2 (2003), 5--43.Google ScholarGoogle ScholarCross RefCross Ref
  2. Insu Han, Dmitry Malioutov, Haim Avron, and Jinwoo Shin. 2017. Approximating Spectral Sums of Large-Scale Matrices using Stochastic Chebyshev Approximations. SIAM J. Scientific Computing , Vol. 39, 1 (2017).Google ScholarGoogle ScholarCross RefCross Ref
  3. W. Hoeffding. 1963. Probability Inequalities for Sums of Bounded Random Variables. J. Amer. Statist. Assoc. (1963), 13--30.Google ScholarGoogle Scholar
  4. Daniel Hsu, Aryeh Kontorovich, and Csaba Szepesvá ri. 2015. Mixing Time Estimation in Reversible Markov Chains from a Single Sample Path. CoRR , Vol. abs/1506.02903 (2015). http://arxiv.org/abs/1506.02903Google ScholarGoogle Scholar
  5. R. Kannan, L. Lovász, and M. Simonovits. 1997. Random walks and an O(n5) volume algorithm for convex bodies. Random Structures & Algorithms , Vol. 11, 1 (1997), 1--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Ashish Khetan and Sewoong Oh. 2017. Matrix Norm Estimation from a Few Entries. In In Proc. of NIPS . Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. David Asher Levin, Yuval Peres, and Elizabeth Lee Wilmer. 2009. Markov chains and mixing times .Providence, R.I. American Mathematical Society.Google ScholarGoogle Scholar
  8. L. Lovász. 1996. Random Walks on Graphs: A Survey. In Combinatorics, Paul ErdHos is Eighty, D. Miklós , V. T. Sós, and T. SzHonyi (Eds.). Vol. 2. János Bolyai Mathematical Society, Budapest, 353--398.Google ScholarGoogle Scholar
  9. L. Lovász. 1999. Hit-and-run mixes fast. Mathematical Programming , Vol. 86, 3 (1999), 443--461.Google ScholarGoogle ScholarCross RefCross Ref
  10. L. Lovász and S. Vempala. 2006. Hit-and-Run from a Corner. SIAM J. Comput. , Vol. 35, 4 (2006), 985--1005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. Mezard and A. Montanari. 2006. Information, Physics and Computation .Oxford Graduate Texts. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Gary W Oehlert. 1992. A note on the delta method. American Statistician , Vol. 46, 1 (1 1 1992), 27--29.Google ScholarGoogle Scholar
  13. J. Propp and D. Wilson. 1998. Coupling from the past: a user's guide. DIMACS Ser. Discrete Math. Theoret. Comput. Sci. (1998), 181--192.Google ScholarGoogle Scholar
  14. Qian Qin, James P. Hobert, and Kshitij Khare. 2017. Estimating the Spectral Gap of a trace-class Markov operator. https://arxiv.org/abs/1704.00850 (2017).Google ScholarGoogle Scholar
  15. Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri. 2007. Numerical Mathematics . Texts in Applied Mathematics, Vol. 37. Springer-Verlag Berlin Heidelberg. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Kong Weihao and Gregory Valiant. 2017. Spectrum estimation from samples. The Annals of Statistics , Vol. 45, 5 (2017).Google ScholarGoogle Scholar

Index Terms

  1. Computationally Efficient Estimation of the Spectral Gap of a Markov Chain

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Article Metrics

          • Downloads (Last 12 months)30
          • Downloads (Last 6 weeks)1

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!