Abstract
We consider the problem of estimating from sample paths the absolute spectral gap 1 - λ of a reversible, irreducible and aperiodic Markov chain (X_t)_t∈N over a finite state space Ω. We propose the UCPI (Upper Confidence Power Iteration) algorithm for this problem, a low-complexity algorithm which estimates the spectral gap in time O(n) and memory space O ((łn n)^2) given n samples. This is in stark contrast with most known methods which require at least memory space O(|Ω|), so that they cannot be applied to large state spaces. We also analyze how n should scale to reach a target estimation error. Furthermore, UCPI is amenable to parallel implementation.
- C. Andrieu, N. de Freitas, A. Doucet, and M. Jordan. 2003. An Introduction to MCMC for Machine Learning. Machine Learning , Vol. 50, 1--2 (2003), 5--43.Google Scholar
Cross Ref
- Insu Han, Dmitry Malioutov, Haim Avron, and Jinwoo Shin. 2017. Approximating Spectral Sums of Large-Scale Matrices using Stochastic Chebyshev Approximations. SIAM J. Scientific Computing , Vol. 39, 1 (2017).Google Scholar
Cross Ref
- W. Hoeffding. 1963. Probability Inequalities for Sums of Bounded Random Variables. J. Amer. Statist. Assoc. (1963), 13--30.Google Scholar
- Daniel Hsu, Aryeh Kontorovich, and Csaba Szepesvá ri. 2015. Mixing Time Estimation in Reversible Markov Chains from a Single Sample Path. CoRR , Vol. abs/1506.02903 (2015). http://arxiv.org/abs/1506.02903Google Scholar
- R. Kannan, L. Lovász, and M. Simonovits. 1997. Random walks and an O(n5) volume algorithm for convex bodies. Random Structures & Algorithms , Vol. 11, 1 (1997), 1--50. Google Scholar
Digital Library
- Ashish Khetan and Sewoong Oh. 2017. Matrix Norm Estimation from a Few Entries. In In Proc. of NIPS . Google Scholar
Digital Library
- David Asher Levin, Yuval Peres, and Elizabeth Lee Wilmer. 2009. Markov chains and mixing times .Providence, R.I. American Mathematical Society.Google Scholar
- L. Lovász. 1996. Random Walks on Graphs: A Survey. In Combinatorics, Paul ErdHos is Eighty, D. Miklós , V. T. Sós, and T. SzHonyi (Eds.). Vol. 2. János Bolyai Mathematical Society, Budapest, 353--398.Google Scholar
- L. Lovász. 1999. Hit-and-run mixes fast. Mathematical Programming , Vol. 86, 3 (1999), 443--461.Google Scholar
Cross Ref
- L. Lovász and S. Vempala. 2006. Hit-and-Run from a Corner. SIAM J. Comput. , Vol. 35, 4 (2006), 985--1005. Google Scholar
Digital Library
- M. Mezard and A. Montanari. 2006. Information, Physics and Computation .Oxford Graduate Texts. Google Scholar
Digital Library
- Gary W Oehlert. 1992. A note on the delta method. American Statistician , Vol. 46, 1 (1 1 1992), 27--29.Google Scholar
- J. Propp and D. Wilson. 1998. Coupling from the past: a user's guide. DIMACS Ser. Discrete Math. Theoret. Comput. Sci. (1998), 181--192.Google Scholar
- Qian Qin, James P. Hobert, and Kshitij Khare. 2017. Estimating the Spectral Gap of a trace-class Markov operator. https://arxiv.org/abs/1704.00850 (2017).Google Scholar
- Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri. 2007. Numerical Mathematics . Texts in Applied Mathematics, Vol. 37. Springer-Verlag Berlin Heidelberg. Google Scholar
Digital Library
- Kong Weihao and Gregory Valiant. 2017. Spectrum estimation from samples. The Annals of Statistics , Vol. 45, 5 (2017).Google Scholar
Index Terms
Computationally Efficient Estimation of the Spectral Gap of a Markov Chain
Recommendations
Analyzing Markov chain Monte Carlo output
AbstractMarkov chain Monte Carlo (MCMC) is a sampling‐based method for estimating features of probability distributions. MCMC methods produce a serially correlated, yet representative, sample from the desired distribution. As such it can be difficult to ...
Visual representation of the multivariate correlation structure in a Markov chain. Output analysis that accounts for this multivariate structure is able to more accurately summarize variability in the simulation process. We review the output analysis ...
Segmentation Based on Improved Markov Chain Monte Carlo
ICECC '12: Proceedings of the 2012 International Conference on Electronics, Communications and ControlTo overcome the drawback of Simulated Annealing (SA), a novel algorithm Using Population based Markov chain Monte Carlo (Pop-MCMC) is proposed for segmentation. It takes less time from the initial state to the state that chains are coupling with Pop-...
Optimal Markov chain Monte Carlo sampling
This article is a review article on the optimal Markov chain Monte Carlo MCMC sampling. The focus is on homogeneous Markov chains. This article first reviews the problem of finding the optimal transition matrix, which is defined to minimize the ...






Comments