Fast particle smoothing: if I had a million particles

ABSTRACT

We propose efficient particle smoothing methods for generalized state-spaces models. Particle smoothing is an expensive O(N²) algorithm, where N is the number of particles. We overcome this problem by integrating dual tree recursions and fast multipole techniques with forward-backward smoothers, a new generalized two-filter smoother and a maximum a posteriori (MAP) smoother. Our experiments show that these improvements can substantially increase the practicality of particle smoothing.

References

1. Bresler, Y. (1986). Two-filter formula for discrete-time non-linear Bayesian smoothing. IJC, 43, 629--641.
2. Briers, M., Doucet, A., & Maskell, S. R. (2004). Smoothing algorithms for state-space models (Technical Report CUED/F-INFENG/TR.498). Cambridge University Engineering Department.
3. Cemgil, A., & Kappen, H. (2003). Monte Carlo methods for tempo tracking and rhythm quantization. JAIR, 18, 45--81.
4. Doucet, A., de Freitas, N., & Gordon, N. J. (Eds.). (2001). Sequential Monte Carlo methods in practice. Springer-Verlag.
5. Felzenswalb, P. F., Huttenlocher, D. P., & Kleinberg, J. M. (2003). Fast Algorithms for Large-State-Space HMMs with Application to Web Usage Analysis. NIPS 16.
6. Fraser, D. C., & Potter, J. E. (1969). The optimum linear smoother as a combination of two optimum linear filters. IEEE Transactions on Automatic Control, 987--390.
7. Godsill, S. J., Doucet, A., & West, M. (2001). Maximum a posteriori sequence estimation using Monte Carlo particle filters. Ann. Inst. Stat. Math., 53, 82--96.
8. Gray, A., & Moore, A. (2000). 'N-Body' Problems in Statistical Learning. NIPS 4 (pp. 521--527).
9. Gray, A., & Moore, A. (2003). Rapid evaluation of multiple density models. Artificial Intelligence and Statistics.
10. Greengard, L., & Rokhlin, V. (1987). A fast algorithm for particle simulations. JCP, 73, 325--348.
11. Greengard, L., & Sun, X. (1998). A new version of the Fast gauss transform. Doc. Math., ICM, 575--584.
12. Isard, M., & Blake, A. (1998). A smoothing filter for Condensation. ECCV (pp. 767--781). Freiburg, Germany.
13. Kim, S., Shephard, N., & Chib, S. (1998). Stochastic volatility: Likelihood inference and comparison with ARCH models. Review of Economic Studies, 65, 361--93.
14. Kitagawa, G. (1996). Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. JCGS, 5, 1--25.
15. Klaas, M., Lang, D., & de Freitas, N. (2005). Fast maximum a posteriori inference in Monte Carlo state spaces. Artificial Intelligence and Statistics.
16. Lang, D., & de Freitas, N. (2004). Beat tracking the graphical model way. NIPS. Cambridge, MA: MIT Press.
17. Mayne, D. Q. (1966). A solution of the smoothing problem for linear dynamic systems. Automatica, 4, 73--92.
18. Moore, A. (2000). The Anchors Hierarchy: Using the triangle inequality to survive high dimensional data (Technical Report CMU-RI-TR-00-05). Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.
19. Okuma, K., Taleghani, A., de Freitas, N., Little, J., & Lowe, D. (2004). A boosted particle filter: Multitarget detection and tracking. ECCV. Prague.
20. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan-Kaufmann.
21. Vermaak, J., Doucet, A., & Perez, P. (2003). Maintaining multi-modality through mixture tracking. ICCV.
22. Yang, C., Duraiswami, R., Gumerov, N. A., & Davis, L. S. (2003). Improved fast Gauss transform and efficient kernel density estimation. ICCV. Nice.

Index Terms

• Mathematics of computing

  • Probability and statistics

    • Probabilistic algorithms

    • Probabilistic reasoning algorithms

      • Markov-chain Monte Carlo methods

      • Sequential Monte Carlo methods

    • Statistical paradigms

      • Statistical graphics