skip to main content
research-article
Public Access

RISE: An Automated Framework for Real-Time Intelligent Video Surveillance on FPGA

Published:27 September 2017Publication History
Skip Abstract Section

Abstract

This paper proposes RISE, an automated Reconfigurable framework for real-time background subtraction applied to Intelligent video SurveillancE. RISE is devised with a new streaming-based methodology that adaptively learns/updates a corresponding dictionary matrix from background pixels as new video frames are captured over time. This dictionary is used to highlight the foreground information in each video frame. A key characteristic of RISE is that it adaptively adjusts its dictionary for diverse lighting conditions and varying camera distances by continuously updating the corresponding dictionary. We evaluate RISE on natural-scene vehicle images of different backgrounds and ambient illuminations. To facilitate automation, we provide an accompanying API that can be used to deploy RISE on FPGA-based system-on-chip platforms. We prototype RISE for end-to-end deployment of three widely-adopted image processing tasks used in intelligent transportation systems: License Plate Recognition (LPR), image denoising/reconstruction, and principal component analysis. Our evaluations demonstrate up to 87-fold higher throughput per energy unit compared to the prior-art software solution executed on ARM Cortex-A15 embedded platform.

References

  1. V. Cevher, A. Sankaranarayanan, M. F. Duarte, D. Reddy, R. G. Baraniuk, and R. Chellappa. 2008. Compressive sensing for background subtraction. 155--168.Google ScholarGoogle Scholar
  2. D. Schreiber and M. Rauter. 2009. GPU-based non-parametric background subtraction for a practical surveillance system. 870--877.Google ScholarGoogle Scholar
  3. F. Porikli. 2006. Achieving real-time object detection and tracking under extreme conditions. Journal of Real-Time Image Processing 1, 1, 33--40.Google ScholarGoogle ScholarCross RefCross Ref
  4. J. Oliveira, A. Printes, R. Freire, E. Melcher, and I.S. Silva. 2006. FPGA architecture for static background subtraction in real time. 26--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. C. Sanchez-Ferreira J. Mori, and C. Llanos. 2012. Background subtraction algorithm for moving object detection in FPGA. 1--6.Google ScholarGoogle Scholar
  6. C. Lu, J. Shi, and J. Jia. 2013. Online robust dictionary learning. 415--422. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. E. J. Candès and M. B. Wakin. 2008. An introduction to compressive sampling. IEEE signal processing magazine, 25, 2, 21--30.Google ScholarGoogle Scholar
  8. J. Tropp, A. C. Gilbert et al. 2007. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53, 12, 4655--4666. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. Mirhoseini, E. Dyer, E. Songhori, R. Baraniuk, and F. Koushanfar. 2015. Rankmap: A platform-aware framework for distributed learning from dense datasets. arXiv preprint arXiv:1503.08169.Google ScholarGoogle Scholar
  10. A. Mirhoseini, B. D. Rouhani, E. M. Songhori, and F. Koushanfar. 2016. Perform-ml: Performance optimized machine learning by platform and content aware customization. 20.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Sauvola and M. Pietikäinen. 2000. Adaptive document image binarization. Pattern recognition, 33, 2, 225--236.Google ScholarGoogle Scholar
  12. M. Karkooti, J. R. Cavallaro, and C. Dick. 2005. FPGA implementation of matrix inversion using qrd-rls algorithm.Google ScholarGoogle Scholar
  13. Å. Björck. 1967. Solving linear least squares problems by gram-schmidt orthogonalization. BIT Numerical Mathematics, 7, 1, 1--21.Google ScholarGoogle ScholarCross RefCross Ref
  14. W. Hoffmann. 1989. Iterative algorithms for gram-schmidt orthogonalization. Computing, 41, 4, 335--348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. XILLYBUS. 2017. http://xillybus.com/.Google ScholarGoogle Scholar
  16. XPower. 2012. {Online}. Available: http://www.xilinx.com/support/documentation/user_guides/ug440.pdf.Google ScholarGoogle Scholar
  17. Rise source codes. https://github.com/Bitadr/RISE.Google ScholarGoogle Scholar
  18. Y. Wen, Y. Lu, J. Yan, Z. Zhou, K. M. Von Deneen, and P. Shi. 2011. An algorithm for license plate recognition applied to intelligent transportation system. IEEE Transactions on Intelligent Transportation Systems, 12, 3, 830--845. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. C. N. E. Anagnostopoulos, I. E. Anagnostopoulos, V. Loumos, and E. Kayafas. 2006. A license plate-recognition algorithm for intelligent transportation system applications. Intelligent Transportation Systems, IEEE Transactions on, 7, 3, 377--392. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. C. Arth, H. Bischof, and C. Leistner. 2006. Tricam-an embedded platform for remote traffic surveillance. 125--125. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. C.-N. E. Anagnostopoulos, I. E. Anagnostopoulos, I. D. Psoroulas, V. Loumos, and E. Kayafas. 2008. License plate recognition from still images and video sequences: A survey. IEEE Transactions on intelligent transportation systems, 9, 3, 377--391. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. Elad and M. Aharon. 2006. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing, 15, 12, 3736--3745. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. S. D. A. LightField. 2014. {Online}. Available: http://lightfield.stanford.edu/.Google ScholarGoogle Scholar
  24. H. R. S. D. Salina. 2014. {Online}. Available: http://www.ehu.es/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.Google ScholarGoogle Scholar
  25. LPR database. 2017. http://www.medialab.ntua.gr/research/LPRdatabase/Still_images/difficult_cases/.Google ScholarGoogle Scholar
  26. Video surveillance frame rate. 2016. http://ipvm.com/reports/.Google ScholarGoogle Scholar
  27. B. Rouhani, E. Songhori, A. Mirhoseini, and F. Koushanfar. 2015. Ssketch: An automated framework for streaming sketch-based analysis of big data on FPGA. In 23rd International Symposium on Field-Programmable Custom Computing Machines conference (FCCM). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. M. Andrecut. 2008. Fast GPU implementation of sparse signal recovery from random projections. arXiv preprint arXiv:0809.1833.Google ScholarGoogle Scholar
  29. J. D. Blanchard and J. Tanner. 2013. GPU accelerated greedy algorithms for compressed sensing. Mathematical Programming Computation, 5, 3, 267--304.Google ScholarGoogle ScholarCross RefCross Ref
  30. Y. Fang, L. Chen, J. Wu, and B. Huang. 2011. GPU implementation of orthogonal matching pursuit for compressive sensing. 1044--1047. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. P. Maechler, P. Greisen, N. Felber, and A. Burg. 2010. Matching pursuit: Evaluation and implementatio for LTE channel estimation. 589--592.Google ScholarGoogle Scholar
  32. A. Septimus and R. Steinberg. 2010. Compressive sampling hardware reconstruction. 3316--3319.Google ScholarGoogle Scholar
  33. L. Bai, P. Maechler, M. Muehlberghuber, and H. Kaeslin. 2012. High-speed compressed sensing reconstruction on FPGA using OMP and AMP. 53--56.Google ScholarGoogle Scholar
  34. J. L. Stanislaus and T. Mohsenin. 2013. Low-complexity FPGA implementation of compressive sensing reconstruction. 671--675. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. A. M. Kulkarni, H. Homayoun, and T. Mohsenin. 2014. A parallel and reconfigurable architecture for efficient OMP compressive sensing reconstruction. 299--304. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. T. Hosaka, T. Kobayashi, and N. Otsu. 2011. Object detection using background subtraction and foreground motion estimation. IPSJ Transactions on Computer Vision and Applications, 3, 9--20.Google ScholarGoogle ScholarCross RefCross Ref
  37. P. Viola, M. J. Jones, and D. Snow. 2005. Detecting pedestrians using patterns of motion and appearance. International Journal of Computer Vision, 63, 2, 153--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. K. Mikolajczyk and C. Schmid. 2005. A performance evaluation of local descriptors. IEEE transactions on pattern analysis and machine intelligence, 27, 10, 1615--1630. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. RISE: An Automated Framework for Real-Time Intelligent Video Surveillance on FPGA

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!