skip to main content
research-article

Fast and Accurate Stereo Vision System on FPGA

Published:01 February 2014Publication History
Skip Abstract Section

Abstract

In this article, we present a fast and high quality stereo matching algorithm on FPGA using cost aggregation (CA) and fast locally consistent (FLC) dense stereo. In many software programs, global matching algorithms are used in order to obtain accurate disparity maps. Although their error rates are considerably low, their processing speeds are far from that required for real-time processing because of their complex processing sequences. In order to realize real-time processing, many hardware systems have been proposed to date. They have achieved considerably high processing speeds; however, their error rates are not as good as those of software programs, because simple local matching algorithms have been widely used in those systems. In our system, sophisticated local matching algorithms (CA and FLC) that are suitable for FPGA implementation are used to achieve low error rate while maintaining the high processing speed. We evaluate the performance of our circuit on Xilinx Vertex-6 FPGAs. Its error rate is comparable to that of top-level software algorithms, and its processing speed is nearly 2 clock cycles per pixel, which reaches 507.9 fps for 640 480 pixel images.

References

  1. Aaron F. Bobick and Stephen S. Intille. 1999. Large occlusion stereo. Int. J. Comput. Vis. 33, 3, 181--200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Yuri Boykov, Olga Veksler, and Ramin Zabih. 2001. Fast approximate energy minimization via graph cuts. Patt. Anal. Mach. Intell. 23, 11, 1222--1239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Jungwook Choi and Rob A. Rutenbar. 2012. Hardware implementation of MRF map inference on an FPGA platform. In Proceedings of the 22nd International Conference on Field Programmable Logic and Applications (FPL). 209--216.Google ScholarGoogle Scholar
  4. Ahmad Darabiha, W. James MacLean, and Jonathan Rose. 2006. Reconfigurable hardware implementation of a phase-correlation stereoalgorithm. Mach. Vis. Appl. 17, 2, 116--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Leonardo De-Maeztu, Arantxa Villanueva, and Rafael Cabeza. 2011. Stereo matching using gradient similarity and locally adaptive support-weight. Patt. Recognit. Lett. 32, 13, 1643--1651. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Leonardo De-Maeztu, Arantxa Villanueva, and Rafael Cabeza. 2012. Near real-time stereo matching using geodesic diffusion. IEEE Trans. Patt. Anal. Mach. Intell. 34, 2, 410--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Michel Devy and Jean-louis Boizard. 2011. Stereovision algorithm to be executed at 100hz on a FPGA-based architecture. In Advances in Theory and Applications of Stereo Vision, Asim Bhatti Ed., Chapter 17.Google ScholarGoogle Scholar
  8. Jingting Ding, Jilin Liu, Wenhui Zhou, Haibin Yu, Yanchang Wang, and Xiaojin Gong. 2011. Real-time stereo vision system using adaptive weight cost aggregation approach. EURASIP J. Image Video Process 1, 20--38.Google ScholarGoogle ScholarCross RefCross Ref
  9. Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR). http://www.cvlibs.net/datasets/kitti/. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Hadjitheophanous, C. Ttofis, A. S. Georghiades, and T. Theocharides. 2010. Towards hardware stereoscopic 3D reconstruction a real-time FPGA computation of the disparity map. In Proceedings of the Design Automation and Test in Europe Conference and Exhibition (DATE). 1743--1748. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Asmaa Hosni and Michael Bleyer. 2009. Local stereo matching using geodesic support weights. In Proceedings of the 16th IEEE International Conference on Image Processing (ICIP’09). IEEE, 2093--2096. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Minxi Jin and Tsutomu Maruyama. 2012a. A real-time stereo vision system using a tree-structured dynamic programming on FPGA. In Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA’12). ACM, New York, 21--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Minxi Jin and Tsutomu Maruyama. 2012b. A fast and high quality stereo matching algorithm on FPGA. In Proceedings of the 22nd International Conference on Field Programmable Logic and Applications (FPL’12).Google ScholarGoogle ScholarCross RefCross Ref
  14. Seunghun Jin, Junguk Cho, Xuan Dai Pham, Kyoung Mu Lee, Sung-Kee Park, Munsang Kim, and Jae Wook Jeon. 2010. FPGA design and implementation of a real-time stereo vision system. IEEE Trans. Circ. Syst. Video Tech. 20, 1, 15--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. John Kalomiros and John Lygouras. 2009. Comparative study of local SAD and dynamic programming for stereo processing using dedicated hardware. EURASIP J. Adv. Sig. Process. 1--18.Google ScholarGoogle Scholar
  16. Andreas Klaus, Mario Sormann, and Konrad Karner. 2006. Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06). IEEE, 15--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Cheng Lei, Jason Selzer, and Yee-Hong Yang. 2006. Region-tree based stereo using dynamic programming optimization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2, IEEE, 2378--2385. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Martin D. Levine, Douglas A. O’Handley, and Gary M. Yagi. 1973. Computer determination of depth maps. Computer Graph. Image Process. 2, 2, 131--150.Google ScholarGoogle ScholarCross RefCross Ref
  19. Eduardo Magdaleno and Jonas P. Lüke. 2010. Design of belief propagation based on FPGA for the multistereo CAFADIS camera. Sensors (Basel Switzerland) 10, 10, 9194--9210.Google ScholarGoogle ScholarCross RefCross Ref
  20. Divyang K. Masrani and W. James MacLean. 2006. A real-time large disparity range stereo-system using FPGAs. In Proceedings of the 4th IEEE International Conference on Computer Vision Systems (ICVS’06). IEEE, 13--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Stefano Mattoccia. 2010. Fast locally consistent dense stereo on multicore. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’10). IEEE, 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  22. Xing Mei, Xun Sun, Mingcai Zhou, Shaohui Jiao, and Haitao Wang. 2011. On building an accurate stereo matching system on graphics hardware. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE, 467--474.Google ScholarGoogle ScholarCross RefCross Ref
  23. Yuichi Ohta and Takeo Kanade. 1985. Stereo by intra- and inter-scanline search using dynamic programming. IEEE Trans. Patt. Anal. Mach. Intell. 7, 2, 139--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jesús Pérez, Pablo Sánchez, and Marcos Martínez. 2009. High-definition belief-propagation based stereo matching FPGA architecture. In Proceedings of the 3rd International Conference on Signals, Circuits and Systems (SCS’09). 1--6.Google ScholarGoogle Scholar
  25. Daniel Scharstein and Richard Szeliski. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 1, 131--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Daniel Scharstein and Richard Szeliski. 2003. High-accuracy stereo depth maps using structured light. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, I--195--I--202. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Daniel Scharstein and Richard Szeliski. 2009. Middlebury stereo evaluation - version 2. http://vision.middlebury.edu/stereo/eval/.Google ScholarGoogle Scholar
  28. Olgierd Stankiewicz and Krzysztof Wegner. 2008. Depth map estimation software version 2. ISO/IEC JTC1/SC29/WG11 MPEG2008/M15338.Google ScholarGoogle Scholar
  29. Jian Sun, Nan-Ning Zheng, and Heung-Yeung Shum. 2003. Stereo matching using belief propagation. Patt. Anal. Mach. Intell. 25, 7, 787--800. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Federico Tombari, Stefano Mattoccia, Luigi Di Stefano, and Elisa Addimanda. 2008. Classification and evaluation of cost aggregation methods for stereo correspondence. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  31. Liang Wang, Miao Liao, Minglun Gong, Ruigang Yang, and David Nister. 2006. High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In Proceedings of the 3rd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’06). IEEE, 798--805. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Qingxiong Yang, Liang Wang, Ruigang Yang, and Shengnan Wang. 2006. Real-time global stereo matching using hierarchical belief propagation. In Proceedings of the British Machine Vision Conference (BMVC). 1--10.Google ScholarGoogle ScholarCross RefCross Ref
  33. Lu Zhang, Ke Zhang, and Tian Sheuan Chang. 2011. Real-time high-definition stereo matching on FPGA. In Proceedings of the ACM/SIGDA 19th International Symposium on Field Programmable Gate Arrays. ACM, New York, 55--64. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Fast and Accurate Stereo Vision System on FPGA

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Reconfigurable Technology and Systems
          ACM Transactions on Reconfigurable Technology and Systems  Volume 7, Issue 1
          February 2014
          117 pages
          ISSN:1936-7406
          EISSN:1936-7414
          DOI:10.1145/2589584
          • Editor:
          • Steve Wilton
          Issue’s Table of Contents

          Copyright © 2014 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 1 February 2014
          • Accepted: 1 October 2013
          • Revised: 1 June 2013
          • Received: 1 December 2012
          Published in trets Volume 7, Issue 1

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!