Abstract
In this article, we present a fast and high quality stereo matching algorithm on FPGA using cost aggregation (CA) and fast locally consistent (FLC) dense stereo. In many software programs, global matching algorithms are used in order to obtain accurate disparity maps. Although their error rates are considerably low, their processing speeds are far from that required for real-time processing because of their complex processing sequences. In order to realize real-time processing, many hardware systems have been proposed to date. They have achieved considerably high processing speeds; however, their error rates are not as good as those of software programs, because simple local matching algorithms have been widely used in those systems. In our system, sophisticated local matching algorithms (CA and FLC) that are suitable for FPGA implementation are used to achieve low error rate while maintaining the high processing speed. We evaluate the performance of our circuit on Xilinx Vertex-6 FPGAs. Its error rate is comparable to that of top-level software algorithms, and its processing speed is nearly 2 clock cycles per pixel, which reaches 507.9 fps for 640 480 pixel images.
- Aaron F. Bobick and Stephen S. Intille. 1999. Large occlusion stereo. Int. J. Comput. Vis. 33, 3, 181--200. Google Scholar
Digital Library
- Yuri Boykov, Olga Veksler, and Ramin Zabih. 2001. Fast approximate energy minimization via graph cuts. Patt. Anal. Mach. Intell. 23, 11, 1222--1239. Google Scholar
Digital Library
- Jungwook Choi and Rob A. Rutenbar. 2012. Hardware implementation of MRF map inference on an FPGA platform. In Proceedings of the 22nd International Conference on Field Programmable Logic and Applications (FPL). 209--216.Google Scholar
- Ahmad Darabiha, W. James MacLean, and Jonathan Rose. 2006. Reconfigurable hardware implementation of a phase-correlation stereoalgorithm. Mach. Vis. Appl. 17, 2, 116--132. Google Scholar
Digital Library
- Leonardo De-Maeztu, Arantxa Villanueva, and Rafael Cabeza. 2011. Stereo matching using gradient similarity and locally adaptive support-weight. Patt. Recognit. Lett. 32, 13, 1643--1651. Google Scholar
Digital Library
- Leonardo De-Maeztu, Arantxa Villanueva, and Rafael Cabeza. 2012. Near real-time stereo matching using geodesic diffusion. IEEE Trans. Patt. Anal. Mach. Intell. 34, 2, 410--416. Google Scholar
Digital Library
- Michel Devy and Jean-louis Boizard. 2011. Stereovision algorithm to be executed at 100hz on a FPGA-based architecture. In Advances in Theory and Applications of Stereo Vision, Asim Bhatti Ed., Chapter 17.Google Scholar
- Jingting Ding, Jilin Liu, Wenhui Zhou, Haibin Yu, Yanchang Wang, and Xiaojin Gong. 2011. Real-time stereo vision system using adaptive weight cost aggregation approach. EURASIP J. Image Video Process 1, 20--38.Google Scholar
Cross Ref
- Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR). http://www.cvlibs.net/datasets/kitti/. Google Scholar
Digital Library
- S. Hadjitheophanous, C. Ttofis, A. S. Georghiades, and T. Theocharides. 2010. Towards hardware stereoscopic 3D reconstruction a real-time FPGA computation of the disparity map. In Proceedings of the Design Automation and Test in Europe Conference and Exhibition (DATE). 1743--1748. Google Scholar
Digital Library
- Asmaa Hosni and Michael Bleyer. 2009. Local stereo matching using geodesic support weights. In Proceedings of the 16th IEEE International Conference on Image Processing (ICIP’09). IEEE, 2093--2096. Google Scholar
Digital Library
- Minxi Jin and Tsutomu Maruyama. 2012a. A real-time stereo vision system using a tree-structured dynamic programming on FPGA. In Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA’12). ACM, New York, 21--24. Google Scholar
Digital Library
- Minxi Jin and Tsutomu Maruyama. 2012b. A fast and high quality stereo matching algorithm on FPGA. In Proceedings of the 22nd International Conference on Field Programmable Logic and Applications (FPL’12).Google Scholar
Cross Ref
- Seunghun Jin, Junguk Cho, Xuan Dai Pham, Kyoung Mu Lee, Sung-Kee Park, Munsang Kim, and Jae Wook Jeon. 2010. FPGA design and implementation of a real-time stereo vision system. IEEE Trans. Circ. Syst. Video Tech. 20, 1, 15--26. Google Scholar
Digital Library
- John Kalomiros and John Lygouras. 2009. Comparative study of local SAD and dynamic programming for stereo processing using dedicated hardware. EURASIP J. Adv. Sig. Process. 1--18.Google Scholar
- Andreas Klaus, Mario Sormann, and Konrad Karner. 2006. Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06). IEEE, 15--18. Google Scholar
Digital Library
- Cheng Lei, Jason Selzer, and Yee-Hong Yang. 2006. Region-tree based stereo using dynamic programming optimization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2, IEEE, 2378--2385. Google Scholar
Digital Library
- Martin D. Levine, Douglas A. O’Handley, and Gary M. Yagi. 1973. Computer determination of depth maps. Computer Graph. Image Process. 2, 2, 131--150.Google Scholar
Cross Ref
- Eduardo Magdaleno and Jonas P. Lüke. 2010. Design of belief propagation based on FPGA for the multistereo CAFADIS camera. Sensors (Basel Switzerland) 10, 10, 9194--9210.Google Scholar
Cross Ref
- Divyang K. Masrani and W. James MacLean. 2006. A real-time large disparity range stereo-system using FPGAs. In Proceedings of the 4th IEEE International Conference on Computer Vision Systems (ICVS’06). IEEE, 13--13. Google Scholar
Digital Library
- Stefano Mattoccia. 2010. Fast locally consistent dense stereo on multicore. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’10). IEEE, 1--8.Google Scholar
Cross Ref
- Xing Mei, Xun Sun, Mingcai Zhou, Shaohui Jiao, and Haitao Wang. 2011. On building an accurate stereo matching system on graphics hardware. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE, 467--474.Google Scholar
Cross Ref
- Yuichi Ohta and Takeo Kanade. 1985. Stereo by intra- and inter-scanline search using dynamic programming. IEEE Trans. Patt. Anal. Mach. Intell. 7, 2, 139--154. Google Scholar
Digital Library
- Jesús Pérez, Pablo Sánchez, and Marcos Martínez. 2009. High-definition belief-propagation based stereo matching FPGA architecture. In Proceedings of the 3rd International Conference on Signals, Circuits and Systems (SCS’09). 1--6.Google Scholar
- Daniel Scharstein and Richard Szeliski. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 1, 131--140. Google Scholar
Digital Library
- Daniel Scharstein and Richard Szeliski. 2003. High-accuracy stereo depth maps using structured light. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, I--195--I--202. Google Scholar
Digital Library
- Daniel Scharstein and Richard Szeliski. 2009. Middlebury stereo evaluation - version 2. http://vision.middlebury.edu/stereo/eval/.Google Scholar
- Olgierd Stankiewicz and Krzysztof Wegner. 2008. Depth map estimation software version 2. ISO/IEC JTC1/SC29/WG11 MPEG2008/M15338.Google Scholar
- Jian Sun, Nan-Ning Zheng, and Heung-Yeung Shum. 2003. Stereo matching using belief propagation. Patt. Anal. Mach. Intell. 25, 7, 787--800. Google Scholar
Digital Library
- Federico Tombari, Stefano Mattoccia, Luigi Di Stefano, and Elisa Addimanda. 2008. Classification and evaluation of cost aggregation methods for stereo correspondence. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.Google Scholar
Cross Ref
- Liang Wang, Miao Liao, Minglun Gong, Ruigang Yang, and David Nister. 2006. High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In Proceedings of the 3rd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’06). IEEE, 798--805. Google Scholar
Digital Library
- Qingxiong Yang, Liang Wang, Ruigang Yang, and Shengnan Wang. 2006. Real-time global stereo matching using hierarchical belief propagation. In Proceedings of the British Machine Vision Conference (BMVC). 1--10.Google Scholar
Cross Ref
- Lu Zhang, Ke Zhang, and Tian Sheuan Chang. 2011. Real-time high-definition stereo matching on FPGA. In Proceedings of the ACM/SIGDA 19th International Symposium on Field Programmable Gate Arrays. ACM, New York, 55--64. Google Scholar
Digital Library
Index Terms
Fast and Accurate Stereo Vision System on FPGA
Recommendations
Hardware Acceleration for an Accurate Stereo Vision System Using Mini-Census Adaptive Support Region
Special Issue on Real-Time and Embedded Technology and Applications, Domain-Specific Multicore Computing, Cross-Layer Dependable Embedded Systems, and Application of Concurrency to System Design (ACSD'13)Domain of stereo vision is highly important in the fields of autonomous cars, video tolling, robotics, and aerial surveys. The specific feature of this domain is that we should handle not only the pixel-by-pixel 2D processing in one image but also the ...
A real-time stereo vision system using a tree-structured dynamic programming on FPGA
FPGA '12: Proceedings of the ACM/SIGDA international symposium on Field Programmable Gate ArraysMany hardware systems for stereo vision have been proposed. Their processing speed is very fast, but the algorithms used in them are limited in order to achieve the high processing speed by simplifying the sequences of the memory accesses and ...
VHDL description of a synthetizable and reconfigurable real-time stereo vision processor
ISPRA'05: Proceedings of the 4th WSEAS International Conference on Signal Processing, Robotics and AutomationThis paper describes a reconfigurable digital architecture to compute dense disparity maps at video-rate for stereo vision. The processor architecture is described in synthetizable VHDL and, by means of the reconfigurability, the hardware requirements ...






Comments