skip to main content
research-article

Large-Area, Multilayered, and High-Resolution Visual Monitoring Using a Dual-Camera System

Published:07 January 2015Publication History
Skip Abstract Section

Abstract

Large-area, high-resolution visual monitoring systems are indispensable in surveillance applications. To construct such systems, high-quality image capture and display devices are required. Whereas high-quality displays have rapidly developed, as exemplified by the announcement of the 85-inch 4K ultrahigh-definition TV by Samsung at the 2013 Consumer Electronics Show (CES), high-resolution surveillance cameras have progressed slowly and remain not widely used compared with displays. In this study, we designed an innovative framework, using a dual-camera system comprising a wide-angle fixed camera and a high-resolution pan-tilt-zoom (PTZ) camera to construct a large-area, multilayered, and high-resolution visual monitoring system that features multiresolution monitoring of moving objects. First, we developed a novel calibration approach to estimate the relationship between the two cameras and calibrate the PTZ camera. The PTZ camera was calibrated based on the consistent property of distinct pan-tilt angle at various zooming factors, accelerating the calibration process without affecting accuracy; this calibration process has not been reported previously. After calibrating the dual-camera system, we used the PTZ camera and synthesized a large-area and high-resolution background image. When foreground targets were detected in the images captured by the wide-angle camera, the PTZ camera was controlled to continuously track the user-selected target. Last, we integrated preconstructed high-resolution background and low-resolution foreground images captured using the wide-angle camera and the high-resolution foreground image captured using the PTZ camera to generate a large-area, multilayered, and high-resolution view of the scene.

References

  1. A. Alahi, D. Marimon, M. Bierlaire, and M. Kunt. 2008. A master-slave approach for object detection and matching with fixed and mobile cameras. In Proceedings of the 15th IEEE International Conference on Image Processing (ICIP). 1712--1715.Google ScholarGoogle Scholar
  2. M. Andriluka, S. Roth, and B. Schiele. 2008. People-tracking-by-detection and people-detection-by-tracking. In Proceedings of the 12th IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). 1511--1522.Google ScholarGoogle Scholar
  3. M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. 2002. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50, 2, 174--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. O. Barnich and M. Van Droogenbroeck. 2011. ViBe: A universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20, 6,1709--1724. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. P. Baudisch, N. Good, V. Bellotti, and P. Schraedley. 2002. Keeping things in context: A comparative evaluation of focus plus context screens, overviews, and zooming. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). 259--266. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. P. Baudisch, N. Good, and P. Stewart. 2001. Focus plus context screens: Combining display technology with visualization techniques. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST). 31--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. 1997. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 7, 711--720. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Brown and D. G. Lowe. 2003. Recognising panoramas. In Proceedings of the 9th IEEE International Conference on Computer Vision (ICCV). 1218--1227. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Y. Cai, G. G. Medioni, and T. B. Dinh. 2013. Towards a practical PTZ face detection and tracking system. In Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV). 31--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. K.-W. Chen, C.-W. Lin, T.-H. Chiu, M. Y.-Y. Chen, and Y.-P. Hung. 2011. Multi-Resolution Design for Large-Scale and High-Resolution Monitoring. IEEE Trans. Multimedia. 13, 6, 1256--1268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. K.-W. Chen, C.-W. Lin, M. Y.-Y. Chen, and Y.-P. Hung. 2010. e-Fovea: A multi-resolution approach with steerable focus to large-scale and high-resolution monitoring. In Proceedings of the ACM International Conference on Multimedia (MM). 311--320. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. C. C. Chen, Y. Yao, A. Drira, A. Koschan, and M. Abidi. 2009. Cooperative mapping of multiple PTZ cameras in automated surveillance systems. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1078--1084Google ScholarGoogle Scholar
  13. C. H. Chen, Y. Yao, D. Page, and B. Abidi. 2008. Heterogeneous fusion of omnidirectional and PTZ cameras for multiple object tracking. IEEE Trans. Circuits Syst. Video Technol. 18, 8, 1052--1063. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. F. Dornaika and J. H. Elder. 2012. Image registration for foveated panoramic sensing. ACM Trans. Multimedia Comput. Commun. Appl. 8, 2, 1--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. H Elder, F. Dornaika, Y. Hou, and R. Goldstein. 2005. Attentive Wide-Field Sensing for Visual Telepresence and Surveillance. Academic Press. Elsevier, San Diego, CA.Google ScholarGoogle Scholar
  16. J. H. Elder, S. J. Prince, Y. Hou, M. Sizintsev, and E. Olevskiy. 2007. Pre-attentive and attentive detection of humans in wide-field scenes. Int. J. Comput. Vis. 72, 1, 47--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Ess, B. Leibe, K. Schindler, and L. Van Gool. 2008. A mobile vision system for robust multi-person tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1--8.Google ScholarGoogle Scholar
  18. N. Gracias, M. Mahoor, S. Negahdaripour, and A. Gleason. 2009. Fast image blending using watersheds and graph cuts. Image Vis. Comput. 27, 5, 597--607. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. T. T. Hu, Y. W. Chia, L. W. Chan, Y. P. Hung, and J. Hsu. 2008. im-Top: An interactive multi- resolution tabletop system accommodating to multi-resolution human vision. 2008. In Proceedings of the 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems. 177--180.Google ScholarGoogle Scholar
  20. X. Kang, G. Ogunmakin, L. Yue, P. A. Vela, and W. Yongtian. 2011. PTZ camera-based adaptive panoramic and multi-layered background model. In Proceedings of the 18th IEEE International Conference on Image Processing (ICIP). 2949--2952.Google ScholarGoogle Scholar
  21. X. Kang, L. Yue, O. Gbolabo, C. Jing, and Z. Jiangen. 2013. Panoramic Gaussian Mixture Model and large-scale range background subtraction method for PTZ camera-based surveillance systems. Mach. Vis. Appl. 24, 3, 477--492.Google ScholarGoogle ScholarCross RefCross Ref
  22. K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis. 2005. Real-time foreground-background segmentation using codebook model. Real-Time Imaging. 11, 3, 172--185. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Lalonde, S. Foucher, L. Gagnon, E. Pronovost, M. Derenne, and A. Janelle. 2007. A system to automatically track humans and vehicles with a PTZ camera. In Proceedings of SPIE 6575, Visual Information Processing XVI.Google ScholarGoogle Scholar
  24. D. G. Lowe. 1999. Object recognition from local scale-invariant features. In Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV). 1150--1157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. L. Marchesotti, L. Marcenaro, and C. Regazzoni. 2003. Dual camera system for face detection in unconstrained environments. In Proceedings of the IEEE International Conference on Image Processing (ICIP). 681--684.Google ScholarGoogle Scholar
  26. A. Mian. 2008. Realtime face detection and tracking using a single Pan, Tilt, Zoom camera. In Proceedings of the 23rd International Conference on Image and Vision Computing New Zealand (IVCNZ). 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  27. C. Micheloni, G. Foresti, and L. Snidaro. 2005. A network of co-operative cameras for visual surveillance. IEE Proc. Vis. Image Signal Proc. 152, 2, 205--212.Google ScholarGoogle ScholarCross RefCross Ref
  28. C. Micheloni, B. Rinner, and G. L. Foresti. 2010. Video analysis in pan-tilt-zoom camera networks. IEEE Signal Proc. Mag. 27, 5, 78--90.Google ScholarGoogle ScholarCross RefCross Ref
  29. C. Micheloni, E. Salvador, F. Bigaran, and G. L. Foresti. 2005. An integrated surveillance system for outdoor security. In Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS). 480--485.Google ScholarGoogle Scholar
  30. P. Natarajan, T. N. Hoang, K. H. Low, and M. Kankanhalli. 2012. Decision-theoretic approach to maximizing observation of multiple targets in multi-camera surveillance. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMS). 155--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. P. Natarajan, T. N. Hoang, K. H. Low, and M. Kankanhalli. 2012. Decision-theoretic coordination and control for active multi-camera surveillance in uncertain, partially observable environments. In Proceedings of the Sixth IEEE International Conference on Distributed Smart Cameras (ICDSC), 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. P. Pérez, M. Gangnet, and A. Blake. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. PENPOWER 2012. TrackIN iDVR, Auto PTZ Tracking System. http://www.penpower.com.hk/product.asp?sn=61.Google ScholarGoogle Scholar
  34. C. Plaisant, D. Carr, and B. Shneiderman. 1995. Image-browser taxonomy and guidelines for designers. IEEE Softw. 12, 2, 21--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. S. Prince, J. H. Elder, Y. Hou, and M. Sizintsev. 2005. Pre-attentive face detection for foveated wide-field surveillance. In Proceedings of the IEEE Workshop on Applications on Computer Vision (WACV). 439--446. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. W. Qing, C. Feng, X. Wenli, and Y. Ming-Hsuan. 2012. Object tracking via partial least squares analysis. IEEE Trans. Image Process. 21, 10, 4454--4465. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Reale, T. Hung, and L. Yin. 2010. Pointing with the eyes: Gaze estimation using a static/active camera system and 3d iris disk model. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). 280--285.Google ScholarGoogle Scholar
  38. W. Ruiping, S. Shiguang, C. Xilin, D. Qionghai, and G. Wen. 2012. Manifold-manifold distance and its application to face recognition with image sets. IEEE Trans. Image Proc. 21, 10, 4466--4479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. G. Sansoni, M. Carocci, and R. Rodella. 1999. Three-dimensional vision based on a combination of gray-code and phase-shift light projection: Analysis and compensation of the systematic errors. Appl. Optics. 38, 31, 6565--6573.Google ScholarGoogle ScholarCross RefCross Ref
  40. V. K. Singh and P. K. Atrey. 2005. Coopetitive visual surveillance using model predictive control. In Proceedings of the 3rd ACM International Workshop on Video Surveillance and Sensor Networks. 149--158. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. V. K. Singh, P. K. Atrey, and M. S. Kankanhalli. 2008. Coopetitive multi-camera surveillance using model predictive control. Mach. Vis. Appl. 19, 5--6, 375--393. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. N. Sinhas and M. Pollefeys. 2006. Pan-tilt-zoom camera calibration and high-resolution mosaic generation. Comput. Vis. Image Understand. 103, 3, 170--183 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. O. G. Staadt, B. A. Ahlborn, O. Kreylos, and B. Hamann. 2006. A foveal inset for large display environments. In Proceedings of the ACM International Conference on Virtual Reality Continuum and its Applications (VRCIA). 281--288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. C. Stauffer and W. E. L. Grimson. 1999. Adaptive background mixture models for real-time tracking. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarGoogle ScholarCross RefCross Ref
  45. C. Stauffer and W. E. L. Grimson. 2000. Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22, 8, 747--757. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. R. Szeliski. 2004. Image alignment and stitching: A tutorial, Tech. Rep., MSR-TR-2004--2092.Google ScholarGoogle Scholar
  47. Z. Tao and R. Nevatia. 2002. Stochastic human segmentation from a static camera. In Proceedings of the Workshop on Motion and Video Computing. 9--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Z. Tao and R. Nevatia. 2003. Bayesian human segmentation in crowded situations. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). 459--466.Google ScholarGoogle Scholar
  49. P. Viola and M. Jones. 2001. Robust real-time object detection. Int. J. Comput. Vis. 4, 34--47.Google ScholarGoogle Scholar
  50. X. Wang, T. X. Han, and S. Yan. 2009. An HOG-LBP human detector with partial occlusion handling. In Proceedings of the IEEE 12th International Conference on Computer Vision (ICCV). 32--39.Google ScholarGoogle Scholar
  51. F. W. Wheeler, R. L. Weiss, and P. H. Tu. 2010. Face recognition at a distance system for surveillance applications. In Proceedings of the 4th IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS). 1--8.Google ScholarGoogle Scholar
  52. Y. Ye, J. K. Tsotsos, E. Harley, and K. Bennet. 2000. Tracking a person with pre-recorded image database and a pan, tilt, and zoom camera. Mach. Vis. Appl. 12, 1, 32--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. C. Zhang, Z. Liu, Z. Zhang, and Q. Zhao. 2008. Semantic saliency driven camera control for personal remote collaboration. In Proceedings of the IEEE 10th Workshop on Multimedia Signal Processing. 28--33.Google ScholarGoogle Scholar
  54. W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld. 2003. Face recognition: A literature survey. ACM Comput. Surv. 35, 4, 399--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. X. Zhou, R. T. Collins, T. Kanade, and P. Metes. 2003. A master-slave system to acquire biometric imagery of humans at distance. In Proceedings of the Ist ACM SIGMM International Workshop on Video Surveillance. 113--120. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Large-Area, Multilayered, and High-Resolution Visual Monitoring Using a Dual-Camera System

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!