skip to main content
research-article

The Cameraman Operating My Virtual Camera is Artificial: Can the Machine Be as Good as a Human?

Published:02 June 2015Publication History
Skip Abstract Section

Abstract

In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.

References

  1. Adel Ahmed and Peter Eades. 2005. Automatic camera path generation for graph navigation in 3D. In Proceedings of the Asia-Pacific Symposium on Information Visualisation. 27--32. http://dl.acm.org/citation.cfm?id&equal;1082315.1082320 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Y. Ariki, S. Kubota, and M. Kumano. 2006. Automatic production system of soccer sports video by digital camera work based on situation recognition. In Proceedings of the IEEE International Symposium on Multimedia. 851--860. DOI:http://dx.doi.org/10.1109/ISM.2006.37 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Peter Carr and Richard Hartley. 2009. Portable multi-megapixel camera with real-time recording and playback. In Proceedings of the Conference on Digital Image Computing: Techniques and Applications. 74--80. DOI:http://dx.doi.org/10.1109/DICTA.2009.62 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Peter Carr, Michael Mistry, and Iain Matthews. 2013. Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording. In Proceedings of the ACM Multimedia Conference. 193--202. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Joel Carranza, Christian Theobalt, Marcus A. Magnor, and Hans-Peter Seidel. 2003. Free viewpoint video of human actors. ACM Trans. Graph. 22, 3, 569--577. DOI:http://dx.doi.org/10.1145/882262.882309 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Fan Chen and Christophe De Vleeschouwer. 2010. Personalized production of basketball videos from multisensored data under limited display resolution. Computer Vision Image Understanding 114, 6, 667--680. DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.005 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Kuan-Ta Chen, Chen-Chi Wu, Yu-Chun Chang, and Chin-Laung Lei. 2009. A crowd-sourceable QoE evaluation framework for multimedia content. In Proceedings of the ACM Multimedia Conference. 491--500. DOI:http://dx.doi.org/10.1145/1631272.1631339 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Shenchang Eric Chen. 1995. QuickTime VR: An image-based approach to virtual environment navigation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 29--38. DOI:http://dx.doi.org/10.1145/218380.218395 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Marc Christie, Rumesh Machap, Jean-Marie Normand, Patrick Olivier, and Jonathan Pickering. 2005. Virtual camera planning: A survey. In Smart Graphics, Lecture Notes in Computer Science, vol. 3638, 40--52. DOI:http://dx.doi.org/10.1007/11536482 4 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A Dearden, Y Demiris, and O Grau. 2007. Learning models of camera control for imitation in football matches. In Proceedings of the Artificial and Ambient Intelligence Symposium. 227--231.Google ScholarGoogle Scholar
  11. Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). ACM, New York, 11--20. DOI:http://dx.doi.org/10.1145/237170.237191 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Myléne C. Q. Farias, John M. Foley, and Sanjit K. Mitra. 2007. Detectability and annoyance of synthetic blocky, blurry, noisy, and ringing artifacts. IEEE Trans. Signal Process. 55, 6, 2954--2964. DOI:http://dx.doi.org/10.1109/TSP.2007.893963 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Christoph Fehn, Christian Weissig, Ingo Feldmann, Markus Muller, Peter Eisert, Peter Kauff, and Hans Bloss. 2006. Creation of high-resolution video panoramas of sport events. In Proceedings of the IEEE International Symposium on Multimedia. 291--298. DOI:http://dx.doi.org/10.1109/ISM.2006.55 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Eric Foote, Peter Carr, Patrick Lucey, Yaser Sheikh, and Iain Matthews. 2013. One-man-band: A touch screen interface for producing live multi-camera sports broadcasts. In Proceedings of the ACM Multimedia Conference. 163--172. DOI:http://dx.doi.org/10.1145/2502081.2502092 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Vamsidhar Reddy Gaddam, Carsten Griwodz, and Påal Halvorsen. 2014a. Automatic exposure for panoramic systems in uncontrolled lighting conditions: a football stadium case study. In Proceedings of SPIE: The Engineering Reality of Virtual Reality. 90120C--90120C--9. DOI:http://dx.doi.org/10.1117/12.2040145Google ScholarGoogle Scholar
  16. Vamsidhar Reddy Gaddam, Ragnar Langseth, Sigurd Ljødal, Pierre Gurdjos, Vincent Charvillat, Carsten Griwodz, and Påal Halvorsen. 2014b. Interactive Zoom and Panning from Live Panoramic Video. In Proceedings of the ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video. Article 19. DOI:http://dx.doi.org/10.1145/2578260.2578264 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Lutz Goldmann, Francesca De Simone, Frederic Dufaux, Touradj Ebrahimi, Rudolf Tanner, and Mauro Lattuada. 2010. Impact of video transcoding artifacts on the subjective quality. In Proceedings of the International Workshop on Quality of Multimedia Experience. 52--57.Google ScholarGoogle ScholarCross RefCross Ref
  18. Patrik Goorts, Steven Maesen, Maarten Dumont, Sammy Rogmans, and Philippe Bekaert. 2014. Free viewpoint video for soccer using histogram-based validity maps in plane sweeping. In Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. 378--386.Google ScholarGoogle Scholar
  19. O. Grau, T. Pullen, and G. A. Thomas. 2004. A combined studio production system for 3-D capturing of live action and immersive actor feedback. IEEE Trans. Circuits Syst. Video Technol. 14, 3, 370--380. DOI:http://dx.doi.org/10.1109/TCSVT.2004.823397 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. O. Grau, G. A. Thomas, A. Hilton, J. Kilner, and J. Starck. 2007. A robust free-viewpoint video system for sport scenes. In Proceedings of the 3DTV Conference. 1--4. DOI:http://dx.doi.org/10.1109/3DTV.2007.4379384Google ScholarGoogle Scholar
  21. Påal Halvorsen, Simen Såegrov, Asgeir Mortensen, David K. C. Kristensen, Alexander Eichhorn, Magnus Stenhaug, Stian Dahl, Håakon Kvale Stensland, Vamsidhar Reddy Gaddam, Carsten Griwodz, and Dag Johansen. 2013. BAGADUS: An Integrated system for arena sports analytics -- A soccer case study. In Proceedings of the ACM Multimedia Conference. 48--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S. Hutchinson, G. D. Hager, and P. I. Corke. 1996. A tutorial on visual servo control. IEEE Trans. Rob. Automation 12, 5, 651--670. DOI:http://dx.doi.org/10.1109/70.538972Google ScholarGoogle ScholarCross RefCross Ref
  23. ITU-R. 2002. BT.500-11. Methodology for the subjective assessment of the quality of television pictures. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.500-11-200206-SIIPDF-E.pdf.Google ScholarGoogle Scholar
  24. ITU-T. 1998. P.911. Subjective audiovisual quality assessment methods for multimedia applications. https://www.itu.int/rec/T-REC-P.911-199812-1/en.Google ScholarGoogle Scholar
  25. Michael Jenkin, James Elder, and Greg Pintilie. 1998. Loosely-coupled telepresence through the panoramic image server. In Vision Interface: Real World Applications of Computer Vision.Google ScholarGoogle Scholar
  26. R. Kaiser, M. Thaler, A. Kriechbaum, H. Fassold, W. Bailer, and J. Rosner. 2011. Real-time person tracking in high-resolution panoramic video for automated broadcast production. In Proceedings of the European Conference on Visual Media Production. 21--29. DOI:http://dx.doi.org/10.1109/CVMP.2011.9 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Takeo Kanade, Peter Rander, and P. J. Narayanan. 1997. Virtualized reality: Constructing virtual worlds from real scenes. IEEE MultiMedia 4, 1, 34--47. DOI:http://dx.doi.org/10.1109/93.580394 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jong-Seok Lee, Lutz Goldmann, and Touradj Ebrahimi. 2012. Paired comparison-based subjective quality assessment of stereoscopic images. Multimedia Tools Appl. 67, 1, 31--48. DOI:http://dx.doi.org/10.1007/s11042-012-1011-6Google ScholarGoogle ScholarCross RefCross Ref
  29. Christian Lipski, Christian Linz, Kai Berger, and Marcus Magnor. 2009. Virtual video camera: Image-based viewpoint navigation through space and time. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. Article 93. DOI:http://dx.doi.org/10.1145/1599301.1599394 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Aditya Mavlankar and Bernd Girod. 2010. Video streaming with interactive pan/tilt/zoom. In High-Quality Visual Experience, Marta Mrak, Mislav Grgic, and Murat Kunt (Eds.), 431--455. DOI:http://dx.doi.org/10.1007/978-3-642-12802-8 19Google ScholarGoogle Scholar
  31. Pengpeng Ni, Ragnhild Eg, Alexander Eichhorn, Carsten Griwodz, and Påal Halvorsen. 2011. Flicker effects in adaptive video streaming to handheld devices. In Proceedings of the ACM Multimedia Conference. 463--472. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. N. Papadakis, A. Baeza, I. Rius, X. Armangue, A. Bugeau, O. D'Hondt, P. Gargallo, V. Caselles, and S. Sagas. 2010. Virtual camera synthesis for soccer game replays. In Proceedings of the Conference on Visual Media Production. 97--106. DOI:http://dx.doi.org/10.1109/CVMP.2010.20 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Jinchang Ren, Ming Xu, James Orwell, and GraemeA. Jones. 2010. Multi-camera video surveillance for real-time analysis and reconstruction of soccer games. Machine Vision Appl. 21, 6, 855--863. DOI:http://dx.doi.org/10.1007/s00138-009-0212-0 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Xinding Sun, J. Foote, D. Kimber, and B. S. Manjunath. 2005. Region of interest extraction and virtual camera control based on panoramic video capturing. IEEE Trans. Multimedia 7, 5, 981--990. DOI:http://dx.doi.org/10.1109/TMM.2005.854388 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Marius Tennøe, Espen Helgedagsrud, Mikkel Nåess, Henrik Kjus Alstad, Håakon Kvale Stensland, Vamsidhar Reddy Gaddam, Dag Johansen, Carsten Griwodz, and Påal Halvorsen. 2013. Efficient implementation and processing of a real-time panorama video pipeline. In Proceedings of the IEEE International Symposium on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Jinjun Wang, Changsheng Xu, Engsiong Chng, Kongwah Wah, and Qi Tian. 2004. Automatic replay generation for soccer video broadcasting. In Proceedings of the ACM Multimedia Conference. 32--39. DOI:http://dx.doi.org/10.1145/1027527.1027535 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Wanmin Wu, Ahsan Arefin, Raoul Rivas, Klara Nahrstedt, Renata M. Sheppard, and Zhenyu Yang. 2009. Quality of experience in distributed interactive multimedia environments: Toward a theoretical framework. In Proceedings of the ACM Multimedia Conference. 481--490. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Xu, J. Orwell, L. Lowey, and D. Thirde. 2005. Architecture and algorithms for tracking football players with multiple cameras. In IEE Proc. Vision Image Signal Process. 152, 2, 232--241. DOI:http://dx.doi.org/10.1049/ip-vis:20041257Google ScholarGoogle ScholarCross RefCross Ref
  39. Wei Xu and Jane Mulligan. 2013. Panoramic video stitching from commodity HDTV cameras. Multimedia Systems 19, 5, 407--426. DOI:http://dx.doi.org/10.1007/s00530-013-0316-2 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. T. Yokoi and H. Fujiyoshi. 2005. Virtual camerawork for generating lecture video from high resolution images. In Proceedings of the IEEE International Conference on Multimedia and Expo. DOI:http://dx.doi.org/10.1109/ICME.2005.1521532Google ScholarGoogle Scholar
  41. Xinguo Yu, Changsheng Xu, Hon Wai Leong, Qi Tian, Qing Tang, and Kong Wah Wan. 2003. Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In Proceedings of the ACM Multimedia Conference. 11--20. DOI:http://dx.doi.org/10.1145/957013.957018 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. The Cameraman Operating My Virtual Camera is Artificial: Can the Machine Be as Good as a Human?

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!