Abstract
In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.
- Adel Ahmed and Peter Eades. 2005. Automatic camera path generation for graph navigation in 3D. In Proceedings of the Asia-Pacific Symposium on Information Visualisation. 27--32. http://dl.acm.org/citation.cfm?id&equal;1082315.1082320 Google Scholar
Digital Library
- Y. Ariki, S. Kubota, and M. Kumano. 2006. Automatic production system of soccer sports video by digital camera work based on situation recognition. In Proceedings of the IEEE International Symposium on Multimedia. 851--860. DOI:http://dx.doi.org/10.1109/ISM.2006.37 Google Scholar
Digital Library
- Peter Carr and Richard Hartley. 2009. Portable multi-megapixel camera with real-time recording and playback. In Proceedings of the Conference on Digital Image Computing: Techniques and Applications. 74--80. DOI:http://dx.doi.org/10.1109/DICTA.2009.62 Google Scholar
Digital Library
- Peter Carr, Michael Mistry, and Iain Matthews. 2013. Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording. In Proceedings of the ACM Multimedia Conference. 193--202. Google Scholar
Digital Library
- Joel Carranza, Christian Theobalt, Marcus A. Magnor, and Hans-Peter Seidel. 2003. Free viewpoint video of human actors. ACM Trans. Graph. 22, 3, 569--577. DOI:http://dx.doi.org/10.1145/882262.882309 Google Scholar
Digital Library
- Fan Chen and Christophe De Vleeschouwer. 2010. Personalized production of basketball videos from multisensored data under limited display resolution. Computer Vision Image Understanding 114, 6, 667--680. DOI:http://dx.doi.org/10.1016/j.cviu.2010.01.005 Google Scholar
Digital Library
- Kuan-Ta Chen, Chen-Chi Wu, Yu-Chun Chang, and Chin-Laung Lei. 2009. A crowd-sourceable QoE evaluation framework for multimedia content. In Proceedings of the ACM Multimedia Conference. 491--500. DOI:http://dx.doi.org/10.1145/1631272.1631339 Google Scholar
Digital Library
- Shenchang Eric Chen. 1995. QuickTime VR: An image-based approach to virtual environment navigation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 29--38. DOI:http://dx.doi.org/10.1145/218380.218395 Google Scholar
Digital Library
- Marc Christie, Rumesh Machap, Jean-Marie Normand, Patrick Olivier, and Jonathan Pickering. 2005. Virtual camera planning: A survey. In Smart Graphics, Lecture Notes in Computer Science, vol. 3638, 40--52. DOI:http://dx.doi.org/10.1007/11536482 4 Google Scholar
Digital Library
- A Dearden, Y Demiris, and O Grau. 2007. Learning models of camera control for imitation in football matches. In Proceedings of the Artificial and Ambient Intelligence Symposium. 227--231.Google Scholar
- Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'96). ACM, New York, 11--20. DOI:http://dx.doi.org/10.1145/237170.237191 Google Scholar
Digital Library
- Myléne C. Q. Farias, John M. Foley, and Sanjit K. Mitra. 2007. Detectability and annoyance of synthetic blocky, blurry, noisy, and ringing artifacts. IEEE Trans. Signal Process. 55, 6, 2954--2964. DOI:http://dx.doi.org/10.1109/TSP.2007.893963 Google Scholar
Digital Library
- Christoph Fehn, Christian Weissig, Ingo Feldmann, Markus Muller, Peter Eisert, Peter Kauff, and Hans Bloss. 2006. Creation of high-resolution video panoramas of sport events. In Proceedings of the IEEE International Symposium on Multimedia. 291--298. DOI:http://dx.doi.org/10.1109/ISM.2006.55 Google Scholar
Digital Library
- Eric Foote, Peter Carr, Patrick Lucey, Yaser Sheikh, and Iain Matthews. 2013. One-man-band: A touch screen interface for producing live multi-camera sports broadcasts. In Proceedings of the ACM Multimedia Conference. 163--172. DOI:http://dx.doi.org/10.1145/2502081.2502092 Google Scholar
Digital Library
- Vamsidhar Reddy Gaddam, Carsten Griwodz, and Påal Halvorsen. 2014a. Automatic exposure for panoramic systems in uncontrolled lighting conditions: a football stadium case study. In Proceedings of SPIE: The Engineering Reality of Virtual Reality. 90120C--90120C--9. DOI:http://dx.doi.org/10.1117/12.2040145Google Scholar
- Vamsidhar Reddy Gaddam, Ragnar Langseth, Sigurd Ljødal, Pierre Gurdjos, Vincent Charvillat, Carsten Griwodz, and Påal Halvorsen. 2014b. Interactive Zoom and Panning from Live Panoramic Video. In Proceedings of the ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video. Article 19. DOI:http://dx.doi.org/10.1145/2578260.2578264 Google Scholar
Digital Library
- Lutz Goldmann, Francesca De Simone, Frederic Dufaux, Touradj Ebrahimi, Rudolf Tanner, and Mauro Lattuada. 2010. Impact of video transcoding artifacts on the subjective quality. In Proceedings of the International Workshop on Quality of Multimedia Experience. 52--57.Google Scholar
Cross Ref
- Patrik Goorts, Steven Maesen, Maarten Dumont, Sammy Rogmans, and Philippe Bekaert. 2014. Free viewpoint video for soccer using histogram-based validity maps in plane sweeping. In Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. 378--386.Google Scholar
- O. Grau, T. Pullen, and G. A. Thomas. 2004. A combined studio production system for 3-D capturing of live action and immersive actor feedback. IEEE Trans. Circuits Syst. Video Technol. 14, 3, 370--380. DOI:http://dx.doi.org/10.1109/TCSVT.2004.823397 Google Scholar
Digital Library
- O. Grau, G. A. Thomas, A. Hilton, J. Kilner, and J. Starck. 2007. A robust free-viewpoint video system for sport scenes. In Proceedings of the 3DTV Conference. 1--4. DOI:http://dx.doi.org/10.1109/3DTV.2007.4379384Google Scholar
- Påal Halvorsen, Simen Såegrov, Asgeir Mortensen, David K. C. Kristensen, Alexander Eichhorn, Magnus Stenhaug, Stian Dahl, Håakon Kvale Stensland, Vamsidhar Reddy Gaddam, Carsten Griwodz, and Dag Johansen. 2013. BAGADUS: An Integrated system for arena sports analytics -- A soccer case study. In Proceedings of the ACM Multimedia Conference. 48--59. Google Scholar
Digital Library
- S. Hutchinson, G. D. Hager, and P. I. Corke. 1996. A tutorial on visual servo control. IEEE Trans. Rob. Automation 12, 5, 651--670. DOI:http://dx.doi.org/10.1109/70.538972Google Scholar
Cross Ref
- ITU-R. 2002. BT.500-11. Methodology for the subjective assessment of the quality of television pictures. https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.500-11-200206-SIIPDF-E.pdf.Google Scholar
- ITU-T. 1998. P.911. Subjective audiovisual quality assessment methods for multimedia applications. https://www.itu.int/rec/T-REC-P.911-199812-1/en.Google Scholar
- Michael Jenkin, James Elder, and Greg Pintilie. 1998. Loosely-coupled telepresence through the panoramic image server. In Vision Interface: Real World Applications of Computer Vision.Google Scholar
- R. Kaiser, M. Thaler, A. Kriechbaum, H. Fassold, W. Bailer, and J. Rosner. 2011. Real-time person tracking in high-resolution panoramic video for automated broadcast production. In Proceedings of the European Conference on Visual Media Production. 21--29. DOI:http://dx.doi.org/10.1109/CVMP.2011.9 Google Scholar
Digital Library
- Takeo Kanade, Peter Rander, and P. J. Narayanan. 1997. Virtualized reality: Constructing virtual worlds from real scenes. IEEE MultiMedia 4, 1, 34--47. DOI:http://dx.doi.org/10.1109/93.580394 Google Scholar
Digital Library
- Jong-Seok Lee, Lutz Goldmann, and Touradj Ebrahimi. 2012. Paired comparison-based subjective quality assessment of stereoscopic images. Multimedia Tools Appl. 67, 1, 31--48. DOI:http://dx.doi.org/10.1007/s11042-012-1011-6Google Scholar
Cross Ref
- Christian Lipski, Christian Linz, Kai Berger, and Marcus Magnor. 2009. Virtual video camera: Image-based viewpoint navigation through space and time. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. Article 93. DOI:http://dx.doi.org/10.1145/1599301.1599394 Google Scholar
Digital Library
- Aditya Mavlankar and Bernd Girod. 2010. Video streaming with interactive pan/tilt/zoom. In High-Quality Visual Experience, Marta Mrak, Mislav Grgic, and Murat Kunt (Eds.), 431--455. DOI:http://dx.doi.org/10.1007/978-3-642-12802-8 19Google Scholar
- Pengpeng Ni, Ragnhild Eg, Alexander Eichhorn, Carsten Griwodz, and Påal Halvorsen. 2011. Flicker effects in adaptive video streaming to handheld devices. In Proceedings of the ACM Multimedia Conference. 463--472. Google Scholar
Digital Library
- N. Papadakis, A. Baeza, I. Rius, X. Armangue, A. Bugeau, O. D'Hondt, P. Gargallo, V. Caselles, and S. Sagas. 2010. Virtual camera synthesis for soccer game replays. In Proceedings of the Conference on Visual Media Production. 97--106. DOI:http://dx.doi.org/10.1109/CVMP.2010.20 Google Scholar
Digital Library
- Jinchang Ren, Ming Xu, James Orwell, and GraemeA. Jones. 2010. Multi-camera video surveillance for real-time analysis and reconstruction of soccer games. Machine Vision Appl. 21, 6, 855--863. DOI:http://dx.doi.org/10.1007/s00138-009-0212-0 Google Scholar
Digital Library
- Xinding Sun, J. Foote, D. Kimber, and B. S. Manjunath. 2005. Region of interest extraction and virtual camera control based on panoramic video capturing. IEEE Trans. Multimedia 7, 5, 981--990. DOI:http://dx.doi.org/10.1109/TMM.2005.854388 Google Scholar
Digital Library
- Marius Tennøe, Espen Helgedagsrud, Mikkel Nåess, Henrik Kjus Alstad, Håakon Kvale Stensland, Vamsidhar Reddy Gaddam, Dag Johansen, Carsten Griwodz, and Påal Halvorsen. 2013. Efficient implementation and processing of a real-time panorama video pipeline. In Proceedings of the IEEE International Symposium on Multimedia. Google Scholar
Digital Library
- Jinjun Wang, Changsheng Xu, Engsiong Chng, Kongwah Wah, and Qi Tian. 2004. Automatic replay generation for soccer video broadcasting. In Proceedings of the ACM Multimedia Conference. 32--39. DOI:http://dx.doi.org/10.1145/1027527.1027535 Google Scholar
Digital Library
- Wanmin Wu, Ahsan Arefin, Raoul Rivas, Klara Nahrstedt, Renata M. Sheppard, and Zhenyu Yang. 2009. Quality of experience in distributed interactive multimedia environments: Toward a theoretical framework. In Proceedings of the ACM Multimedia Conference. 481--490. Google Scholar
Digital Library
- M. Xu, J. Orwell, L. Lowey, and D. Thirde. 2005. Architecture and algorithms for tracking football players with multiple cameras. In IEE Proc. Vision Image Signal Process. 152, 2, 232--241. DOI:http://dx.doi.org/10.1049/ip-vis:20041257Google Scholar
Cross Ref
- Wei Xu and Jane Mulligan. 2013. Panoramic video stitching from commodity HDTV cameras. Multimedia Systems 19, 5, 407--426. DOI:http://dx.doi.org/10.1007/s00530-013-0316-2 Google Scholar
Digital Library
- T. Yokoi and H. Fujiyoshi. 2005. Virtual camerawork for generating lecture video from high resolution images. In Proceedings of the IEEE International Conference on Multimedia and Expo. DOI:http://dx.doi.org/10.1109/ICME.2005.1521532Google Scholar
- Xinguo Yu, Changsheng Xu, Hon Wai Leong, Qi Tian, Qing Tang, and Kong Wah Wan. 2003. Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In Proceedings of the ACM Multimedia Conference. 11--20. DOI:http://dx.doi.org/10.1145/957013.957018 Google Scholar
Digital Library
Index Terms
The Cameraman Operating My Virtual Camera is Artificial: Can the Machine Be as Good as a Human?
Recommendations
Scaling virtual camera services to a large number of users
MMSys '15: Proceedings of the 6th ACM Multimedia Systems ConferenceBy processing video footage from a camera array, one can easily make wide-field-of-view panorama videos. From the single panorama video, one can further generate multiple virtual cameras supporting personalized views to a large number of users based on ...
Be your own cameraman: real-time support for zooming and panning into stored and live panoramic video
MMSys '14: Proceedings of the 5th ACM Multimedia Systems ConferenceHigh-resolution panoramic video with a wide field-of-view is popular in many contexts. However, in many examples, like surveillance and sports, it is often desirable to zoom and pan into the generated video. A challenge in this respect is real-time ...
Automatic Real-Time Zooming and Panning on Salient Objects from a Panoramic Video
MM '14: Proceedings of the 22nd ACM international conference on MultimediaThe proposed demo shows how our system automatically zooms and pans into tracked objects in panorama videos. At the conference site, we will set up a two-camera version of the system, generating live panorama videos, where the system zooms and pans ...






Comments