skip to main content
research-article

Sprite generation using sprite fusion

Published:22 May 2012Publication History
Skip Abstract Section

Abstract

There has been related research for sprite or mosaic generation for over 15 years. In this article, we try to understand the methodologies for sprite generation and identify what has not actually been covered for sprite generation. We first identify issues and focus on the domain of videos for sprite generation. We introduce a novel sprite fusion method that blends two sprites. Sprite fusion method produces good results for tracking videos and does not require object segmentation. We present sample results of our experiments.

References

  1. Alzoubi, H. and Pan, W. D. 2008. Fast and accurate global motion estimation algorithm using pixel subsampling. Inf. Sci. 178, 17, 3415--3425. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Asif, M. and Soraghan, J. J. 2008. MPEG-7 Motion Descriptor Extraction for Panning Camera Using Sprite Generated. In Proceedings of the 5th International Conference on Advanced Video and Signal Based Surveillance. IEEE Computer Society, Los Alamitos, CA, 60--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Atrey, P. K., Hossain, M. A., Saddik A. E., and Kankahalli, M. S. 2010. Multimodal fusion for multimedia analysis: Survey. Multimedia Syst. 16, 345--379.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Aygün, R. S. and Zhang, A. 2002. Reducing blurring-effect in high resolution mosaic generation. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '02). IEEE Computer Society, Los Alamitos, CA, 537--540.Google ScholarGoogle Scholar
  5. Aygün, R. S. and Zhang, A. 2004. Integrating virtual camera controls into digital video. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '04). IEEE Computer Society, Los Alamitos, CA,1503--1506.Google ScholarGoogle Scholar
  6. Azzari, P., Di Stefano, L., and Bevilacqua, A. 2005. An effective real-time mosaicing algorithm apt to detect motion through background subtraction using a PTZ camera. In Proceedings of the IEEE Conference on Advanced Video and Signal-Based Surveillance. 511--516.Google ScholarGoogle Scholar
  7. Chen, L., Lai, Y., and Liao, H. 2006. Video Scene Extraction Using Mosaic Technique. In Proceedings of the 18th International Conference on Pattern Recognition. IEEE Computer Society, Los Alamitos, CA, 723--726. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Chen, S. Y., Chen, C. Y., Huang, Y. W., and Chen, L. G. 2002. Multiple sprites and frame skipping techniques for sprite generation with high subjective quality and fast speed. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '02). IEEE Computer Society, Los Alamitos, CA, 785--788.Google ScholarGoogle Scholar
  9. Chen, Y. and Aygün, R. S. 2010. Synthetic video generation for evaluation of the sprite generation. Int. J. Multimedia Data Engin. Manage. 2, 34--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Cherng, D.-C. and Chien S.-Y. 2007. Video Segmentation with model-based sprite generation for panning surveillance cameras. In Proceedings of the IEEE International Symposium on Circuits and Systems. IEEE Computer Society, Los Alamitos, CA, 27--30.Google ScholarGoogle Scholar
  11. Cheung, H.-K. and Siu, W.-C. 2002. Fast global motion estimation for sprite generation. In Proceedings of the IEEE International Symposium on Circuits and Systems. IEEE Computer Society, Los Alamitos, CA.Google ScholarGoogle Scholar
  12. Cheung, H.-K. and Siu, W.-C. 2007. Robust global motion estimation and novel updating strategy for sprite generation. IET Image Process. 1, 1, 13--20.Google ScholarGoogle ScholarCross RefCross Ref
  13. Cheung, H.-K., Siu, W.-C., and Feng, D. 2008. New block-based motion estimation for sequences with brightness variation and its application to static sprite generation for video compression. IEEE Trans Circ. Syst. Video Technol. 18, 522--527. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Coorg, S. and Teller, S. 2000. Spherical mosaics with quaternions and dense correlation. Int. J. Comput. Vision 37, 3, 259--273. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Dasu, A. R. and Panchanathan, S. 2004. A wavelet-based sprite codec. IEEE Trans Circ. Syst. Video Technol. 14, 2, 244--255. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Deshpande, A. and Aygün, R. S. 2009. Motion-based video classification for sprite generation. In Proceedings of the International Workshop on Database and Expert Systems Applications. 231--235. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Dufaux, F. and Konrad, J. 2000. Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9, 3, 497--501. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Farin, D. and de With, P. H. N. 2006. Enabling arbitrary rotational camera motion using multisprites with minimum coding cost. IEEE Trans. Circ. Syst. Video Technol. 16, 4, 492--506. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Fraunhaufer. 2009. http://www.iis.fraunhofer.de/amm/download/mpeg4.Google ScholarGoogle Scholar
  20. Geys, H. and Van Gool, L. 2006. On-line, interactive view synthesis and augmentation. Signal Process.Image Comm. 21, 9, 709--723.Google ScholarGoogle ScholarCross RefCross Ref
  21. Grammalidis, N., Beletsiotis, D., and Strintzis, M. G. 1999. Multi View sprite generation and coding. InProceedings of the International Conference on Image Processing (ICIP '99). IEEE Computer Society, Los Alamitos, CA, 477--481. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. H264. Iso/iec 14496-10:2003. information technology: Coding of audio-visual objects - part 2, also itu-t recommendation h.264 advanced video coding for generic audiovisual services.Google ScholarGoogle Scholar
  23. H265. 2009. http://www.h265.net.Google ScholarGoogle Scholar
  24. Hsu, C.-T. and Tsan Y.-C. 2004. Mosaics of video sequences with moving objects. Signal Process.Image Comm. 19, 1, 81--98.Google ScholarGoogle ScholarCross RefCross Ref
  25. Irani, M. and Anandan, P. 1998. Video indexing based on mosaic representations. Proc. IEEE, 905--921.Google ScholarGoogle Scholar
  26. Krutz, A., Frater, M., Kunter, M., and Sikora, T. 2006. Windowed image registration for robust mosaicing of scenes with large background occlusions. In Proceedings of the International Conference on Image Processing (ICIP '06). 353--356.Google ScholarGoogle Scholar
  27. Krutz, A., Glantz, A., Sikora, T., Nunes, P., and Pereira, F. 2008. Automatic object segmentation algorithms for sprite coding using MPEG-4. In Proceedings of the 50th International ELMAR Symposium. 459--462.Google ScholarGoogle Scholar
  28. Krutz, A., Glantz, A., Haller, M., Droese, M., and Sikora, T. 2008. Multiple background sprite generation using camera motion characterization for object-based video coding. In Proceedings of the 5th International Conference on Advanced Video and Signal Based Surveillance. IEEE Computer Society, Los Alamitos, CA, 313--316.Google ScholarGoogle Scholar
  29. Kunter, M., Krey, P., Krutz, A., and Sikora, T. 2008. Extending H.264/AVC with a background sprite prediction mode. In Proceedings of the International Conference on Image Processing (ICIP'08).Google ScholarGoogle Scholar
  30. Lai, J., Kao, C., and Chien, S. 2009. Super-resolution sprite with foreground removal. In Proceedings of the IEEE International Conference on Multimedia and Expo. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Lee, M.-C., Chen, W.-G., Lin, C.-B., Chuang, G., Markoc, T., Zabinsky, S. I., and Szeliski, R. 1997. A layered video object coding system using sprite and affine motion model. IEEE Trans. Circ. Syst. Video Technol. 7, 1, 130--145. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Lu, Y., Gao, W., and Wu, F. 2001a. Fast and robust sprite generation for MPEG-4 video coding. In Proceedings of the 2nd IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia information Processing. H. Shum, M. Liao, and S. Chang, Eds., Lecture Notes In Computer Science, vol. 2195, Springer, 118--125. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Lu, Y., Gao, W., and Wu, F. 2001b. Sprite generation for frame-based video coding. In Proceedings of the International Conference on Image Processing. 473--476.Google ScholarGoogle Scholar
  34. Lu, Y., Gao, W., and Wu, F. 2003. Efficient background video coding with static sprite generation and arbitrary-shape spatial prediction techniques. IEEE Trans. Circ. Syst. Video Technol. 13, 5, 394--405. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Marzotto, R., Fusiello, A., and Murino, V. 2004. High resolution video mosaicing with global alignment. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 692--698.Google ScholarGoogle Scholar
  36. MPEG4 Software. Iso/iec 14496-7:2001. Information technology: Coding of audio-visual objects - part 7: Optimized software for mpeg-4 visual tools.Google ScholarGoogle Scholar
  37. MPEG4-2, Iso/iec 14496-2:2004. Information technology: Coding of audio-visual objects: part 2.Google ScholarGoogle Scholar
  38. Nagaraj, R. C., Dasu, A. R., and Panchanathan, S. 2001. Complexity analysis of sprites in MPEG. In Proceedings of SPIE, vol. 4313, 69--73.Google ScholarGoogle Scholar
  39. Ostermann, J., Bormans, J., List, P., Marpe, D., Narroschke, M., Perreira, F., Stockhammer, T., and Wedi, T. 2004. Video coding with h.264/avc: tools, performance, and complexity. IEEE Circ. Syst. Mag. 4, 1, 7--28.Google ScholarGoogle ScholarCross RefCross Ref
  40. Parikh, P. and Jawahar, C. V. 2007. Enhanced video mosaicing using camera motion properties. In Proceedings of the IEEE Workshop on Motion and Video Computing. IEEE Computer Society, Los Alamitos, CA, 26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Peleg, S., Rousso, B., Ravacha, A., and Zomet, A. 2000. Mosaicing on adaptive methods. IEEE Trans. Pattern Anal. Mach. Intell. 22, 10, 1144--1154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Prodys. 2009. http://www.prodys.com/.Google ScholarGoogle Scholar
  43. Richter, H., Smolic, A., Stabernack, B., and Müller, E., Real time global motion estimation for an MPEG-4 video encoder. In Proceedings of the Picture Coding Symposium.Google ScholarGoogle Scholar
  44. Salembier, P., Pujol, O., and Garrido, L. 1998. Connected operators for sprite creation and layered representation of image sequences. In Proceedings of the European Signal Processing Conference. 2105--2108.Google ScholarGoogle Scholar
  45. Shen, Y. and Zhang, L. 2004. A Novel Method of Sprite Generation Based on Pixel Similarity. In Proceedings of the 3rd International Conference on Image and Graphics. IEEE Computer Society, Los Alamitos, CA, 560-- 563. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Sikora, T. 1997. The MPEG-4 video standard verification model. IEEE Trans. Circ. Syst. Video Technol. 7, 19--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Smolic, A. and Ohm, J.-R. 2000. Robust global motion estimation using a simplified m-estimator approach. In Proceedings of the IEEE International Conference on Image Processing.Google ScholarGoogle Scholar
  48. Smolic, A., Sikora, T., and Ohm, J.-R. 1999. Long-term global motion estimation and its application for sprite coding, content description and segmentation. IEEE Trans. Circ. Syst. Video Technol. 9, 8, 1227--1242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Snoek, C. G., Worring, M., and Smeulders, A. W. 2005. Early versus late fusion in semantic video analysis. In Proceedings of the 13th Annual ACM International Conference on Multimedia (MULTIMEDIA '05). ACM, New York, NY, 399--402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Steedly, D., Pal, C., and Szeliski, R. 2005. Efficiently registering video into panoramic mosaics. In Proceedings of the 10th International Conference on Computer Vision. 1300--1307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Szeliski, R. 2006. Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vision 2, 1, 1--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Szeliski, R. and Shum, H.-Y. 1997. Creating full view panoramic image mosaics and environment maps. In Proceedings of ACM SIGGRAPH. 251--258. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Taubman, D. and Marcellin, M. 2002. JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer Academic Publishers, chapter 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Teodosio, L. and Bender, W. 1993. Salient video stills: content and context preserved. In Proceedings of the 1st ACM International Conference on Multimedia (MULTIMEDIA '93). ACM, New York, NY, 39--46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Teodosio, L. and Bender, W. 2005. Salient stills. ACM Trans. Multimedia Comput. Comm. Appl. 1, 1, 16--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. To, L. T. 2005. Video object segmentation using phase-based detection of moving object buondaries, Ph.D. thesis, University of New South Wales (2005).Google ScholarGoogle Scholar
  57. Ye, G., Pickering, M., Frater, M., and Arnold, J. 2005. A robust approach to super-resolution sprite generation. In Proceedings of the IEEE International Conference on Image Processing. IEEE Computer Society, Los Alamitos, CA, 11--14.Google ScholarGoogle Scholar
  58. Ye, G., Wang, Y., Xu, J., Herman, G., and Zhang, B. 2008. A practical approach to multiple super-resolution sprite generation. In Proceedings of the IEEE 10th Workshop on Multimedia Signal Processing. IEEE Computer Society, Los Alamitos, CA, 70--75, 8-10.Google ScholarGoogle Scholar
  59. Zhu, Z., Xu, G., Riseman, E. M., and Hanson, A. R. 1999. Fast generation of dynamic and multi-resolution 360-degree panorama from video sequences. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems. IEEE Computer Society, Los Alamitos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Zoghlami, I., Faugeras, O., and Deriche, R. 1997. Using geometric corners to build a 2d mosaic from a set of images. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 420--425. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Sprite generation using sprite fusion

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 8, Issue 2
      May 2012
      144 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/2168996
      Issue’s Table of Contents

      Copyright © 2012 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 22 May 2012
      • Accepted: 1 February 2011
      • Revised: 1 September 2010
      • Received: 1 March 2010
      Published in tomm Volume 8, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!