article

Animating Chinese paintings through stroke-based decomposition

Abstract

This article proposes a technique to animate a Chinese style painting given its image. We first extract descriptions of the brush strokes that hypothetically produced it. The key to the extraction process is the use of a brush stroke library, which is obtained by digitizing single brush strokes drawn by an experienced artist. The steps in our extraction technique are first to segment the input image, then to find the best set of brush strokes that fit the regions, and, finally, to refine these strokes to account for local appearance. We model a single brush stroke using its skeleton and contour, and we characterize texture variation within each stroke by sampling perpendicularly along its skeleton. Once these brush descriptions have been obtained, the painting can be animated at the brush stroke level. In this article, we focus on Chinese paintings with relatively sparse strokes. The animation is produced using a graphical application we developed. We present several animations of real paintings using our technique.

References

  1. Alexa, M., Cohen-Or, D., and Levin, D. 2000. As-rigid-as-possible shape interpolation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'00). ACM Press/Addison-Wesley Publishing Co., New York, NY, 157--164. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Barrett, W. A. and Cheney, A. S. 2002. Object-based image editing. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '02). ACM Press, New York, NY, 777--784. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Baxter, B., Scheib, V., Lin, M. C., and Manocha, D. 2001. DAB: Interactive haptic painting with 3D virtual brushes. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'01). ACM Press, New York, NY, 461--468. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Blostein, D. and Haken, L. 1999. Using diagram generation software to improve diagram recognition: A case study of music notation. IEEE Trans. Pattern Anal. Mach. Intell. 21, 11, 1121--1136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Chen, H. Ed. 1994. Encyclopaedia of China, Film Volume (in Chinese). Encyclopaedia of China Press.Google ScholarGoogle Scholar
  6. Chen, J. and Zhang, J. Eds. 1995. Dictionary of Chinese Films (in Chinese). Shanghai Dictionary Press.Google ScholarGoogle Scholar
  7. Chuang, Y.-Y. et al. 2001. A Bayesian approach to digital matting. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'01). Kauai, Hawaii, vol. II, 264--271.Google ScholarGoogle Scholar
  8. Comaniciu, D. and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603--619. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Daniels, E. 1999. Deep canvas in Disney's Tarzan. In ACM SIGGRAPH 99 Conference Abstracts and Applications. ACM Press, New York, NY, 200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. DeCarlo, D. and Santella, A. 2002. Stylization and abstraction of photographs. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'02). ACM Press, New York, NY, 769--776. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Farid, H. and Adelson, E. 1999. Separating reflections from images by use of independent components analysis. J. Optical Society America 16, 9, 2136--2145.Google ScholarGoogle ScholarCross RefCross Ref
  12. Feldman, J. A. and Yakimovsky, Y. 1974. Decision theory and artificial intelligence: I. a semantics-based region analyzer. Artific. Intell. 5, 4, 349--371.Google ScholarGoogle ScholarCross RefCross Ref
  13. Forsyth, D. A. and Ponce, J. 2002. Computer Vision: A Modern Approach. Prentice Hall. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Gooch, B., Coombe, G., and Shirley, P. 2002. Artistic vision: Painterly rendering using computer vision techniques. In Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering (NPAR'02). ACM Press, New York, NY, 83--90. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Hertzmann, A. 1999. Introduction to 3D non-photorealistic rendering: Silhouettes and outlines. S. Green, Ed. Non-Photorealistic Rendering. SIGGRAPH 99 Course Notes.Google ScholarGoogle Scholar
  16. Horry, Y., Anjyo, K.-I., and Arai, K. 1997. Tour into the picture: Using a spidery mesh interface to make animation from a single image. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'97). ACM Press/Addison-Wesley Publishing Co., New York, NY, 225--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Hsu, S., Lee, I., Lou, C., and Siu, S. 1999. Software. Creature House (www.creaturehouse.com).Google ScholarGoogle Scholar
  18. Hsu, S. C. and Lee, I. H. H. 1994. Drawing and animation using skeletal strokes. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'94). ACM Press, New York, NY, 109--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jain, A. 1989. Fundamentals of Digital Image Processing. Prentice Hall. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Joseph, S. H. and Pridmore, T. P. 1992. Knowledge-directed interpretation of mechanical engineering drawings. IEEE Trans. Pattern Anal. Mach. Intell. 14, 9, 928--940. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Judd, D. B. and Wyszecki, G. 1975. Color in Business, Science, and Industry. John Wiley and Sons, New York, NY.Google ScholarGoogle Scholar
  22. Kalnins, R. D., Markosian, L., Meier, B. J., Kowalski, M. A., Lee, J. C., Davidson, P. L., Webb, M., Hughes, J. F., and Finkelstein, A. 2002. WYSIWYG NPR: Drawing strokes directly on 3D models. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'02). ACM Press, New York, NY, 755--762. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Kumar, K. and Desai, U. 1999. Joint segmentation and image interpretation. Patt. Recog. 32, 4, 577--589.Google ScholarGoogle ScholarCross RefCross Ref
  24. Levin, A. and Weiss, Y. 2004. User assisted separation of reflections from a single image using a sparsity prior. In European Conference on Computer Vision (ECCV'04). 602--613.Google ScholarGoogle Scholar
  25. Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'04). 306--313.Google ScholarGoogle Scholar
  26. Litwinowicz, P. and Williams, L. 1994. Animating images with drawings. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'94). ACM Press, New York, NY, 409--412. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Liu, L. and Sclaroff, S. 2001. Region segmentation via deformable model-guided split and merge. In Proceedings of the International Conference on Computer Vision (ICCV'01), vol. 1. 98--104.Google ScholarGoogle Scholar
  28. Marroquin, J. L., Santana, E. A., and Botello, S. 2003. Hidden Markov measure field models for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 25, 11, 1380--1387. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Mulder, J. A., Mackworth, A. K., and Havens, W. S. 1988. Knowledge structuring and constraint satisfaction: The mapsee approach. IEEE Trans. Pattern Anal. Mach. Intell. 10, 6, 866--879. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Neumann, A. 2003. Graphical Gaussian shape models and their application to image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 25, 3, 316--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Porter, T. and Duff, T. 1984. Compositing digital images. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'84). ACM Press, New York, NY, 253--259. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Salisbury, M. P., Anderson, S. E., Barzel, R., and Salesin, D. H. 1994. Interactive pen-and-ink illustration. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'94). ACM Press, New York, NY, 101--108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Schneider, P. J. 1990. An algorithm for automatically fitting digitized curves. Graphics Gems A. S. Glassner, Ed. Academic Press, 612 --626. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Sclaroff, S. and Liu, L. 2001. Deformable shape detection and description via model-based region grouping. IEEE Trans. Pattern Anal. Mach. Intell. 23, 5, 475--489. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sederberg, T. W. and Farouki, R. T. 1992. Approximation by interval Bezier curves. IEEE Comput. Graph. Appl. 12, 5, 87--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Smith, R. and Lloyd, E. J. 1997. Art School. Dorling Kindersley Ltd, Inc.Google ScholarGoogle Scholar
  37. Strassmann, S. 1986. Hairy brushes. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'86). ACM Press, New York, NY, 225--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Su, S., Xu, Y., Shum, H., and Chen, F. 2002. Simulating artistic brushstrokes using interval splines. In Proceedings of the 5th IASTED International Conference on Computer Graphics and Imaging. Kauai, Hawaii, 85--90.Google ScholarGoogle Scholar
  39. Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'00). Hilton Head, SC. vol. I, 246--253.Google ScholarGoogle Scholar
  40. Tenenbaum, J. M. and Barrow, H. G. 1977. Experiments in interpretation-guided segmentation. Artific. Intell. 8, 3, 241--274.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Veltkamp, R. C. and Hagedoorn, M. 1999. State of the art in shape matching. Tech. rep. UU-CS-1999-27, Utrecht University, Netherlands.Google ScholarGoogle Scholar
  42. Wang, C. and Srihari, S. 1988. A framework for object recognition in a visually complex environment and its application to locating address blocks on mail pieces. Intl. J. Comput. Vision 2, 125--151.Google ScholarGoogle ScholarCross RefCross Ref
  43. Wang, J. and Jean, J. 1993. Segmentation of merged characters by neural networks and shortest-path. In Proceedings of the 1993 ACM/SIGAPP Symposium on Applied Computing: States of the Art and Practice (SAC'93). ACM Press, New York, NY, 762--769. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Wang, S. and Siskind, J. M. 2003. Image segmentation with ratio cut. IEEE Trans. Pattern Anal. Mach. Intell. 25, 6, 675--690. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Xu, S., Tang, M., Lau, F. C. M., and Pan, Y. 2004. Virtual hairy brush for painterly rendering. Graph. Models 66, 5, 263--302. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Zanibbi, R., Blostein, D., and Cordy, J. R. 2002. Recognizing mathematical expressions using tree transformation. IEEE Trans. Pattern Anal. Mach. Intell. 24, 11, 1455--1467. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Zhou, R. W., Quek, C., and Ng, G. S. 1995. A novel single-pass thinning algorithm and an effective set of performance criteria. Pattern Recogn. Lett. 16, 12, 1267--1275. Google ScholarGoogle ScholarDigital LibraryDigital Library

Supplemental Material

animatepainting.avi

Index Terms

  1. Animating Chinese paintings through stroke-based decomposition

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!