Painting by feature: texture boundaries for example-based image creation

Abstract

In this paper we propose a reinterpretation of the brush and the fill tools for digital image painting. The core idea is to provide an intuitive approach that allows users to paint in the visual style of arbitrary example images. Rather than a static library of colors, brushes, or fill patterns, we offer users entire images as their palette, from which they can select arbitrary contours or textures as their brush or fill tool in their own creations. Compared to previous example-based techniques related to the painting-by-numbers paradigm we propose a new strategy where users can generate salient texture boundaries by our randomized graph-traversal algorithm and apply a content-aware fill to transfer textures into the delimited regions. This workflow allows users of our system to intuitively create visually appealing images that better preserve the visual richness and fluidity of arbitrary example images. We demonstrate the potential of our approach in various applications including interactive image creation, editing and vector image stylization.

References

1. Ashikhmin, M. 2001. Synthesizing natural textures. In Proceedings of Symposium on Interactive 3D graphics, 217--226.
2. Assa, J., and Cohen-Or, D. 2012. More of the same: Synthesizing a variety by structural layering. Computers & Graphics 36, 4, 250--256.
3. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics 28, 3, 24:1--24:11.
4. Burt, J. R., and Adelson, E. H. 1983. A multiresolution spline with application to image mosaics. ACM Transactions on Graphics 2, 4, 217--236.
5. Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. 1997. Computer-generated watercolor. In Proceedings of SIGGRAPH 97, 421--430.
6. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics 31, 4, 82:1--82:10.
7. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., and Santella, A. 2003. Suggestive contours for conveying shape. ACM Transactions on Graphics 22, 3, 848--855.
8. Dijkstra, E. 1959. A note on two problems in connexion with graphs. Numerische Mathematik 1, 5, 269--271.
9. DiVerdi, S., Krishnaswamy, A., Mech, R., and Ito, D. 2013. Painting with polygons: A procedural watercolor engine. IEEE Transactions on Visualization and Computer Graphics 19, 5, 723--735.
10. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Transactions on Graphics 21, 3, 257--266.
11. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In Proceedings of SIGGRAPH 2001, 341--346.
12. Fang, H., and Hart, J. C. 2007. Detail preserving shape deformation in image editing. ACM Transactions on Graphics 26, 3, 12:1--12:5.
13. Guo, C.-e., Zhu, S.-C., and Wu, Y. N. 2007. Primal sketch: Integrating structure and texture. Computer Vision and Image Understanding 106, 1, 5--19.
14. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of SIGGRAPH 2001, 327--340.
15. Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. International Journal of Computer Vision 1, 4, 321--331.
16. Kwatra, V., Schödl, A., Essa, I. A., Turk, G., and Bobick, A. F. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics 22, 3, 277--286.
17. Kwatra, V., Essa, I. A., Bobick, A. F., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Transactions on Graphics 24, 3, 795--802.
18. Lefebvre, S., and Hoppe, H. 2005. Parallel controllable texture synthesis. ACM Transactions on Graphics 24, 3, 777--786.
19. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Transactions on Graphics 23, 3, 303--308.
20. McCann, J., and Pollard, N. S. 2008. Real-time gradient-domain painting. ACM Transactions on Graphics 27, 3, 93:1--93:7.
21. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., and Salesin, D. 2008. Diffusion curves: A vector representation for smooth-shaded images. ACM Transactions on Graphics 27, 3, 92:1--92:8.
22. Risser, E., Han, C., Dahyot, R., and Grinspun, E. 2010. Synthesizing structured image hybrids. ACM Transactions on Graphics 29, 4, 85:1--85:6.
23. Ritter, L., Li, W., Curless, B., Agrawala, M., and Salesin, D. 2006. Painting with texture. In Proceedings of Eurographics Symposium on Rendering, 371--376.
24. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proceedings of SIGGRAPH 2000, 489--498.
25. Shechtman, E., Rav-Acha, A., Irani, M., and Seitz, S. M. 2010. Regenerative morphing. In IEEE Conference on Computer Vision and Pattern Recognition, 615--622.
26. Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Transactions on Graphics 24, 3, 861--868.
27. Todorovic, S., and Ahuja, N. 2009. Texel-based texture segmentation. In IEEE International Conference on Computer Vision, 841--848.
28. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 3, 463--476.
29. Yücer, K., Jacobson, A., Hornung, A., and Sorkine, O. 2012. Transfusive image manipulation. ACM Transactions on Graphics 31, 6, 176:1--176:9.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

    • Image manipulation

    • Shape modeling

      • Parametric curve and surface models

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Graphics input devices