RealBrush: painting with examples of physical media

Abstract

Conventional digital painting systems rely on procedural rules and physical simulation to render paint strokes. We present an interactive, data-driven painting system that uses scanned images of real natural media to synthesize both new strokes and complex stroke interactions, obviating the need for physical simulation. First, users capture images of real media, including examples of isolated strokes, pairs of overlapping strokes, and smudged strokes. Online, the user inputs a new stroke path, and our system synthesizes its 2D texture appearance with optional smearing or smudging when strokes overlap. We demonstrate high-fidelity paintings that closely resemble the captured media style, and also quantitatively evaluate our synthesis quality via user studies.

References

1. Ando, R., and Tsuruno, R. 2010. Segmental brush synthesis with stroke images. In Proceedings of Eurographics -- Short papers, 89--92.
2. Baxter, W., and Govindaraju, N. 2010. Simple data-driven modeling of brushes. In Proceedings of I3D, 135--142.
3. Baxter, W. V., Wendt, J., and Lin, M. C. 2004. IMPaSTo: A realistic, interactive model for paint. In Proceedings of NPAR, 45--56.
4. Chu, N., and Tai, C.-L. 2005. MoXi: Real-time ink dispersion in absorbent paper. In Proceedings of SIGGRAPH, 504--511.
5. Chu, N., Baxter, W., Wei, L.-Y., and Govindaraju, N. 2010. Detail-preserving paint modeling for 3d brushes. In Proceedings of NPAR, 27--34.
6. Darabi, S., Shechtman, E., Barnes, C., Goldman, D., and Sen, P. 2012. Image melding: combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics 31, 4, 82:1--82:10.
7. DiVerdi, S., Krishnaswamy, A., and Hadap, S. 2010. Industrial-strength painting with a virtual bristle brush. In Proceedings of Virtual Reality Science and Technology, 119--126.
8. DiVerdi, S., Krishnaswamy, A., Měch, R., and Ito, D. 2012. A lightweight, procedural, vector watercolor painting engine. In Proceedings of I3D, 63--70.
9. Hjelle, Ø. 2001. Approximation of scattered data with multilevel b-splines. Tech. rep., SINTEF.
10. Hsu, S. C., and Lee, I. H. H. 1994. Drawing and animation using skeletal strokes. In Proceedings of SIGGRAPH, 109--118.
11. Kim, M., and Shin, H. J. 2010. An example-based approach to synthesize artistic strokes using graphs. Computer Graphics Forum 29, 7, 2145--2152.
12. Kwatra, V., Schodl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics 22, 3, 277--286.
13. Lu, J., Yu, F., Finkelstein, A., and DiVerdi, S. 2012. Helpinghand: Example-based stroke stylization. In Proceedings of SIGGRAPH, 46:1--46:10.
14. Pitié, F., Kokaram, A., and Dahyot, R. 2007. Automated colour grading using colour distribution transfer. Computer Vision and Image Understanding 107, 1, 123--137.
15. Ritter, L., Li, W., Curless, B., Agrawala, M., and Salesin, D. 2006. Painting with texture. In Proceedings of the Eurographics conference on Rendering Techniques, 371--376.
16. Schretter, C. 2005. A brush tool for interactive texture synthesis. ICGST International Journal on Graphics, Vision and Image Processing 6, 5, 55--60.
17. Van Haevre, W., Van Laerhoven, T., Di Fiore, F., and Van Reeth, F. 2007. From dust till drawn: A real-time bidirectional pastel simulation. The Visual Computer, 925--934.
18. Wang, B., Wang, W., Yang, H., and Sun, J. 2004. Efficient example-based painting and synthesis of 2D directional texture. IEEE Trans. Visualization and Computer Graphics, 3, 266--277.
19. Xie, N., Laga, H., Saito, S., and Nakajima, M. 2011. Contour-driven sumi-e rendering of real photos. Computers & Graphics 35, 1, 122--134.
20. Xie, N., Hachiya, H., and Sugiyama, M. 2012. Artist agent: A reinforcement learning approach to automatic stroke generation in oriental ink painting. In Proceedings of the International Conference on Machine Learning, 153--160.
21. Xu, M., and Dong, J. 2006. Generating new styles of chinese stroke based on statistical model. In Proc. International Multiconference on Computer Science and I. T., 215--222.
22. Xu, S., Tang, M., Lau, F. C. M., and Pan, Y. 2004. Virtual hairy brush for painterly rendering. Graphical Models 66, 5.
23. Xu, S., Xu, Y., Kang, S. B., Salesin, D. H., Pan, Y., and Shum, H.-Y. 2006. Animating chinese paintings through stroke-based decomposition. ACM Transactions on Graphics 25, 2, 239--267.
24. Xu, S., Tan, H., Jiao, X., Lau, F., and Pan, Y. 2007. A generic pigment model for digital painting. Computer Graphics Forum 26, 609--618.
25. Yan, C.-R., Chi, M.-T., Lee, T.-Y., and Lin, W.-C. 2008. Stylized rendering using samples of a painted image. IEEE Trans. Visualization and Computer Graphics 14, 2, 468--480.
26. Zeng, K., Zhao, M., Xiong, C., and Zhu, S.-C. 2009. From image parsing to painterly rendering. ACM Transactions on Graphics 29, 1.
27. Zhou, S., Lasram, A., and Lefebvre, S. 2013. By-example synthesis of curvilinear structured patterns. Computer Graphics Forum 32, 2.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces