HelpingHand: example-based stroke stylization

Abstract

Digital painters commonly use a tablet and stylus to drive software like Adobe Photoshop. A high quality stylus with 6 degrees of freedom (DOFs: 2D position, pressure, 2D tilt, and 1D rotation) coupled to a virtual brush simulation engine allows skilled users to produce expressive strokes in their own style. However, such devices are difficult for novices to control, and many people draw with less expensive (lower DOF) input devices. This paper presents a data-driven approach for synthesizing the 6D hand gesture data for users of low-quality input devices. Offline, we collect a library of strokes with 6D data created by trained artists. Online, given a query stroke as a series of 2D positions, we synthesize the 4D hand pose data at each sample based on samples from the library that locally match the query. This framework optionally can also modify the stroke trajectory to match characteristic shapes in the style of the library. Our algorithm outputs a 6D trajectory that can be fed into any virtual brush stroke engine to make expressive strokes for novices or users of limited hardware.

References

1. Ashikhmin, M. 2001. Synthesizing natural textures. In Symposium on Interactive 3D Graphics, 217--226.
2. Barnes, C., Goldman, D., Shechtman, E., and Finkelstein, A. 2011. The PatchMatch randomized matching algorithm for image manipulation. Commun. ACM 54 (Nov.), 103--110.
3. Baxter, W., and Lin, M. 2004. A versatile interactive 3D brush model. In Pacific Graphics, 319--328.
4. Belongie, S., Malik, J., and Puzicha, J. 2001. Shape matching and object recognition using shape contexts. Trans. on Patt. Anal. and Mach. Int. 24, 509--522.
5. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? In SIGGRAPH, vol. 28.
6. DiVerdi, S., Krishnaswamy, A., and Hadap, S. 2010. Industrial-strength painting with a virtual bristle brush. In Virtual Reality Software and Technology, 119--126.
7. Efros, A., and Leung, T. 1999. Texture synthesis by non-parametric sampling. In International Conference on Computer Vision, 1033--1038.
8. Franke, K., Schomaker, L., and Köppen, M. 2005. Pen force emulating robotic writing device and its application. In Workshop on Advanced Robotics and its Social Impacts, 36--46.
9. Goodwin, T., Vollick, I., and Hertzmann, A. 2007. Isophote distance: a shading approach to artistic stroke thickness. In Non-Photorealistic Animation and Rendering, 53--62.
10. Hertzmann, A., Oliver, N., Curless, B., and Seitz, S. 2002. Curve analogies. In Eurographics Workshop on Rendering, 233--246.
11. Hinton, G., and Nair, V. 2005. Inferring motor programs from images of handwritten digits. In Advances in Neural Information Processing, 515--522.
12. House, D., and Singh, M. 2007. Line drawing as a dynamic process. In Pacific Graphics, 351--360.
13. Hsu, E., Gentry, S., and Popović, J. 2004. Example-based control of human motion. In Symposium on Computer Animation, 69--77.
14. Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. In SIGGRAPH, 473--482.
15. McCrae, J., and Singh, K. 2009. Sketching piecewise clothoid curves. In Sketch-Based Interfaces and Modeling.
16. McCrae, J., and Singh, K. 2011. Neatening sketched strokes using piecewise french curves. In Sketch-Based Interfaces and Modeling, 141--148.
17. Muja, M., and Lowe, D. G. 2009. Fast approximate nearest neighbors with automatic algorithm configuration. In Internat. Conf. on Comp. Vision Theory and Application, 331--340.
18. Okabe, Y., Saito, S., and Nakajima, M. 2005. Paintbrush rendering of lines using HMMs. In GRAPHITE, 91--98.
19. Plamondon, R. 1995. A kinematic theory of rapid human movements. Biological Cybernetics 72, 295--320.
20. Saito, S., Kani, A., Chang, Y., and Nakajima, M. 2007. Curvature-based stroke rendering. Visual Computing 24 (November), 1--11.
21. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH, 489--498.
22. Silver, N., and Dunlap, W. P. 1987. Averaging correlation coefficients: Should fisher's z transformation be used? Journal of App. Psych. 72(1), 146--148.
23. Thiel, Y., Singh, K., and Balakrishnan, R. 2011. Elasti-curves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves. In User Interface Software and Technology, 383--392.
24. Vandoren, P., Van Laerhoven, T., Claesen, L., Taelman, J., Raymaekers, C., and Van Reeth, F. 2008. IntuPaint: Bridging the gap between physical and digital painting. In Horizontal Interactive Human Computer Systems, 65--72.
25. Varga, T., Kilchhofer, D., and Bunke, H. 2005. Template-based synthetic handwriting generation for the training of recognition systems. In Conference of the International Graphonomics Society, 206--211.
26. Wei, L.-Y., Lefebvre, S., Kwatra, V., and Turk, G. 2009. State of the art in example-based texture synthesis. In Eurographics 2009, State of the Art Report.
27. Yu, K., Wang, Y., and Tan, T. 2004. Writer identification using dynamic features. In Biometric Authentication, D. Zhang and A. Jain, Eds., vol. 3072 of Lec. Notes in Comp. Sci. 1--8.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces