Three-dimensional proxies for hand-drawn characters

Abstract

Drawing shapes by hand and manipulating computer-generated objects are the two dominant forms of animation. Though each medium has its own advantages, the techniques developed for one medium are not easily leveraged in the other medium because hand animation is two-dimensional, and inferring the third dimension is mathematically ambiguous. A second challenge is that the character is a consistent three-dimensional (3D) object in computer animation while hand animators introduce geometric inconsistencies in the two-dimensional (2D) shapes to better convey a character's emotional state and personality. In this work, we identify 3D proxies to connect hand-drawn animation and 3D computer animation. We present an integrated approach to generate three levels of 3D proxies: single-points, polygonal shapes, and a full joint hierarchy. We demonstrate how this approach enables one medium to take advantage of techniques developed for the other; for example, 3D physical simulation is used to create clothes for a hand-animated character, and a traditionally trained animator is able to influence the performance of a 3D character while drawing with paper and pencil.

References

1. Agarwal, A. and Triggs, B. 2006. Recovering 3d human pose from monocular images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1, 44--58.
2. Baran, I., Vlasic, D., Grinspun, E., and Popović, J. 2009. Semantic deformation transfer. ACM Trans. Graph. 28, 3, 1--6.
3. Bourdev, L. and Malik, J. 2009. Poselets: Body part detectors trained using 3d human pose annotations. In Proceedings of the IEEE International Conference on Computer Vision.
4. Bregler, C. and Malik, J. 1998. Tracking people with twists and exponential maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
5. Cooper, D. 2002. 2D/3D Hybrid character animation on Spirit. In Proceedings of the ACM SIGGRAPH '02 Conference Abstracts and Applications Conference.
6. Corrêa, W. T., Jensen, R. J., Thayer, C. E., and Finkelstein, A. 1998. Texture mapping for cel animation. In Proceedings of the ACM SIGGRAPH '98. 435--446.
7. Culhane, S. 1990. Animation From Script to Screen. St. Martin's Press, New York.
8. Daniels, E. 1999. Deep canvas in Disney's Tarzan. In Proceedings of the ACM SIGGRAPH '99 Conference Abstracts and Applications.
9. Davis, J., Agrawala, M., Chuang, E., Popovic, Z., and Salesin, D. H. 2003. A sketching interface for articulated figure animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 320--328.
10. Demirdjian, D., Ko, T., and Darrel, T. 2003. Constraining human body tracking. In Proceedings of the IEEE International Conference on Computer Vision.
11. Elgammal, A. and Lee, C. 2004. Inferring 3d body pose from silhouettes using activity manifold learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
12. Ellis, D. 2003. Dynamic time warp (DTW) in Matlab. www.ee.columbia.edu/~dpwe/resources/matlab/dtw/.
13. Forsyth, D. A., Arikan, O., Ikemoto, L., O'Brien, J., and Ramanan, D. 2005. Computational studies of human motion: part 1, tracking and motion synthesis. Found. Trends Comput. Graph. Vis. 1, 2-3, 77--254.
14. Gleicher, M. 1998. Retargetting motion to new characters. In Proceedings of the ACM SIGGRAPH '98. 33--42.
15. Grochow, K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Implicit surface joint limits to constrain video-based motion capture. ACM Trans. Graph. 23, 3, 522--531.
16. Hartley, R. and Zisserman, A. 2003. Multiple View Geometry, 2 ed. Cambridge University Press.
17. Herda, L., Urtasun, R., and Fua, P. 2004. Implicit surface joint limits to constrain video-based motion capture. In Proceedings of the European Conference on Computer Vision. 405--418.
18. Hornung, A., Dekkers, E., and Kobbelt, L. 2007. Character animation from 2d pictures and 3d motion data. ACM Trans. Graph. 26, 1, 1--9.
19. Ikemoto, L., Arikan, O., and Forsyth, D. 2009. Generalizing motion edits with gaussian processes. ACM Trans. Graph. 28, 1, 1--12.
20. Jain, E., Sheikh, Y., and Hodgins, J. K. 2009. Leveraging the talent of hand animators to create three-dimensional animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation.
21. Jain, E., Sheikh, Y., Mahler, M., and Hodgins, J. 2010. Augmenting hand animation with three-dimensional secondary motion. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation.
22. Johnston, O. and Thomas, F. 1995. The Illusion of Life: Disney Animation. Disney Editions.
23. Johnston, S. F. 2002. Lumo: Illumination for cel animation. In Proceedings of the (NPAR'02) Symposium on Non-Photorealistic Animation and Rendering. 45--52.
24. Lasseter, J. 1994. Tricks to animating characters with a computer. In ACM SIGGRAPH Course Notes.
25. Lee, H. J. and Chen, Z. 1985. Determination of 3d human body postures from a single view. Comput. Vis. Graph. Image Process. 30, 148--168.
26. Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3, 491--500.
27. Li, Y., Gleicher, M., Xu, Y.-Q., and Shum, H.-Y. 2003. Stylizing motion with drawings. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 309--319.
28. Moeslund, T. B. and Granum, E. 2006. A survey of computer vision-based human motion capture. Comput. Vis. Image Understand. 81, 3, 231--268.
29. Petrović, L., Fujito, B., Williams, L., and Finkelstein, A. 2000. Shadows for cel animation. In Proceedings of the ACM SIGGRAPH'00. 511--516.
30. Rademacher, P. 1999. View-dependent geometry. In Proceedings of the ACM SIGGRAPH'99 Conference. 439--446.
31. Ramanan, D., Forsyth, D., and Zisserman, A. 2004. Tracking people by learning their appearance. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1, 65--81.
32. Rivers, A., Igarashi, T., and Durand, F. 2010. 2.5d cartoon models. ACM Trans. Graph. 29, 4.
33. Robertson, B. 1998. Mixed media. Comput. Graph. World, 32--35.
34. Rosenhahn, B., Brox, T., Cremers, D., and Seidel, H.-P. 2007a. Online smoothing for markerless motion capture. In Proceedings of the DAGM Symposium on Pattern Recognition 4713, 163--172.
35. Rosenhahn, B., Brox, T., Cremers, D., and Seidel, H.-P. 2008. Staying well grounded in markerless motion capture. In Proceedings of the DAGM Symposium on Pattern Recognition 5096, 385--395.
36. Rosenhahn, B., Brox, T., and Seidel, H.-P. 2007b. Scaled motion dynamics for markerless motion capture. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
37. Safonova, A., Hodgins, J. K., and Pollard, N. S. 2004. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. Graph. 23, 3.
38. Sakoe, H. and Chiba, S. 1990. Dynamic programming algorithm optimization for spoken word recognition. Read. Speech Recogn. 159--165.
39. Sidenbladh, H., Black, M., and Sigal, L. 2002. Implicit probabilistic models of human motion for synthesis and tracking. In Proceedings of the European Conference on Computer Vision.
40. Sidenbladh, H., Black, M. J., and Fleet, D. J. 2000. Stochastic tracking of 3d human figures using 2d image motion. In Proceedings of the European Conference on Computer Vision, 702--718.
41. Sminchisescu, C., Kanaujia, A., and Metaxas, D. 2005. Discriminative density propagation for 3d human motion estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
42. Sminchisescu, C. and Triggs, B. 2003. Estimating articulated human motion with covariance scaled sampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
43. Stam, J. 2009. Nucleus: Towards a unified dynamics solver for computer graphics. In Proceedings of the IEEE International Conference on Computer-Aided Design and Computer Graphics, 1--11.
44. Sýkora, D., Sedláček, D., Jinchao, S., Dingliana, J., and Collins, S. 2010. Adding depth to cartoons using sparse depth (in)equalities. Comput. Graph. Forum 29, 2, 615--623.
45. Taylor, C. J. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Comput. Vis. Image Understand. 80, 349--363.
46. Urtasun, R., Fleet, D. J., and Fua, P. 2006. Temporal motion models for monocular and multiview 3d human body tracking. Comput. Vis. Image Understand. 104, 2, 157--177.
47. Wei, X. and Chai, J. 2010. Videomocap: Modeling physically realistic human motion from monocular video sequences. ACM Trans. Graph. 29, 4, 1--10.
48. Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., and Salesin, D. H. 1997. Multiperspective panoramas for cel animation. In Proceedings of the ACM SIGGRAPH'97 Conference, 243--250.
49. Wu, Y., Hua, G., and Yu, T. 2003. Tracking articulated body by dynamic markov network. In Proceedings of the IEEE International Conference on Computer Vision.
50. Zhao, L. and Safonova, A. 2009. Achieving good connectivity in motion graphs. Graph. Models 71, 4, 139--152.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

    • Shape modeling

      • Parametric curve and surface models

      • Volumetric models