Capturing and animating skin deformation in human motion

Abstract

During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors.

References

1. Albrecht, I., Haber, J., and Seidel, H.-P. 2003. Construction and animation of anatomically based human hand models. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 98--109.
2. Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Transactions on Graphics 21, 3, 612--619.
3. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Transactions on Graphics 22, 3, 587--594.
4. Anguelov, D., Koller, D., Pang, H., Srinivasan, P., and Thrun, S. 2004. Recovering articulated object models from 3d range data. In the 20th Conference on Uncertainty in Artificial Intelligence, 18--26.
5. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: shape completion and animation of people. ACM Transactions on Graphics 24, 3, 408--416.
6. Chadwick, J. E., Haumann, D. R., and Parent, R. E. 1989. Layered construction for deformable animated characters. Computer Graphics (Proceedings of SIGGRAPH 89) 23, 3, 243--252.
7. Chai, J., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3d facial animation. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 193--206.
8. Choe, B., Lee, H., and Ko, H.-S. 2001. Performance-driven muscle-based facial animation. The Journal of Visualization and Computer Animation 12, 2, 67--79.
9. Chuang, E., and Bregler, C. 2005. Mood swings: expressive speech animation. ACM Transactions on Graphics 24, 2, 331--347.
10. Cosker, D., Paddock, S., Marshall, D., Rosin, P. L., and Rushton, S. 2004. Towards perceptually realistic talking heads: models, methods and mcgurk. In APGV 2004, 151--157.
11. Dong, F., Clapworthy, G. J., Krokos, M. A., and Yao, J. 2002. An anatomy-based approach to human muscle modeling and deformation. IEEE Transactions on Visualization and Computer Graphics 8, 2, 154--170.
12. Fidaleo, D., and Neumann, U. 2004. Analysis of co-articulation regions for performance-driven facial animation. Computer Animation and Virtual Worlds 15, 1, 15--26.
13. Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 55--66.
14. Guo, Z., and Wong, K. C. 2005. Skinning with deformable chunks. Computer Graphics Forum 24, 3, 373--382.
15. Horn, B. K. P. 1987. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A 4, 4, 629--642.
16. Huang, K.-S., Chang, C.-F., Hsu, Y.-Y., and Yang, S.-N. 2005. Key probe: a technique for animation keyframe extraction. The Visual Computer 21, 8--10, 532--541.
17. Hwang, B.-W., and Lee, S.-W. 2003. Reconstruction of partially damaged face images based on a morphable face model. IEEE Trans. Pattern Anal. Mach. Intell. 25, 3, 365--372.
18. Hyun, D.-E., Yoon, S.-H., Chang, J.-W., Seong, J.-K., Kim, M.-S., and Jüttler, B. 2005. Sweep-based human deformation. The Visual Computer 21, 8--10, 542--550.
19. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Transactions on Graphics 24, 3, 1134--1141.
20. James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM Transactions on Graphics 24, 3, 399--407.
21. Kanatani, K. 1994. Analysis of 3-d rotation fitting. IEEE Trans. Pattern Anal. Mach. Intell. 16, 5, 543--549.
22. Kavan, L., and Zara, J. 2005. Spherical blend skinning: A real-time deformation of articulated models. In 2005 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, ACM Press, 9--16.
23. Kshirsagar, S., Molet, T., and Magnenat-Thalmann, N. 2001. Principal components of expressive speech animation. In Computer Graphics International 2001, 38--44.
24. Kurihara, T., and Miyata, N. 2004. Modeling deformable human hands from medical images. In 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 355--363.
25. La Torre, F. D., and Black, M. J. 2001. Dynamic coupled component analysis. In CVPR, 643--650.
26. Larboulette, C., Cani, M.-P., and Arnaldi, B. 2005. Dynamic skinning: adding real-time dynamic effects to an existing character animation. In Spring Conference on Computer Graphics 2005, 87--93.
27. Lemos, R. R., Rokne, J., Baranoski, G. V. G., Kawakami, Y., and Kurihara, T. 2005. Modeling and simulating the deformation of human skeletal muscle based on anatomy and physiology. Computer Animation and Virtual Worlds 16, 3--4, 319--330.
28. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformations: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 165--172.
29. Lin, I.-C., and Ouhyoung, M. 2005. Mirror mocap: Automatic and efficient capture of dense 3d facial motion parameters from video. The Visual Computer 21, 6, 355--372.
30. Lin, I.-C., Yeng, J.-S., and Ouhyoung, M. 2002. Extracting 3d facial animation parameters from multiview video clips. IEEE Computer Graphics & Applications 22, 6, 72--80.
31. Lipman, Y., Sorkine, O., Levin, D., and Cohen-Or, D. 2005. Linear rotation-invariant coordinates for meshes. ACM Transactions on Graphics 24, 3, 479--487.
32. Magnenat-Thalmann, N., and Thalmann, D. 2005. Virtual humans: thirty years of research, what next? The Visual Computer 21, 12, 997--1015.
33. Mohr, A., and Gleicher, M. 2003. Building efficient, accurate character skins from examples. ACM Transactions on Graphics 22, 3, 562--568.
34. Müller, M., Heidelberger, B., Teschner, M., and Gross, M. 2005. Meshless deformations based on shape matching. ACM Transactions on Graphics 24, 3, 471--478.
35. Nedel, L. P., and Thalmann, D. 2000. Anatomic modeling of deformable human bodies. The Visual Computer 16, 6, 306--321.
36. Pratscher, M., Coleman, P., Laszlo, J., and Singh, K. 2005. Outside-in anatomy based character rigging. In 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 329--338.
37. Pronost, N., Dumont, G., Berillon, G., and Nicolas, G. 2006. Morphological and stance interpolations in database for simulating bipedalism of virtual humans. The Visual Computer 22, 1, 4--13.
38. Sand, P., Mcmillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM Transactions on Graphics 22, 3, 578--586.
39. Scheepers, F., Parent, R. E., Carlson, W. E., and May, S. F. 1997. Anatomy-based modeling of the human musculature. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 163--172.
40. Seo, H., and Magnenat-Thalmann, N. 2003. An automatic modeling of human bodies from sizing parameters. In 2003 ACM Symposium on Interactive 3D Graphics, 19--26.
41. Seo, H., Cordier, F., and Magnenat-Thalmann, N. 2003. Synthesizing animatable body models with parameterized shape modifications. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 120--125.
42. Shum, H.-Y., Ikeuchi, K., and Reddy, R. 1995. Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 17, 9, 854--867.
43. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Transactions on Graphics 24, 3, 417--425.
44. Singh, K., and Kokkevis, E. 2000. Skinning characters using surface oriented free-form deformations. In Graphics Interface, 35--42.
45. Sloan, P.-P. J., III, C. F. R., and Cohen, M. F. 2001. Shape by example. In 2001 ACM Symposium on Interactive 3D Graphics, 135--144.
46. Sumner, R. W., Zwicker, M., Gotsman, C., and Popović, J. 2005. Mesh-based inverse kinematics. ACM Transactions on Graphics 24, 3, 488--495.
47. Sun, W., Hilton, A., Smith, R., and Illingworth, J. 2001. Layered animation of captured data. The Visual Computer 17, 8, 457--474.
48. Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Transactions on Visualization and Computer Graphics 11, 3, 317--328.
49. Teran, J., Sifakis, E., Irving, G., and Fedkiw, R. 2005. Robust quasistatic finite elements and flesh simulation. In 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 181--190.
50. Vicon Motion Systems, 2006. http://www.vicon.com/.
51. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Transactions on Graphics 24, 3, 426--433.
52. Wallraven, C., Breidt, M., Cunningham, D. W., and Bülthoff, H. H. 2005. Psychophysical evaluation of animated facial expressions. In APGV 2005, 17--24.
53. Wang, X. C., and Phillips, C. 2002. Multi-weight enveloping: Least-squares approximation techniques for skin animation. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 129--138.
54. Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dynamic 3-d facial expressions. Computer Graphics Forum 23, 3, 677--686.
55. Wilhelms, J., and Gelder, A. V. 1997. Anatomically based modeling. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 173--180.
56. Zalewski, L., and Gong, S. 2005. 2d statistical models of facial expressions for realistic 3d avatar animation. In CVPR, 217--222.
57. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3, 548--558.
58. Zordan, V. B., and Horst, N. C. V. D. 2003. Mapping optical motion capture data to skeletal motion using a physical model. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 245--250.
59. Zordan, V. B., Celly, B., Chiu, B., and Dilorenzo, P. C. 2004. Breathe easy: model and control of simulated respiration for animation. In 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 29--37.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation