Direct delta mush skinning compression with continuous examples

Abstract

Direct Delta Mush (DDM) is a high-quality, direct skinning method with a low setup cost. However, its storage and run-time computing cost are relatively high for two reasons: its skinning weights are 4 X 4 matrices instead of scalars like other direct skinning methods, and its computation requires one 3 X 3 Singular Value Decomposition per vertex.

In this paper, we introduce a compression method that takes a DDM model and splits it into two layers: the first layer is a smaller DDM model that computes a set of virtual bone transformations and the second layer is a Linear Blend Skinning model that computes per-vertex transformations from the output of the first layer. The two-layer model can approximate the deformation of the original DDM model with significantly lower costs.

Our main contribution is a novel problem formulation for the DDM compression based on a continuous example-based technique, in which we minimize the compression error on an uncountable set of example poses. This formulation provides an elegant metric for the compression error and simplifies the problem to the common linear matrix factorization. Our formulation also takes into account the skeleton hierarchy of the model, the bind pose, and the range of motions. In addition, we propose a new update rule to optimize DDM weights of the first layer and a modification to resolve the floating-point cancellation issue of DDM.

References

1. M. Aharon, M. Elad, and A. Bruckstein. 2006. K -SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Trans. Sig. Proc. 54, 11 (Nov. 2006), 4311--4322.
2. Marc Alexa. 2002. Linear Combination of Transformations. ACM Trans. Graph. 21, 3 (July 2002), 380--387.
3. Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: Shape Completion and Animation of People. ACM Trans. Graph. 24, 3 (July 2005), 408--416.
4. Paolo Baerlocher. 2001. Inverse kinematics techniques of the interactive posture control of articulated figures. Ph.D. Dissertation. École Polytechnique Fédérale de Lausanne.
5. Stephen W. Bailey, Dave Otte, Paul Dilorenzo, and James F. O'Brien. 2018. Fast and Deep Deformation Approximations. ACM Trans. Graph. 37, 4, Article 119 (July 2018).
6. Ake Bjorck. 1996. Numerical methods for least squares problems.
7. Wei-Wen Feng, Byung-Uck Kim, and Yizhou Yu. 2008. Real-time Data Driven Deformation Using Kernel Canonical Correlation Analysis. ACM Trans. Graph. 27, 3, Article 91 (Aug. 2008).
8. Lin Gao, Yu-Kun Lai, Dun Liang, Shu-Yu Chen, and Shihong Xia. 2016. Efficient and Flexible Deformation Representation for Data-Driven Surface Modeling. ACM Trans. Graph. 35, 5, Article 158 (July 2016).
9. Fabian Hahn, Sebastian Martin, Bernhard Thomaszewski, Robert Sumner, Stelian Coros, and Markus Gross. 2012. Rig-space Physics. ACM Trans. Graph. 31, 4, Article 72 (July 2012).
10. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W. Sumner, and Markus Gross. 2013. Efficient Simulation of Secondary Motion in Rig-space. In Proc. of ACM/Eurographics Symposium on Computer Animation (SCA 2013). 165--171.
11. Nils Hasler, Thorsten Thormählen, Bodo Rosenhahn, and Hans-Peter Seidel. 2010. Learning Skeletons for Shape and Pose. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2010). Association for Computing Machinery, New York, NY, USA, 23--30.
12. Alexandru-Eugen Ichim, Petr Kadleček, Ladislav Kavan, and Mark Pauly. 2017. Phace: Physics-based Face Modeling and Animation. ACM Trans. Graph. 36, 4, Article 153 (July 2017).
13. Alec Jacobson, Ilya Baran, Ladislav Kavan, Jovan Popović, and Olga Sorkine. 2012. Fast Automatic Skinning Transformations. ACM Trans. Graph. 31, 4, Article 77 (July 2012).
14. Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-Time Deformation. ACM Trans. Graph. 30, 4, Article 78 (July 2011).
15. Alec Jacobson, Zhigang Deng, Ladislav Kavan, and JP Lewis. 2014. Skinning: Real-time Shape Deformation. In ACM SIGGRAPH 2014 Courses. http://skinning.org/
16. Alec Jacobson and Olga Sorkine. 2011. Stretchable and Twistable Bones for Skeletal Shape Deformation. ACM Trans. Graph. 30, 6 (Dec. 2011).
17. Doug L. James and Christopher D. Twigg. 2005. Skinning Mesh Animations. ACM Trans. Graph. 24, 3 (July 2005), 399--407.
18. Ben Jones, Nils Thuerey, Tamar Shinar, and Adam W. Bargteil. 2016. Example-based Plastic Deformation of Rigid Bodies. ACM Trans. Graph. 35, 4, Article 34 (July 2016).
19. Petr Kadleček, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Křivánek, and Ladislav Kavan. 2016. Reconstructing Personalized Anatomical Models for Physics-based Body Animation. ACM Trans. Graph. 35, 6, Article 213 (Nov. 2016).
20. Ladislav Kavan, Steven Collins, and Carol O'Sullivan. 2009. Automatic Linearization of Nonlinear Skinning. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2009). 49--56.
21. Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O'Sullivan. 2008. Geometric Skinning with Approximate Dual Quaternion Blending. ACM Trans. Graph. 27, 4, Article 105 (Nov. 2008).
22. L. Kavan, P.-P. Sloan, and C. O'Sullivan. 2010. Fast and Efficient Skinning of Animated Meshes. Computer Graphics Forum 29, 2 (2010), 327--336.
23. Ladislav Kavan and Olga Sorkine. 2012. Elasticity-Inspired Deformers for Character Articulation. ACM Trans. Graph. 31, 6, Article 196 (Nov. 2012).
24. Ladislav Kavan and Jiří Žára. 2005. Spherical Blend Skinning: A Real-time Deformation of Articulated Models. In Proc. ACM Symposium on Interactive 3D Graphics and Games (I3D 2005). 9--16.
25. Hyunsoo Kim and Haesun Park. 2008. Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method. SIAM J. Matrix Anal. Appl. 30, 2 (July 2008), 713--730.
26. Paul G. Kry, Doug L. James, and Dinesh K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In Proc. of ACM/Eurographics Symposium on Computer Animation (SCA 2002). 153--159.
27. Eric Landreneau and Scott Schaefer. 2010. Poisson-Based Weight Reduction of Animated Meshes. Computer Graphics Forum 29, 6 (2010), 1945--1954.
28. Binh Huy Le and Zhigang Deng. 2012. Smooth Skinning Decomposition with Rigid Bones. ACM Trans. Graph. 31, 6, Article 199 (Nov. 2012).
29. Binh Huy Le and Zhigang Deng. 2013. Two-layer Sparse Compression of Dense-weight Blend Skinning. ACM Trans. Graph. 32, 4, Article 124 (July 2013).
30. Binh Huy Le and Zhigang Deng. 2014. Robust and Accurate Skeletal Rigging from Mesh Sequences. ACM Trans. Graph. 33, 4, Article 84 (July 2014).
31. Binh Huy Le and Jessica K. Hodgins. 2016. Real-time Skeletal Skinning with Optimized Centers of Rotation. ACM Trans. Graph. 35, 4, Article 37 (July2016).
32. Binh Huy Le and J P Lewis. 2019. Direct Delta Mush Skinning and Variants. ACM Trans. Graph. 38, 4, Article 113 (July 2019).
33. Daniel D. Lee and H. Sebastian Seung. 2000. Algorithms for Non-Negative Matrix Factorization. In Proc. of International Conference on Neural Information Processing (NIPS 2000). 535--541.
34. Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive Biomechanical Modeling and Simulation of the Upper Body. ACM Trans. Graph. 28, 4, Article 99 (Sept. 2009).
35. J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-driven Deformation. In Proc. of International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2000). 165--172.
36. Duo Li, Shinjiro Sueda, Debanga R. Neog, and Dinesh K. Pai. 2013. Thin Skin Elastodynamics. ACM Trans. Graph. 32, 4, Article 49 (July 2013).
37. Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-driven Soft Body Characters. ACM Trans. Graph. 32, 6, Article 215 (Nov. 2013).
38. S. Lloyd. 1982. Least Squares Quantization in PCM. IEEE Transactions on Information Theory 28, 2 (Sept. 1982), 129--137.
39. Matthew Loper, Naureen Mahmood, and Michael J. Black. 2014. MoSh: Motion and Shape Capture from Sparse Markers. ACM Trans. Graph. 33, 6, Article 220 (Nov. 2014).
40. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (Oct. 2015).
41. N. Magnenat-Thalmann, F. Cordier, Hyewon Seo, and G. Papagianakis. 2004. Modeling of bodies and clothes for virtual environments. In IEEE International Conference on Cyberworlds (CW 2004). 201--208.
42. N. Magnenat-Thalmann, R. Laperrière, and D. Thalmann. 1988. Joint-dependent Local Deformations for Hand Animation and Object Grasping. In Proc. on Graphics Interface (GI 1988). 26--33. http://dl.acm.org/citation.cfm?id=102313.102317
43. Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. 2010. Online Learning for Matrix Factorization and Sparse Coding. Journal of Machine Learning Research 11 (March 2010), 19--60. http://dl.acm.org/citation.cfm?id=1756006.1756008
44. Joe Mancewicz, Matt L. Derksen, Hans Rijpkema, and Cyrus A. Wilson. 2014. Delta Mush: Smoothing Deformations While Preserving Detail. In Proc. of ACM Symposium on Digital Production (DigiPro 2014). 7--11.
45. Aleka McAdams, Andrew Selle, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011a. Computing the singular value decomposition of 3x3 matrices with minimal branching and elementary floating point operations. Technical Report. University of Wisconsin-Madison.
46. Aleka McAdams, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. 2011b. Efficient Elasticity for Character Skinning with Contact and Collisions. ACM Trans. Graph. 30, 4, Article 37 (July 2011).
47. Alex Mohr and Michael Gleicher. 2003. Building Efficient, Accurate Character Skins from Examples. ACM Trans. Graph. 22, 3 (July 2003), 562--568.
48. Tomohiko Mukai. 2015. Building Helper Bone Rigs from Examples. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2015). 77--84.
49. Tomohiko Mukai and Shigeru Kuriyama. 2016. Efficient Dynamic Skinning with Low-rank Helper Bone Controllers. ACM Trans. Graph. 35, 4, Article 36 (July 2016).
50. Olivier Rémillard and Paul G. Kry. 2013. Embedded Thin Shells for Wrinkle Simulation. ACM Trans. Graph. 32, 4, Article 50 (July 2013).
51. Shunsuke Saito, Zi-Ye Zhou, and Ladislav Kavan. 2015. Computational Bodybuilding: Anatomically-based Modeling of Human Bodies. ACM Trans. Graph. 34, 4, Article 41 (July 2015).
52. Thomas Schlömer, Daniel Heck, and Oliver Deussen. 2011. Farthest-Point Optimized Point Sets with Maximized Minimum Distance. In Proc. of ACM SIGGRAPH Symposium on High Performance Graphics (HPG 2011). 135--142.
53. Peter-Pike J. Sloan, Charles F. Rose, III, and Michael F. Cohen. 2001. Shape by Example. In Proc. of ACM Symposium on Interactive 3D Graphics and Games (I3D 2001). 135--143.
54. Breannan Smith, Fernando De Goes, and Theodore Kim. 2018. Stable Neo-Hookean Flesh Simulation. ACM Trans. Graph. 37, 2, Article 12 (March 2018).
55. Olga Sorkine and Marc Alexa. 2007. As-Rigid-as-Possible Surface Modeling. In Proc. of Eurographics Symposium on Geometry Processing (SGP 2007). 109--116.
56. Robert W. Sumner and Jovan Popović. 2004. Deformation Transfer for Triangle Meshes. ACM Trans. Graph. 23, 3 (Aug. 2004), 399--405.
57. Robert W. Sumner, Matthias Zwicker, Craig Gotsman, and Jovan Popović. 2005. Mesh-based Inverse Kinematics. ACM Trans.Graph. 24, 3 (July 2005), 488--495.
58. Joseph Teran, Eftychios Sifakis, Geoffrey Irving, and Ronald Fedkiw. 2005. Robust Quasistatic Finite Elements and Flesh Simulation. In Proc. of ACM/Eurographics Symposium on Computer Animation (SCA 2005). 181--190.
59. Rodolphe Vaillant, Loïc Barthe, Gaël Guennebaud, Marie-Paule Cani, Damien Rohmer, Brian Wyvill, Olivier Gourmel, and Mathias Paulin. 2013. Implicit Skinning: Real-time Skin Deformation with Contact Modeling. ACM Trans. Graph. 32, 4, Article 125 (July 2013).
60. Rodolphe Vaillant, Gäel Guennebaud, Loïc Barthe, Brian Wyvill, and Marie-Paule Cani. 2014. Robust Iso-surface Tracking for Interactive Character Skinning. ACM Trans. Graph. 33, 6, Article 189 (Nov. 2014).
61. Robert Y. Wang, Kari Pulli, and Jovan Popović. 2007. Real-time Enveloping with Rotational Regression. ACM Trans. Graph. 26, 3, Article 73 (July 2007).

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation