Abstract
Character rigging is universally needed in computer graphics but notoriously laborious. We present a new method, HeterSkinNet, aiming to fully automate such processes and significantly boost productivity. Given a character mesh and skeleton as input, our method builds a heterogeneous graph that treats the mesh vertices and the skeletal bones as nodes of different types and uses graph convolutions to learn their relationships. To tackle the graph heterogeneity, we propose a new graph network convolution operator that transfers information between heterogeneous nodes. The convolution is based on a new distance HollowDist that quantifies the relations between mesh vertices and bones. We show that HeterSkinNet is robust for production characters by providing the ability to incorporate meshes and skeletons with arbitrary topologies and morphologies (e.g., out-of-body bones, disconnected mesh components, etc.). Through exhaustive comparisons, we show that HeterSkinNet outperforms state-of-the-art methods by large margins in terms of rigging accuracy and naturalness. HeterSkinNet provides a solution for effective and robust character rigging.
- Marc Alexa. 2002. Linear Combination of Transformations. ACM Transactions on Graphics (TOG) 21, 3 (2002), 380--387. Autodesk. 2020. Maya. https://www.autodesk.com/products/maya/overviewGoogle Scholar
Digital Library
- Stephen W. Bailey, Dave Otte, Paul Dilorenzo, and James F. O'Brien. 2018. Fast and Deep Deformation Approximations. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--12.Google Scholar
Digital Library
- Ilya Baran and Jovan Popović. 2007. Automatic Rigging and Animation of 3D Characters. ACM Transactions on Graphics (TOG) 26, 3 (2007), 72-es.Google Scholar
Digital Library
- Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. 2016. Learning Shape Correspondence with Anisotropic Convolutional Neural Networks. In Advances in Neural Information Processing Systems (NIPS), Vol. 29. 3189--3197.Google Scholar
- Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. 2017. Geometric Deep Learning: Going beyond Euclidean Data. IEEE Signal Processing Magazine (SPM) 34, 4 (2017), 18--42.Google Scholar
Cross Ref
- Olivier Dionne and Martin de Lasa. 2013. Geodesic Voxel Binding for Production Character Meshes. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA). 173--180.Google Scholar
Digital Library
- Olivier Dionne and Martin De Lasa. 2014. Geodesic Binding for Degenerate Character Geometry using Sparse Voxelization. IEEE Transactions on Visualization and Computer Graphics (TVCG) 20, 10 (2014), 1367--1378.Google Scholar
Cross Ref
- Elmar Eisemann and Xavier Décoret. 2008. Single-Pass GPU Solid Voxelization for Real-Time Applications. In Proceedings of Graphics Interface (GI). 73--80.Google Scholar
- Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.Google Scholar
- Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. 2019. MeshCNN: A Network with an Edge. ACM Transactions on Graphics (TOG) 38, 4 (July 2019), 1--12.Google Scholar
Digital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on Imagenet Classification. In Proceedings of the IEEE International Conference on Computer Vision(ICCV). 1026--1034.Google Scholar
Digital Library
- Alec Jacobson, Ilya Baran, Jovan Popovic, and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-Time Deformation. ACM Transactions on Graphics (TOG) 30, 4 (2011), 78.Google Scholar
Digital Library
- Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O'Sullivan. 2007. Skinning with Dual Quaternions. In ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D). 39--46.Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014).Google Scholar
- Tianxing Li, Rui Shi, and Takashi Kanai. 2020. DenseGATs: A Graph-Attention-Based Network for Nonlinear Character Deformation. In ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D). 1--9.Google Scholar
- Hu Linmei, Tianchi Yang, Chuan Shi, Houye Ji, and Xiaoli Li. 2019. Heterogeneous Graph Attention Networks for Semi-Supervised Short Text Classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 4820--4829.Google Scholar
Cross Ref
- Lijuan Liu, Youyi Zheng, Di Tang, Yi Yuan, Changjie Fan, and Kun Zhou. 2019. NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks. ACM Transactions on Graphics (TOG) 38, 4 (July 2019), 1--12.Google Scholar
Digital Library
- Ran Luo, Tianjia Shao, Huamin Wang, Weiwei Xu, Xiang Chen, Kun Zhou, and Yin Yang. 2020. NNWarp: Neural Network-Based Nonlinear Deformation. IEEE Transactions on Visualization and Computer Graphics (TVCG) 26 (April 2020), 1745--1759.Google Scholar
- Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. 2015. Geodesic Convolutional Neural Networks on Riemannian Manifolds. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW). 37--45.Google Scholar
Digital Library
- Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. 2017. Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5425--5434.Google Scholar
Cross Ref
- Yi-Ling Qiao, Lin Gao, Yu-Kun Lai, and Shihong Xia. 2018. Learning Bidirectional LSTM Networks for Synthesizing 3D Mesh Animation Sequences. CoRR abs/1810.02042 (2018).Google Scholar
- R.M. Rustamov, Y. Lipman, and T. Funkhouser. 2009. Interior Distance Using Barycentric Coordinates. Computer Graphics Forum (CGF) 28, 5 (July 2009), 1279--1288.Google Scholar
Cross Ref
- Michael Schwarz and Hans-Peter Seidel. 2010. Fast Parallel Surface and Solid Voxelization on GPUs. ACM Transactions on Graphics (TOG) 29, 6, Article 179 (2010), 10 pages.Google Scholar
Digital Library
- SideFX. 2020. Houdini. https://www.sidefx.com/Google Scholar
- Justin Solomon, Raif Rustamov, Leonidas Guibas, and Adrian Butscher. 2014. Earth Mover's Distances on Discrete Surfaces. ACM Transactions on Graphics (TOG) 33, 4 (July 2014), 1--12.Google Scholar
Digital Library
- Qingyang Tan, Lin Gao, Y. Lai, Jie Yang, and S. Xia. 2018. Mesh-based Autoencoders for Localized Deformation Component Analysis. CoRR abs/1709.04304 (2018).Google Scholar
- Nitika Verma, Edmond Boyer, and Jakob Verbeek. 2018. FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2598--2606.Google Scholar
Cross Ref
- Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019a. Heterogeneous Graph Attention Network. In The World Wide Web Conference (WWW). 2022--2032.Google Scholar
- Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. 2019b. Dynamic Graph CNN for Learning on Point Clouds. ACM Transactions on Graphics (TOG) 38, 5 (2019), 1--12.Google Scholar
Digital Library
- Rich Wareham and Joan Lasenby. 2008. Bone Glow: An Improved Method for the Assignment of Weights for Mesh Deformation. In Articulated Motion and Deformable Objects (AMDO), Francisco J. Perales and Robert B. Fisher (Eds.). 63--71.Google Scholar
- Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. 2019. Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 1042--1051.Google Scholar
Cross Ref
- Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S. Yu, Philip. 2020. A Comprehensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems (TNNLS) (2020).Google Scholar
- Yun-Peng Xiao, Yu-Kun Lai, Fang-Lue Zhang, Chunpeng Li, and Lin Gao. 2020. A Survey on Deep Geometry Learning: From a Representation Perspective. Computational Visual Media (CVM) 6, 2 (2020), 113--133.Google Scholar
Cross Ref
- Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh. 2020. RigNet: Neural Rigging for Articulated Characters. ACM Transactions on Graphics (TOG) 39, 4 (2020), 1--12.Google Scholar
Digital Library
- Zhan Xu, Yang Zhou, Evangelos Kalogerakis, and Karan Singh. 2019. Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets. In International Conference on 3D Vision (3DV). 298--307.Google Scholar
Cross Ref
- Li Yi, Hao Su, Xingwen Guo, and Leonidas J. Guibas. 2017. SyncSpecCNN: Synchronized spectral CNN for 3D Shape Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2282--2290.Google Scholar
- Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2020. Deep Learning on Graphs: A Survey. IEEE Transactions on Knowledge and Data Engineering (TKDE) (2020).Google Scholar
Digital Library
- Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, and M. Sun. 2018. Graph Neural Networks: A Review of Methods and Applications. CoRR abs/1812.08434 (2018).Google Scholar
Index Terms
HeterSkinNet: A Heterogeneous Network for Skin Weights Prediction
Recommendations
RigNet: neural rigging for articulated characters
We present RigNet, an end-to-end automated method for producing animation rigs from input character models. Given an input 3D model representing an articulated character, RigNet predicts a skeleton that matches the animator expectations in joint ...
Learning skeletal articulations with neural blend shapes
Animating a newly designed character using motion capture (mocap) data is a long standing problem in computer animation. A key consideration is the skeletal structure that should correspond to the available mocap data, and the shape deformation in the ...
Morig: Motion-Aware Rigging of Character Meshes from Point Clouds
SA '22: SIGGRAPH Asia 2022 Conference PapersWe present MoRig, a method that automatically rigs character meshes driven by single-view point cloud streams capturing the motion of performing characters. Our method is also able to animate the 3D meshes according to the captured point cloud motion. ...






Comments