High-order differentiable autoencoder for nonlinear model reduction

Abstract

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton's method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., 10+ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable weight-varying Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.

References

1. Lars V Ahlfors. 1973. Complex Analysis. 1979.
2. Steven S An, Theodore Kim, and Doug L James. 2008. Optimizing cubature for efficient integration of subspace deformations. ACM transactions on graphics (TOG) 27, 5 (2008), 1--10.
3. Johannes Ballé, Valero Laparra, and Eero P Simoncelli. 2016. End-to-end optimized image compression. arXiv preprint arXiv:1611.01704 (2016).
4. Jernej Barbič, Marco da Silva, and Jovan Popović. 2009. Deformable object animation using reduced optimal control. In ACM SIGGRAPH 2009 papers. 1--9.
5. Jernej Barbič, Funshing Sin, and Eitan Grinspun. 2012. Interactive editing of deformable simulations. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--8.
6. Jernej Barbič and Yili Zhao. 2011. Real-time large-deformation substructuring. ACM transactions on graphics (TOG) 30, 4 (2011), 1--8.
7. Jernej Barbič and Doug L James. 2005. Real-time subspace integration for St. Venant-Kirchhoff deformable models. In ACM Trans. Graph. (TOG), Vol. 24. ACM, 982--990.
8. Atilim Günes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2017. Automatic differentiation in machine learning: a survey. The Journal of Machine Learning Research 18, 1 (2017), 5595--5637.
9. Hervé Bourlard and Yves Kamp. 1988. Auto-association by multilayer perceptrons and singular value decomposition. Biological cybernetics 59, 4--5 (1988), 291--294.
10. Alain J Brizard. 2014. Introduction To Lagrangian Mechanics, An. World Scientific Publishing Company.
11. H Martin Bücker, George Corliss, Paul Hovland, Uwe Naumann, and Boyana Norris. 2006. Automatic differentiation: applications, theory, and implementations. Vol. 50. Springer Science & Business Media.
12. Steve Capell, Seth Green, Brian Curless, Tom Duchamp, and Zoran Popović. 2002. Interactive skeleton-driven dynamic deformations. In ACM Trans. Graph. (TOG), Vol. 21. ACM, 586--593.
13. Min Gyu Choi and Hyeong-Seok Ko. 2005. Modal warping: Real-time simulation of large rotational deformation and manipulation. IEEE Trans. on Visualization and Computer Graphics 11, 1 (2005), 91--101.
14. Marco Fratarcangeli, Valentina Tibaldo, and Fabio Pellacini. 2016. Vivace: A practical gauss-seidel method for stable soft body dynamics. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--9.
15. Lawson Fulton, Vismay Modi, David Duvenaud, David IW Levin, and Alec Jacobson. 2019. Latent-space Dynamics for Reduced Deformable Simulation. In Computer graphics forum, Vol. 38. Wiley Online Library, 379--391.
16. Benjamin Gilles, Guillaume Bousquet, Francois Faure, and Dinesh K Pai. 2011. Frame-based elastic models. ACM Trans. Graph. (TOG) 30, 2 (2011), 15.
17. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. 6.2. 2.3 softmax units for multinoulli output distributions. Deep learning (2016), 180--184.
18. Gaël Guennebaud, Benoit Jacob, et al. 2010. Eigen. @URl: http://eigen.tuxfamily.org (2010).
19. Brian Guenter. 2007. Efficient symbolic differentiation for graphics applications. In ACM SIGGRAPH 2007 papers. 108--es.
20. Amirhossein Habibian, Ties van Rozendaal, Jakub M Tomczak, and Taco S Cohen. 2019. Video compression with rate-distortion autoencoders. In Proceedings of the IEEE International Conference on Computer Vision. 7033--7042.
21. Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W Sumner, Forrester Cole, Mark Meyer, Tony DeRose, and Markus Gross. 2014. Subspace clothing simulation using adaptive bases. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--9.
22. David Harmon and Denis Zorin. 2013. Subspace integration with local deformations. ACM Transactions on Graphics(TOG) 32, 4 (2013), 1--10.
23. Kris K Hauser, Chen Shen, and James F O'Brien. 2003. Interactive Deformation Using Modal Analysis with Constraints.. In Graphics Interface, Vol. 3. 16--17.
24. Florian Hecht, Yeon Jin Lee, Jonathan R Shewchuk, and James F O'Brien. 2012. Updated sparse cholesky factors for corotational elastodynamics. ACM Trans. Graph. (TOG) 31, 5 (2012), 123.
25. Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. In Neural networks for perception. Elsevier, 65--93.
26. Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. science 313, 5786 (2006), 504--507.
27. Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. 2001. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies.
28. Theodore Kim and John Delaney. 2013. Subspace fluid re-simulation. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--9.
29. Theodore Kim and Doug L James. 2009. Skipping steps in deformable simulation with online model reduction. In ACM Trans. Graph. (TOG), Vol. 28. ACM, 123.
30. Theodore Kim and Doug L James. 2012. Physics-based character skinning using multidomain subspace deformations. IEEE transactions on visualization and computer graphics 18, 8 (2012), 1228--1240.
31. Dana A Knoll and David E Keyes. 2004. Jacobian-free Newton-Krylov methods: a survey of approaches and applications. J. Comput. Phys. 193, 2 (2004), 357--397.
32. L'ubor Ladickỳ, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1--9.
33. Lei Lan, Ran Luo, Marco Fratarcangeli, Weiwei Xu, Huamin Wang, Xiaohu Guo, Junfeng Yao, and Yin Yang. 2020. Medial Elastics: Efficient and Collision-Ready Deformation via Medial Axis Transform. ACM Transactions on Graphics (TOG) 39, 3 (2020), 1--17.
34. Gregory Lantoine, Ryan P Russell, and Thierry Dargent. 2012. Using multicomplex variables for automatic computation of high-order derivatives. ACM Transactions on Mathematical Software (TOMS) 38, 3 (2012), 1--21.
35. Tiantian Liu, Adam W. Bargteil, James F. O'Brien, and Ladislav Kavan. 2013. Fast Simulation of Mass-spring Systems. ACM Trans. Graph. (TOG) 32, 6 (2013), 214:1--214:7.
36. Tiantian Liu, Sofien Bouaziz, and Ladislav Kavan. 2017. Quasi-newton methods for real-time simulation of hyperelastic materials. ACM Transactions on Graphics (TOG) 36, 3 (2017), 1--16.
37. RanLuo, Weiwei Xu, Tianjia Shao, Hongyi Xu, and Yin Yang. 2019. Accelerated complex-step finite difference for expedient deformable simulation. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--16.
38. Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2015. Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015).
39. Sebastian Martin, Peter Kaufmann, Mario Botsch, Eitan Grinspun, and Markus Gross. 2010. Unified simulation of elastic rods, shells, and solids. In ACM Trans. Graph. (TOG), Vol. 29. ACM, 39.
40. Joaquim RRA Martins, Peter Sturdza, and Juan J Alonso. 2003. The complex-step derivative approximation. ACM Transactions on Mathematical Software (TOMS) 29, 3 (2003), 245--262.
41. Rajaditya Mukherjee, Xiaofeng Wu, and Huamin Wang. 2016. Incremental deformation subspace reconstruction. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 169--178.
42. Matthias Müller, Bruno Heidelberger, Matthias Teschner, and Markus Gross. 2005. Meshless deformations based on shape matching. In ACM Trans. Graph. (TOG), Vol. 24. ACM, 471--478.
43. Przemyslaw Musialski, Christian Hafner, Florian Rist, Michael Birsak, Michael Wimmer, and Leif Kobbelt. 2016. Non-linear shape optimization using local subspace projections. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1--13.
44. Vinod Nair and Geoffrey E Hinton. 2009. 3D object recognition with deep belief nets. Advances in neural information processing systems 22 (2009), 1339--1347.
45. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML.
46. HM Nasir. 2013. A new class of multicomplex algebra with applications. Mathematical Sciences International Research Journals 2, 2 (2013), 163--168.
47. CUDA Nvidia. 2008. Cublas library. NVIDIA Corporation, Santa Clara, California 15, 27 (2008), 31.
48. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019).
49. Alex Pentland and John Williams. 1989. Good vibrations: Modal dynamics for graphics and animation. In Proceedings of the 16th annual conference on Computer graphics and interactive techniques. 215--222.
50. Eric Pesheck, Nicolas Boivin, Christophe Pierre, and Steven W Shaw. 2001. Nonlinear modal analysis of structural systems using multi-mode invariant manifolds. Nonlinear Dynamics 25, 1--3 (2001), 183--205.
51. Salah Rifai, Grégoire Mesnil, Pascal Vincent, Xavier Muller, Yoshua Bengio, Yann Dauphin, and Xavier Glorot. 2011a. Higher order contractive auto-encoder. In Joint European conference on machine learning and knowledge discovery in databases. Springer, 645--660.
52. Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. 2011b. Contractive auto-encoders: Explicit invariance during feature extraction. In Icml.
53. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision 115, 3 (2015), 211--252.
54. Sangriyadi Setio, Herlien D Setio, and Louis Jezequel. 1992. Modal analysis of nonlinear multi-degree-of-freedom structures. IJAEM 7, 2 (1992), 75--93.
55. Ahmed A Shabana. 2003. Dynamics of multibody systems. Cambridge university press.
56. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 conference on empirical methods in natural language processing. 151--161.
57. Rasmus Tamstorf, Toby Jones, and Stephen F McCormick. 2015. Smoothed aggregation multigrid for cloth simulation. ACM Trans. Graph. (TOG) 34, 6 (2015), 245.
58. Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng. 2020. Fourier features let networks learn high frequency functions in low dimensional domains. arXiv preprint arXiv:2006.10739 (2020).
59. Adrien Treuille, Andrew Lewis, and Zoran Popović. 2006. Model reduction for real-time fluids. ACM Transactions on Graphics (TOG) 25, 3 (2006), 826--834.
60. Christoph Von-Tycowicz, Christian Schulz, Hans-Peter Seidel, and Klaus Hildebrandt. 2015. Real-time nonlinear shape interpolation. ACM Transactions on Graphics (TOG) 34, 3 (2015), 1--10.
61. Huamin Wang, James F O'Brien, and Ravi Ramamoorthi. 2011. Data-driven elastic models for cloth: modeling and measurement. ACM transactions on graphics (TOG) 30, 4 (2011), 1--12.
62. Huamin Wang and Yin Yang. 2016. Descent methods for elastic body simulation on the GPU. ACM Trans. Graph. (TOG) 35, 6 (2016), 212.
63. Yu Wang, Alec Jacobson, Jernej Barbič, and Ladislav Kavan. 2015. Linear subspace design for real-time shape deformation. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1--11.
64. Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2019. Latent space physics: Towards learning the temporal evolution of fluid flow. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 71--82.
65. Stephen Wolfram et al. 1999. The MATHEMATICA® book, version 4. Cambridge university press.
66. Xiaofeng Wu, Rajaditya Mukherjee, and Huamin Wang. 2015. A unified approach for subspace simulation of deformable bodies in multiple domains. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1--9.
67. Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (2020).
68. Hongyi Xu and Jernej Barbič. 2016. Pose-space subspace dynamics. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1--14.
69. Hongyi Xu, Yijing Li, Yong Chen, and Jernej Barbič. 2015. Interactive material design using model reduction. ACM Transactions on Graphics (TOG) 34, 2 (2015), 1--14.
70. Yin Yang, Dingzeyu Li, Weiwei Xu, Yuan Tian, and Changxi Zheng. 2015. Expediting precomputation for reduced deformable simulation. ACM Trans. Graph. (TOG) 34, 6 (2015).
71. Yin Yang, Weiwei Xu, Xiaohu Guo, Kun Zhou, and Baining Guo. 2013. Boundary-aware multidomain subspace deformation. IEEE transactions on visualization and computer graphics 19, 10 (2013), 1633--1645.
72. Nianyin Zeng, Hong Zhang, Baoye Song, Weibo Liu, Yurong Li, and Abdullah M Dobaie. 2018. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273 (2018), 643--649.
73. Jiayi Eris Zhang, Seungbae Bang, David I.W. Levin, and Alec Jacobson. 2020. Complementary Dynamics. ACM Transactions on Graphics (2020).
74. Yongning Zhu, Eftychios Sifakis, Joseph Teran, and Achi Brandt. 2010. An efficient multigrid method for the simulation of high-resolution elastic solids. ACM Trans. Graph. (TOG) 29, 2 (2010), 16.
75. Olgierd Cecil Zienkiewicz, Robert Leroy Taylor, Olgierd Cecil Zienkiewicz, and Robert Lee Taylor. 1977. The finite element method. Vol. 36. McGraw-hill London.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

      • Physical simulation

  • Machine learning

    • Learning paradigms

      • Unsupervised learning

        • Dimensionality reduction and manifold learning