Synthesis of biologically realistic human motion using joint torque actuation

Abstract

Using joint actuators to drive the skeletal movements is a common practice in character animation, but the resultant torque patterns are often unnatural or infeasible for real humans to achieve. On the other hand, physiologically-based models explicitly simulate muscles and tendons and thus produce more human-like movements and torque patterns. This paper introduces a technique to transform an optimal control problem formulated in the muscle-actuation space to an equivalent problem in the joint-actuation space, such that the solutions to both problems have the same optimal value. By solving the equivalent problem in the joint-actuation space, we can generate human-like motions comparable to those generated by musculotendon models, while retaining the benefit of simple modeling and fast computation offered by joint-actuation models. Our method transforms constant bounds on muscle activations to nonlinear, state-dependent torque limits in the joint-actuation space. In addition, the metabolic energy function on muscle activations is transformed to a nonlinear function of joint torques, joint configuration and joint velocity. Our technique can also benefit policy optimization using deep reinforcement learning approach, by providing a more anatomically realistic action space for the agent to explore during the learning process. We take the advantage of the physiologically-based simulator, OpenSim, to provide training data for learning the torque limits and the metabolic energy function. Once trained, the same torque limits and the energy function can be applied to drastically different motor tasks formulated as either trajectory optimization or policy learning.

References

1. Marko Ackermann and Antonie J. Van den Bogert. 2010. Optimality principles for model-based prediction of human gait. Journal of biomechanics 43, 6 (2010), 1055--1060.
2. Andrew A. Amis, Duncan Dowson, and Verna Wright. 1980. Elbow joint force predictions for some strenuous isometric actions. Journal of Biomechanics 13, 9 (1980), 765--775.
3. Dennis E. Anderson, Michael L. Madigan, and Maury A. Nussbaum. 2007. Maximum voluntary joint torque as a function of joint angle and angular velocity: model development and application to the lower limb. Journal of biomechanics 40, 14 (2007), 3105--3113.
4. Frank C. Anderson and Marcus G. Pandy. 1999. A dynamic optimization solution for vertical jumping in three dimensions. Computer methods in biomechanics and biomedical engineering 2, 3 (1999), 201--231.
5. Frank C. Anderson and Marcus G. Pandy. 2001a. Dynamic optimization of human walking. Journal of biomechanical engineering 123, 5 (2001), 381--390.
6. Frank C. Anderson and Marcus G. Pandy. 2001b. Static and dynamic optimization solutions for gait are practically equivalent. Journal of biomechanics 34, 2 (2001), 153--161.
7. Lindsay J. Bhargava, Marcus G. Pandy, and Frank C. Anderson. 2004. A phenomenological model for estimating metabolic energy consumption in muscle contraction. Journal of biomechanics 37, 1 (2004), 81--88.
8. Alexander Clegg, Wenhao Yu, Jie Tan, C. Karen Liu, and Greg Turk. 2018. Learning to Dress: Synthesizing Human Dressing Motion via Deep Reinforcement Learning. ACM Trans. Graph. 37, 6, Article 179 (2018), 10 pages.
9. Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and accurate deep network learning by exponential linear units (ELUs). ICLR 2016 (2015).
10. Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2009. Robust Task-based Control Policies for Physics-based Characters. ACM Trans. Graph. 28, 5, Article 170 (Dec. 2009), 9 pages.
11. Friedl De Groote, Allison L. Kinney, Anil V. Rao, and Benjamin J. Fregly. 2016. Evaluation of direct collocation optimal control problem formulations for solving the muscle redundancy problem. Annals of biomedical engineering 44, 10 (2016), 2922--2936.
12. Scott L. Delp, J. Peter Loan, Melissa G. Hoy, Felix E. Zajac, Eric L. Topp, and Joseph M. Rosen. 1990. An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Transactions on Biomedical engineering 37, 8 (1990), 757--767.
13. Anthony C. Fang and Nancy S. Pollard. 2003. Efficient Synthesis of Physically Valid Human Motion. ACM Trans. Graph. 22, 3 (July 2003), 417--426.
14. Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible Muscle-based Locomotion for Bipedal Creatures. ACM Trans. Graph. 32, 6, Article 206 (Nov. 2013), 11 pages.
15. Thomas Geijtenbeek, Antonie J. van den Bogert, Ben J.H. van Basten, and Arjan Egges. 2010. Evaluatingthe Physical Realism of Character Animations Using Musculoskeletal Models. In Third International Conference in Motion in Games, 2010. (Lecture Notes in Computer Science).
16. Karin G.M. Gerritsen, Antonie J. van den Bogert, Manuel Hulliger, and Ronald F. Zernicke. 1998. Intrinsic muscle properties facilitate locomotor control - a computer simulation study. Motor Control 2, 3 (Aug. 1998), 206--220.
17. Michael Gleicher. 1998. Retargetting Motion to New Characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98). ACM, New York, NY, USA, 33--42.
18. Sampath K. Gollapudi and David C. Lin. 2009. Experimental determination of sarcomere force-length relationship in type-I human skeletal muscle fibers. Journal of biomechanics 42, 13 (2009), 2011--2016.
19. Jehee J. Lee and Kang Hoon Lee. 2004. Precomputing Avatar Behavior from Human Motion Data. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '04). 79--87.
20. GC Joyce, PMH Rack, and DR Westbury. 1969. The mechanical properties of cat soleus muscle during controlled lengthening and shortening movements. The Journal of physiology 204, 2 (1969), 461--474.
21. Michael Kass and John Anderson. 2008. Animating Oscillatory Motion with Overlap: Wiggly Splines. ACM Trans. Graph. 27, 3, Article 28 (Aug. 2008), 8 pages.
22. Lukasz Kidzinski, Sharada P. Mohanty, Carmichael F. Ong, Jennifer L. Hicks, Sean F. Carroll, Sergey Levine, Marcel Salathé, and Scott L. Delp. 2018a. Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning. (2018). http://arxiv.org/abs/1804.00198
23. Lukasz Kidzinski, Sharada Prasanna Mohanty, Carmichael F. Ong, Zhewei Huang, Shuchang Zhou, Anton Pechenko, Adam Stelmaszczyk, Piotr Jarosik, Mikhail Pavlov, Sergey Kolesnikov, Sergey M. Plis, Zhibo Chen, Zhizheng Zhang, Jiale Chen, Jun Shi, Zhuobin Zheng, Chun Yuan, Zhihui Lin, Henryk Michalewski, Piotr Milos, Blazej Osinski, Andrew Melnik, Malte Schilling, Helge J. Ritter, Sean F. Carroll, Jennifer L. Hicks, Sergey Levine, Marcel Salathé, and Scott L. Delp. 2018b. Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. (2018). http://arxiv.org/abs/1804.00361
24. Anne D. Koelewijn, Eva Dorschky, and Antonie J. Van den Bogert. 2018. A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait. Computer methods in biomechanics and biomedical engineering 21, 8 (2018), 521--531.
25. Taku Komura, Yoshihisa Shinagawa, and L. Tosiyasu Kunii. 2000. Creating and retargetting motion by the musculoskeletal human body model. Visual Computer 16 (2000), 254--270. Issue 5.
26. Yi-Chung Lin, Tim W. Dorn, Anthony G. Schache, and Marcus G. Pandy. 2012. Comparison of different methods for estimating muscle forces in human movement. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine 226, 2 (2012), 103--112.
27. C. Karen Liu, Aaron Hertzmann, and Zoran Popović. 2005. Learning Physics-based Motion Style with Nonlinear Inverse Optimization. ACM Trans. Graph. 24, 3 (July 2005), 1071--1081.
28. C. Karen Liu and Zoran Popović. 2002. Synthesis of Complex Dynamic Character Motion from Simple Animations. ACM Trans. Graph. 21, 3 (July 2002), 408--416.
29. Wan-Yen Lo and Matthias Zwicker. 2008. Real-time Planning for Parameterized Human Motion. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '08). 29--38.
30. MathWorks 2017. MATLAB 2017b. (2017). The MathWorks, Natick, MA, USA.
31. James McCann and Nancy Pollard. 2007. Responsive Characters from Motion Fragments. ACM Trans. Graph. 26, 3 (July 2007).
32. Matthew Millard, Thomas Uchida, Ajay Seth, and Scott L. Delp. 2013. Flexing computational muscle: modeling and simulation of musculotendon dynamics. Journal of biomechanical engineering 135, 2 (2013).
33. Ross H. Miller. 2014. A comparison of muscle energy models for simulating human walking in three dimensions. Journal of biomechanics 47, 6 (2014), 1373--1381.
34. Jianyuan Min, Huajun Liu, and Jinxiang Chai. 2010. Synthesis and Editing of Personalized Stylistic Human Motion. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D '10). ACM, New York, NY, USA, 39--46.
35. Igor Mordatch, Emanuel Todorov, and Zoran Popović. 2012. Discovery of Complex Behaviors Through Contact-invariant Optimization. ACM Trans. Graph. 31, 4, Article 43 (July 2012), 8 pages.
36. Igor Mordatch, Jack M. Wang, Emanuel Todorov, and Vladlen Koltun. 2013. Animating Human Lower Limbs Using Contact-invariant Optimization. ACM Trans. Graph. 32, 6, Article 203 (Nov. 2013), 8 pages.
37. Masaki Nakada, Tao Zhou, Honglin Chen, Tomer Weiss, and Demetri Terzopoulos. 2018. Deep Learning of Biomimetic Sensorimotor Control for Biomechanical Human Animation. ACM Trans. Graph. 37, 4, Article 56 (July 2018), 15 pages.
38. Evert Jan Nijhofa and David A. Gabriel. 2006. Maximum isometric arm forces in the horizontal plane. Journal of Biomechanics 39, 4 (2006), 708--716.
39. Marcus G. Pandy, Felix E. Zajac, Eunsup Sim, and William S. Levine. 1990. An optimal control model for maximum-height human jumping. Journal of biomechanics 23, 12 (1990), 1185--1198.
40. Antonio Pedotti, V.V. Krishnan, and L. Stark. 1978. Optimization of muscle-force sequencing in human locomotion. Mathematical Biosciences 38, 1--2 (1978), 57--76.
41. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michielvan de Panne. 2018. DeepMimic: Example-guided Deep Reinforcement Learning of Physics-based Character Skills. ACM Trans. Graph. 37, 4, Article 143 (July 2018), 14 pages.
42. Xue Bin Peng, Glen Berseth, Kangkang Yin, and Michiel Van De Panne. 2017. DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning. ACM Trans. Graph. 36, 4, Article 41 (July 2017), 13 pages.
43. Xue Bin Peng and Michiel van de Panne. 2017. Learning locomotion skills using DeepRL: Does the choice of action space matter?. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. ACM, 12.
44. Zoran Popović and Andrew Witkin. 1999. Physically Based Motion Transformation. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99). 11--20.
45. Anil V. Rao. 2009. A survey of numerical methods for optimal control. Advances in the Astronautical Sciences 135, 1 (2009), 497--528.
46. Charles Rose, Brian Guenter, Bobby Bodenheimer, and Michael F. Cohen. 1996. Efficient Generation of Motion Transitions Using Spacetime Constraints. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). 147--154.
47. Seunghwan S. Lee, Ri Yu, Jungnam Park, Mridul Aanjaneya, Eftychios Sifakis, and Jehee Lee. 2018. Dexterous Manipulation and Control with Volumetric Muscles. ACM Trans. Graph. 37, 4, Article 57 (July 2018), 13 pages.
48. Alla Safonova, Jessica K. Hodgins, and Nancy S. Pollard. 2004. Synthesizing Physically Realistic Human Motion in Low-dimensional, Behavior-specific Spaces. ACM Trans. Graph. 23, 3 (Aug. 2004), 514--521.
49. Ajay Seth, Michael Sherman, Jeffrey A. Reinbolt, and Scott L. Delp. 2011. OpenSim: a musculoskeletal modeling and simulation framework for in silico investigations and exchange. Procedia Iutam 2 (2011), 212--232.
50. Sung-Hee S.H. Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive Biomechanical Modeling and Simulation of the Upper Body. ACM Trans. Graph. 28, 4, Article 99 (Sept. 2009), 17 pages.
51. Sung-Hee S.H. Lee and Demetri Terzopoulos. 2006. Heads Up!: Biomechanical Modeling and Neuromuscular Control of the Neck. ACM Trans. Graph. 25, 3 (July 2006), 1188--1198.
52. Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic Biomechanical Simulation and Control of Human Swimming. ACM Trans. Graph. 34, 1, Article 10 (Dec. 2014), 15 pages.
53. Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data. ACM Trans. Graph. 24, 3 (July 2005), 417--425.
54. Shinjiro Sueda, Andrew Kaufman, and Dinesh K. Pai. 2008. Musculotendon Simulation for Hand Animation. ACM Trans. Graph. 27, 3, Article 83 (Aug. 2008), 8 pages.
55. Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal Character Animation with Continuous Control. ACM Trans. Graph. 26, 3 (July 2007).
56. Winnie Tsang, Karan Singh, and Eugene Fiume. 2005. Helping Hand: An Anatomically Accurate Inverse Dynamics Solution for Unconstrained Hand Motion. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '05). ACM, New York, NY, USA, 319--328.
57. Thomas K. Uchida, Jennifer L. Hicks, Christopher L. Dembia, and Scott L. Delp. 2016. Stretching your energetic budget: how tendon compliance affects the metabolic cost of running. PloS one 11, 3 (2016).
58. Brian R. Umberger. 2010. Stance and swing phase costs in human walking. Journal of the Royal Society Interface 7, 50 (2010), 1329--1340.
59. Brian R. Umberger, Karin GM Gerritsen, and Philip E. Martin. 2003. A model of human muscle energy expenditure. Computer methods in biomechanics and biomedical engineering 6, 2 (2003), 99--111.
60. Marjolein M. van der Krogt, Wendy W. de Graaf, Claire T. Farley, Chet T. Moritz, L. J. Richard Casius, and Maarten F. Bobbert. 2009. Robust passive dynamics of the musculoskeletal system compensate for unexpected surface changes during human hopping. Journal of Applied Physiology 107, 3, Article 112 (Sept. 2009), 7 pages.
61. Andreas Wächter and Lorenz T. Biegler. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006), 25--57.
62. Kevin Wampler, Zoran Popović, and Jovan Popović. 2014. Generalizing Locomotion Style to New Animals with Inverse Optimal Regression. ACM Trans. Graph. 33, 4, Article 49 (July 2014), 11 pages.
63. Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing Locomotion Controllers Using Biologically-based Actuators and Objectives. ACM Trans. Graph. 31, 4, Article 25 (July 2012), 11 pages.
64. Andrew Witkin and Michael Kass. 1988. Spacetime Constraints. In Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '88). ACM, New York, NY, USA, 159--168.
65. Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics Control of Flying Creatures via Self-regulated Learning. In SIGGRAPH Asia 2018 Technical Papers (SIGGRAPH Asia '18). ACM, New York, NY, USA, Article 181, 10 pages.
66. Yuencheng Y.C. Lee, Demetri Terzopoulos, and Keith Waters. 1995. Realistic Modeling for Facial Animation. In Proceedings of the 22Nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95). ACM, New York, NY, USA, 55--62.
67. Yuting Ye and C. Karen Liu. 2008. Animating Responsive Characters with Dynamic Constraints in Near-unactuated Coordinates. ACM Trans. Graph. 27, 5, Article 112 (Dec. 2008), 5 pages.
68. Yongjoon Y.J. Lee, Seong Jae Lee, and Zoran Popović. 2009. Compact Character Controllers. ACM Trans. Graph. 28, 5, Article 169 (Dec. 2009), 8 pages.
69. Yoonsang Y.S. Lee, Moon Seok Park, Taesoo Kwon, and Jehee Lee. 2014. Locomotion Control for Many-muscle Humanoids. ACM Trans. Graph. 33, 6, Article 218 (Nov. 2014), 11 pages.
70. Wenhao Yu, Greg Turk, and C. Karen Liu. 2018. Learning Symmetric and Low-energy Locomotion. ACM Trans. Graph. 37, 4, Article 144 (July 2018), 12 pages.
71. Felix E. Zajac. 1989. Muscle and tendon Properties models scaling and application to biomechanics and motor. Critical reviews in biomedical engineering 17, 4 (1989), 359--411.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

      • Physical simulation

  • Machine learning

    • Learning paradigms

      • Reinforcement learning

      • Supervised learning