Learning time-critical responses for interactive character control

Abstract

Creating agile and responsive characters from a collection of unorganized human motion has been an important problem of constructing interactive virtual environments. Recently, learning-based approaches have successfully been exploited to learn deep network policies for the control of interactive characters. The agility and responsiveness of deep network policies are influenced by many factors, such as the composition of training datasets, the architecture of network models, and learning algorithms that involve many threshold values, weights, and hyper-parameters. In this paper, we present a novel teacher-student framework to learn time-critically responsive policies, which guarantee the time-to-completion between user inputs and their associated responses regardless of the size and composition of the motion databases. We demonstrate the effectiveness of our approach with interactive characters that can respond to the user's control quickly while performing agile, highly dynamic movements.

References

1. Okan Arikan, David A. Forsyth, and James F. O'Brien. 2005. Pushing People Around. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2005. 56--66.
2. Kevin Bergamin, Simon Claver, Daniel Holden, and James Richard Forbes. 2019. DReCon: Data-Driven Responsive Control of Physics-Based Characters. ACM Transactions on Graphics 38, 6, Article 206 (2019).
3. Myung Geol Choi, Manmyung Kim, Kyunglyul Hyun, and Jehee Lee. 2011. Deformable Motion: Squeezing into Cluttered Environments. Computer Graphics Forum 30, 2 (2011).
4. Simon Clavet. 2016. Motion Matching and The Road to Next-Gen Animation. In GDC 2016.
5. Alexaander Clegg, Wenhao Yu, Jie Tan, Karen C. Liu, and Greg Turk. 2018. Learning To Dress: Synthesizing Human Dressing Motion via Deep Reinforcement Learning. ACM Transactions on Graphics 37, 6 (2018).
6. Epic Games. 2019. Unreal Engine. https://www.unrealengine.com
7. Katerina Fragkiadaki, Sergey Levine, Panna Felsen, and Jitendra Malik. 2015. Recurrent network models for human dynamics. In Proceedings of the IEEE International Conference on Computer Vision. 4346--4354.
8. Felix Harvey, Mike Yurick, Christopher Pal, and Derek Nowrouzezahrai. 2020. Robust Motion In-Betweening. ACM Transactions on Graphics 39, 4 (2020).
9. Rachel Heck and Michael Gleicher. 2007. Parametric motion graphs. In Proceedings of the 2007 symposium on Interactive 3D graphics and games. 129--136.
10. Gustav Eje Henter, Simon Alexanderson, and Jonas Beskow. 2020. MoGlow: Probabilistic and Controllable Motion Synthesis Using Normalising Flows. ACM Trans. Graph. 39, 6, Article 236 (2020).
11. Daniel Holden, Oussama Kanoun, Maksym Perepichika, and Tiberiu Popa. 2020. Learned Motion Matching. ACM Transactions on Graphics 39, 4 (2020).
12. Daniel Holden, Taku Komura, and Jun Saito. 2017. Phase-functioned neural networks for character control. ACM Transactions on Graphics 36, 4, Article 42 (2017).
13. Daniel Holden, Jun Saito, and Taku Komura. 2016. A Deep Learning Framework for Character Motion Synthesis and Editing. ACM Transactions on Graphics 35, 4, Article 138 (2016).
14. Kyunglyul Hyun, Kyungho Lee, and Jehee Lee. 2016. Motion grammars for character animation. Computer Graphics Forum 35, 2 (2016), 103--113.
15. Leslie Ikemoto, Okan Arikan, and David Forsyth. 2007. Quick transitions with cached multi-way blends. In Proceedings of the 2007 symposium on Interactive 3D graphics and games. 145--151.
16. Yifeng Jiang, Tom Wouwe, Friedl De Groote, and Karen C. Liu. 2019. Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation. ACM Transactions on Graphics 38, 4 (2019).
17. Sophie Jörg, Aline Normoyle, and Alla Safonova. 2012. How responsiveness affects players' perception in digital games. In Proceedings of the ACM Symposium on Applied Perception. 33--38.
18. Mubbasir Kapadia, Xu Xianghao, Maurizio Nitti, Marcelo Kallmann, Stelian Coros, Robert Sumner, and Markus Gross. 2016. Precision: precomputing environment semantics for contact-rich character animation. In Proceedings of the 2016 symposium on Interactive 3D graphics and games. 29--37.
19. Manmyung Kim, Kyunglyul Hyun, Jongmin Kim, and Jehee Lee. 2009. Synchronized Multi-character Motion Editing. ACM Transactions on Graphics 28, 3, Article 79 (2009).
20. Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion Graphs. ACM Transactions on Graphics 21, 3 (2002), 473--482.
21. Jehee Lee, Jinxiang Chai, Paul S. A. Reitsma, Jessica K. Hodgins, and Nancy S. Pollard. 2002. Interactive Control of Avatars Animated with Human Motion Data. ACM Transactions on Graphics 21, 3 (2002), 491--500.
22. Jehee Lee and Kang Hoon Lee. 2004. Precomputing Avatar Behavior from Human Motion Data. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 79--87.
23. Kyungho Lee, Seyoung Lee, and Jehee Lee. 2018. Interactive Character Animation by Learning Multi-Objective Control. ACM Transactions on Graphics 37, 6, Article 180 (2018).
24. Kang Hoon Lee, Myung Geol Choi, and Jehee Lee. 2006. Motion Patches: Building blocks for virtual environments annotated with motion data. ACM Transactions on Graphics 25, 3 (2006), 898--906.
25. Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable Muscle-actuated Human Simulation and Control. ACM Transactions on Graphics 38, 4, Article 73 (2019).
26. Yongjoon Lee, Kevin Wampler, Gilbert Bernstein, Jovan Popović, and Zoran Popović. 2010. Motion Fields for Interactive Character Locomotion. ACM Transactions on Graphics 29, 6, Article 138 (2010).
27. Sergey Levine, Yongjoon Lee, Vladren Koltun, and Zoran Popović. 2011. Space-time planning with parameterized locomotion controllers. ACM Transactions on Graphics 30, 3 (2011).
28. Sergey Levine, Jack M. Wang, Alexis Haraux, Zoran Popović, and Vladlen Koltun. 2012. Continuous Character Control with Low-Dimensional Embeddings. ACM Transactions on Graphics 31, 4 (2012).
29. Hung Yu Ling, Fabio Zinno, George Cheng, and Michiel van de Panne. 2020. Character Controllers Using Motion VAEs. ACM Transactions on Graphics 39, 4 (2020).
30. Libin Liu and Jessica Hodgins. 2017. Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning. ACM Transactions on Graphics 36, 3 (2017).
31. Libin Liu and Jessica Hodgins. 2018. Learning Basketball Dribbling Skills Using Trajectory Optimization and Deep Reinforcement Learning. ACM Transactions on Graphics 37, 4 (2018).
32. Wan-Yen Lo and Matthias Zwicker. 2008. Real-time planning for parameterized human motion. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2008. 29--38.
33. Ying-Sheng Luo, Jonathan Hans Soeseno, Trista Pei-Chun Chen, and Wei-Chao Chen. 2020. CARL: Controllable Agent With Reinforcement Learning for Quadruped Locomotion. ACM Transactions on Graphics 39, 4 (2020).
34. Julieta Martinez, Michael J Black, and Javier Romero. 2017. On human motion prediction using recurrent neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 4674--4683.
35. James McCann and Nancy Pollard. 2007. Responsive characters from motion fragments. In ACM Transactions on Graphics, Vol. 26. 6.
36. Jianyuan Min and Jinxiang Chai. 2012. Motion graphs++ a compact generative model for semantic motion analysis and synthesis. ACM Transactions on Graphics 31, 6 (2012), 1--12.
37. Sehee Min, Jungdam Won, Seunghwan Lee, Jungnam Park, and Jehee Lee. 2019. SoftCon: Simulation and Control of Soft-Bodied Animals with Biomimetic Actuators. ACM Transactions on Graphics 38, 6, Article 208 (2019).
38. Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and Jehee Lee. 2019. Learning predict-and-simulate policies from unorganized human motion data. ACM Transactions on Graphics 38, 6, Article 205 (2019).
39. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics 37, 4, Article 143 (2018).
40. Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Lee, Jie Tan, and Sergey Levine. 2020. Learning Agile Robotic Locomotion Skills by Imitating Animals. arXiv preprint (2020).
41. Paul SA Reitsma and Nancy S Pollard. 2007. Evaluating motion graphs for character animation. ACM Transactions on Graphics 26, 4 (2007), 18.
42. Cheng Ren, Liming Zhao, and Alla Safonova. 2010. Human motion synthesis with optimization-based graphs. In Computer Graphics Forum, Vol. 29. 545--554.
43. Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. 2016. Policy Distillation. arXiv:1511.06295 [cs.LG]
44. Alla Safonova and Jessica K Hodgins. 2007. Construction and optimal search of interpolated motion graphs. ACM Transactions on Graphics 26, 3, Article 106 (2007).
45. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG]
46. Jeremy M Sheppard and Warren B Young. 2006. Agility literature review: Classifications, training and testing. Journal of sports sciences 24, 9 (2006), 919--932.
47. Hubert Shum, Taku Komura, and Shuntaro Yamazaki. 2012. Simulating Multiple Character Interactions with Collaborative and Adversarial Goals. IEEE Transactions on Visualization and Computer Graphics 18 (2012), 741--52.
48. Hubert PH Shum, Taku Komura, Masashi Shiraishi, and Shuntaro Yamazaki. 2008. Interaction patches for multi-character animation. ACM Transactions on Graphics 27, 5, Article 114 (2008).
49. Bernard W Silverman. 1986. Density estimation for statistics and data analysis. Vol. 26. CRC press.
50. Sebastian Starke, He Zhang, Taku Komura, and Jun Saito. 2019. Neural State Machine for Character-Scene Interactions. ACM Transactions on Graphics 38, 6, Article 178 (2019).
51. Sebastian Starke, Yiwei Zhao, Taku Komura, and Kazi Zaman. 2020. Local Motion Phases for Learning Multi-Contact Character Movements. ACM Transactions on Graphics 39, 4 (2020).
52. TensorFlow. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/
53. Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal character animation with continuous control. ACM Transactions on Graphics 26, 3, Article 7 (2007).
54. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arXiv:1706.03762 [cs.CL]
55. Jungdam Won, Deepa Gopinath, and Jessica Hodgins. 2020. A Scalable Approach to Control Diverse Behaviors for Physically Simulated Characters. ACM Transactions on Graphics 39, 4 (2020).
56. Jungdam Won, Kyungho Lee, Carol O'Sullivan, Jessica K Hodgins, and Jehee Lee. 2014. Generating and ranking diverse multi-character interactions. ACM Transactions on Graphics 33, 6, Article 219 (2014).
57. Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to Train Your Dragon: Example-guided Control of Flapping Flight. ACM Transactions on Graphics 36, 6, Article 198 (2017).
58. Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics control of flying creatures via self-regulated learning. ACM Transactions on Graphics 37, 6, Article 181 (2018).
59. He Zhang, Sebastian Starke, Taku Komura, and Jun Saito. 2018. Mode-adaptive neural networks for quadruped motion control. ACM Transactions on Graphics 37, 4, Article 145 (2018).
60. Liming Zhao and Alla Safonova. 2009. Achieving good connectivity in motion graphs. Graphical Models 71, 4 (2009), 139--152.
61. Victor Zordan, Anna Majkowska, Bill Chiu, and Matthew Fast. 2005. Dynamic response for motion capture animation. ACM Transactions on Graphics 24 (2005), 697--701.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

      • Motion capture

      • Motion processing