Abstract
This paper presents an approach to assist the generation of agent-based cooperative animation using reinforcement learning. We focus on manipulation skills for box-shaped objects, including pushing, pulling, and moving objects in a relay way. There are a learning process and an animation process. In the learning process, different kinds of agents are trained using reinforcement learning. Policies are learned to control the agents to perform specific tasks. A physics simulator is adopted to simulate the interaction among objects. In the animation process, users animate agents with the learned policies. We propose several tools to intuitively create cooperative animations. We applied our method to generate several animations in which agents work together to finish tasks. A user study indicates that by using our tools, diverse cooperative animations can be easily created.
- Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866 (2017).Google Scholar
- Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35, 8 (2013), 1798--1828. Google Scholar
Digital Library
- Hsuan Chen and Sai-Keung Wong. 2018. Transporting objects by multiagent cooperation in crowd simulation. Computer Animation and Virtual Worlds 29, 3-4 (2018), e1826.Google Scholar
Cross Ref
- Alexander Clegg, Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. 2018. Learning to dress: Synthesizing human dressing motion via deep reinforcement learning. ACM Transactions on Graphics 37, 6 (2018). Google Scholar
Digital Library
- Hongliang Guo and Yan Meng. 2010. Distributed reinforcement learning for coordinate multi-robot foraging. Journal of intelligent & robotic systems 60, 3-4 (2010), 531--551. Google Scholar
Digital Library
- Shihui Guo, Meili Wang, Gabriel Notman, Jian Chang, Jianjun Zhang, and Minghong Liao. 2017. Simulating collective transport of virtual ants. Computer Animation and Virtual Worlds 28, 3-4 (2017), e1779.Google Scholar
Cross Ref
- Nicolas Heess, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang, Ali Eslami, Martin Riedmiller, et al. 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017).Google Scholar
- Arthur Juliani, Vincent-Pierre Berges, Esh Vckay, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. 2018. Unity: A General Platform for Intelligent Agents. arXiv preprint arXiv:1809.02627 (2018).Google Scholar
- Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Jaśkowski. 2016. Vizdoom: A doom-based ai research platform for visual reinforcement learning. In Computational Intelligence and Games, 2016 IEEE Conference on. 1--8.Google Scholar
Cross Ref
- Guan-Wen Lin and Sai-Keung Wong. 2018. Evacuation simulation with consideration of obstacle removal and using game theory. Phys. Rev. E 97 (Jun 2018), 062303. Issue 6.Google Scholar
- Pingchuan Ma, Yunsheng Tian, Zherong Pan, Bo Ren, and Dinesh Manocha. 2018. Fluid directed rigid body control using deep reinforcement learning. ACM Transactions on Graphics 37, 4 (2018), 96. Google Scholar
Digital Library
- Tekin Meriçli, Manuela Veloso, and H Levent Akın. 2015. Push-manipulation of complex passive mobile objects using experimentally acquired motion models. Autonomous Robots 38, 3 (2015), 317--329. Google Scholar
Digital Library
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.Google Scholar
- Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee. 2016. Control of memory, active perception, and action in minecraft. arXiv preprint arXiv:1605.09128 (2016). Google Scholar
Digital Library
- Liviu Panait and Sean Luke. 2005. Cooperative multi-agent learning: The state of the art. Autonomous agents and multi-agent systems 11, 3 (2005), 387--434. Google Scholar
Digital Library
- Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. arXiv preprint arXiv:1804.02717 (2018).Google Scholar
- Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel Van De Panne. 2017. Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Transactions on Graphics) 36, 4 (2017), 41. Google Scholar
Digital Library
- Martin Riedmiller. 2005. Neural fitted Q iteration--first experiences with a data efficient neural reinforcement learning method. In European Conference on Machine Learning. Springer, 317--328. Google Scholar
Digital Library
- Samuel Rodriguez, Marco Morales, and Nancy M Amato. 2016. Multi-agent push behaviors for large sets of passive objects. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 4437--4442.Google Scholar
Cross Ref
- Eric Rohmer, Surya PN Singh, and Marc Freese. 2013. V-REP: A versatile and scalable robot simulation framework. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on. IEEE, 1321--1326.Google Scholar
Cross Ref
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
- Peter Stone, Richard S Sutton, and Gregory Kuhlmann. 2005. Reinforcement learning for robocup soccer keepaway. Adaptive Behavior 13, 3 (2005), 165--188.Google Scholar
Cross Ref
- Elio Tuci, Muhanad Hayder Mohammed Alkilabi, and Otar Akanyeti. 2018. Cooperative object transport in multi-robot systems: A review of the state-of-the-art. Frontiers in Robotics and AI 5 (2018), Article:59.Google Scholar
- Kai Wang, Manolis Savva, Angel X Chang, and Daniel Ritchie. 2018. Deep convolutional priors for indoor scene synthesis. ACM Transactions on Graphics (TOG) 37, 4 (2018), 70. Google Scholar
Digital Library
- Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to train your dragon: example-guided control of flapping flight. ACM Transactions on Graphics 36, 6 (2017), 198. Google Scholar
Digital Library
- Sai-Keung Wong, Yi-Hung Chou, and Hsiang-Yu Yang. 2018. A framework for simulating agent-based cooperative tasks in crowd simulation. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 11. Google Scholar
Digital Library
- Sai-Keung Wong, Yu-Shuen Wang, Pao-Kun Tang, and Tsung-Yu Tsai. 2017. Optimized evacuation route based on crowd simulation. Computational Visual Media 3, 3 (2017), 243--261.Google Scholar
Cross Ref
- Wei Xiang, Jiaping Ren, Kuan Wang, Zhigang Deng, and Xiaogang Jin. 2018. Biologically inspired ant colony simulation. Computer Animation and Virtual Worlds (2018), e1867.Google Scholar
- Jun Xing, Rubaiat Habib Kazi, Tovi Grossman, Li-Yi Wei, Jos Stam, and George Fitzmaurice. 2016. Energy-brushes: Interactive tools for illustrating stylized elemental dynamics. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 755--766. Google Scholar
Digital Library
- Fangyi Zhang, Jürgen Leitner, Michael Milford, Ben Upcroft, and Peter Corke. 2015. Towards vision-based deep reinforcement learning for robotic motion control. arXiv preprint arXiv: 1511.03791 (2015).Google Scholar
- Xinyi Zhang and Michiel van de Panne. 2018. Data-driven autocompletion for keyframe animation. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games. ACM, 10. Google Scholar
Digital Library
Index Terms
Agent-based cooperative animation for box-manipulation using reinforcement learning
Recommendations
Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulation
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent SystemsWe consider solving a cooperative multi-robot object manipulation task using reinforcement learning (RL). We propose two distributed multi-agent RL approaches: distributed approximate RL (DA-RL), where each agent applies Q-learning with individual ...
Cooperative reinforcement learning in topology-based multi-agent systems
Topology-based multi-agent systems (TMAS), wherein agents interact with one another according to their spatial relationship in a network, are well suited for problems with topological constraints. In a TMAS system, however, each agent may have a ...
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent SystemsRecent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...






Comments