skip to main content
research-article

Agent-based cooperative animation for box-manipulation using reinforcement learning

Authors Info & Claims
Published:03 June 2019Publication History
Skip Abstract Section

Abstract

This paper presents an approach to assist the generation of agent-based cooperative animation using reinforcement learning. We focus on manipulation skills for box-shaped objects, including pushing, pulling, and moving objects in a relay way. There are a learning process and an animation process. In the learning process, different kinds of agents are trained using reinforcement learning. Policies are learned to control the agents to perform specific tasks. A physics simulator is adopted to simulate the interaction among objects. In the animation process, users animate agents with the learned policies. We propose several tools to intuitively create cooperative animations. We applied our method to generate several animations in which agents work together to finish tasks. A user study indicates that by using our tools, diverse cooperative animations can be easily created.

References

  1. Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866 (2017).Google ScholarGoogle Scholar
  2. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35, 8 (2013), 1798--1828. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Hsuan Chen and Sai-Keung Wong. 2018. Transporting objects by multiagent cooperation in crowd simulation. Computer Animation and Virtual Worlds 29, 3-4 (2018), e1826.Google ScholarGoogle ScholarCross RefCross Ref
  4. Alexander Clegg, Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. 2018. Learning to dress: Synthesizing human dressing motion via deep reinforcement learning. ACM Transactions on Graphics 37, 6 (2018). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Hongliang Guo and Yan Meng. 2010. Distributed reinforcement learning for coordinate multi-robot foraging. Journal of intelligent & robotic systems 60, 3-4 (2010), 531--551. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Shihui Guo, Meili Wang, Gabriel Notman, Jian Chang, Jianjun Zhang, and Minghong Liao. 2017. Simulating collective transport of virtual ants. Computer Animation and Virtual Worlds 28, 3-4 (2017), e1779.Google ScholarGoogle ScholarCross RefCross Ref
  7. Nicolas Heess, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang, Ali Eslami, Martin Riedmiller, et al. 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017).Google ScholarGoogle Scholar
  8. Arthur Juliani, Vincent-Pierre Berges, Esh Vckay, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. 2018. Unity: A General Platform for Intelligent Agents. arXiv preprint arXiv:1809.02627 (2018).Google ScholarGoogle Scholar
  9. Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Jaśkowski. 2016. Vizdoom: A doom-based ai research platform for visual reinforcement learning. In Computational Intelligence and Games, 2016 IEEE Conference on. 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  10. Guan-Wen Lin and Sai-Keung Wong. 2018. Evacuation simulation with consideration of obstacle removal and using game theory. Phys. Rev. E 97 (Jun 2018), 062303. Issue 6.Google ScholarGoogle Scholar
  11. Pingchuan Ma, Yunsheng Tian, Zherong Pan, Bo Ren, and Dinesh Manocha. 2018. Fluid directed rigid body control using deep reinforcement learning. ACM Transactions on Graphics 37, 4 (2018), 96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Tekin Meriçli, Manuela Veloso, and H Levent Akın. 2015. Push-manipulation of complex passive mobile objects using experimentally acquired motion models. Autonomous Robots 38, 3 (2015), 317--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.Google ScholarGoogle Scholar
  14. Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee. 2016. Control of memory, active perception, and action in minecraft. arXiv preprint arXiv:1605.09128 (2016). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Liviu Panait and Sean Luke. 2005. Cooperative multi-agent learning: The state of the art. Autonomous agents and multi-agent systems 11, 3 (2005), 387--434. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. arXiv preprint arXiv:1804.02717 (2018).Google ScholarGoogle Scholar
  17. Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel Van De Panne. 2017. Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Transactions on Graphics) 36, 4 (2017), 41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Martin Riedmiller. 2005. Neural fitted Q iteration--first experiences with a data efficient neural reinforcement learning method. In European Conference on Machine Learning. Springer, 317--328. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Samuel Rodriguez, Marco Morales, and Nancy M Amato. 2016. Multi-agent push behaviors for large sets of passive objects. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 4437--4442.Google ScholarGoogle ScholarCross RefCross Ref
  20. Eric Rohmer, Surya PN Singh, and Marc Freese. 2013. V-REP: A versatile and scalable robot simulation framework. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on. IEEE, 1321--1326.Google ScholarGoogle ScholarCross RefCross Ref
  21. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google ScholarGoogle Scholar
  22. Peter Stone, Richard S Sutton, and Gregory Kuhlmann. 2005. Reinforcement learning for robocup soccer keepaway. Adaptive Behavior 13, 3 (2005), 165--188.Google ScholarGoogle ScholarCross RefCross Ref
  23. Elio Tuci, Muhanad Hayder Mohammed Alkilabi, and Otar Akanyeti. 2018. Cooperative object transport in multi-robot systems: A review of the state-of-the-art. Frontiers in Robotics and AI 5 (2018), Article:59.Google ScholarGoogle Scholar
  24. Kai Wang, Manolis Savva, Angel X Chang, and Daniel Ritchie. 2018. Deep convolutional priors for indoor scene synthesis. ACM Transactions on Graphics (TOG) 37, 4 (2018), 70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to train your dragon: example-guided control of flapping flight. ACM Transactions on Graphics 36, 6 (2017), 198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Sai-Keung Wong, Yi-Hung Chou, and Hsiang-Yu Yang. 2018. A framework for simulating agent-based cooperative tasks in crowd simulation. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Sai-Keung Wong, Yu-Shuen Wang, Pao-Kun Tang, and Tsung-Yu Tsai. 2017. Optimized evacuation route based on crowd simulation. Computational Visual Media 3, 3 (2017), 243--261.Google ScholarGoogle ScholarCross RefCross Ref
  28. Wei Xiang, Jiaping Ren, Kuan Wang, Zhigang Deng, and Xiaogang Jin. 2018. Biologically inspired ant colony simulation. Computer Animation and Virtual Worlds (2018), e1867.Google ScholarGoogle Scholar
  29. Jun Xing, Rubaiat Habib Kazi, Tovi Grossman, Li-Yi Wei, Jos Stam, and George Fitzmaurice. 2016. Energy-brushes: Interactive tools for illustrating stylized elemental dynamics. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 755--766. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Fangyi Zhang, Jürgen Leitner, Michael Milford, Ben Upcroft, and Peter Corke. 2015. Towards vision-based deep reinforcement learning for robotic motion control. arXiv preprint arXiv: 1511.03791 (2015).Google ScholarGoogle Scholar
  31. Xinyi Zhang and Michiel van de Panne. 2018. Data-driven autocompletion for keyframe animation. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games. ACM, 10. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Agent-based cooperative animation for box-manipulation using reinforcement learning

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Article Metrics

        • Downloads (Last 12 months)49
        • Downloads (Last 6 weeks)5

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!