skip to main content
research-article

Interactive partner control in close interactions for real-time applications

Published:03 July 2013Publication History
Skip Abstract Section

Abstract

This article presents a new framework for synthesizing motion of a virtual character in response to the actions performed by a user-controlled character in real time. In particular, the proposed method can handle scenes in which the characters are closely interacting with each other such as those in partner dancing and fighting. In such interactions, coordinating the virtual characters with the human player automatically is extremely difficult because the system has to predict the intention of the player character. In addition, the style variations from different users affect the accuracy in recognizing the movements of the player character when determining the responses of the virtual character. To solve these problems, our framework makes use of the spatial relationship-based representation of the body parts called interaction mesh, which has been proven effective for motion adaptation. The method is computationally efficient, enabling real-time character control for interactive applications. We demonstrate its effectiveness and versatility in synthesizing a wide variety of motions with close interactions.

Skip Supplemental Material Section

Supplemental Material

References

  1. Arikan, O., Forsyth, D. A., and O'Brien, J. F. 2003. Motion synthesis from annotations. ACM Trans. Graph. 22, 3, 402--408. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Chan, J. C. P., Leung, H., Tang, J. K. T., and Komura, T. 2011. A virtual reality dance training system using motion capture technology. IEEE Trans. Learn. Technol. 4, 2, 187--195. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Davis, T. A. 2004. Algorithm 832: UMFPACK V4.3—An unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30, 2, 196--199. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Deng, L., Leung, H., Gu, N., and Yang, Y. 2011. Real-time mocap dance recognition for an interactive dancing game. Comput. Animation Virtual Worlds 22, 2--3, 229--237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Gleicher, M. 1998. Retargetting motion to new characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). ACM Press, New York, 33--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Goto, K. and Van De Geijn, R. 2008. High-performance implementation of the level-3 blas. ACM Trans. Math. Softw. 35, 1, 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Grochow, K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Style-based inverse kinematics. ACM Trans. Graph. 23, 3, 522--531. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Gross, R. and Shi, J. 2001. The cmu motion of body (mobo). Tech. rep. CMU-RI-TR-01-18. Database Robotics Institute, Carnegie Mellon University, Pittsburgh, PA. http://mocap.cs.cmu.edu.Google ScholarGoogle Scholar
  9. Ho, E. S. L. and Komura, T. 2009a. Character motion synthesis by topology coordinates. Comput. Graph. Forum 28, 2, 299--308.Google ScholarGoogle ScholarCross RefCross Ref
  10. Ho, E. S. L. and Komura, T. 2009b. Indexing and retrieving motions of characters in close contact. IEEE Trans. Vis. Comput. Graph. 15, 3, 481--492. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Ho, E. S. L., Komura, T., and Tai, C.-L. 2010. Spatial relationship preserving character motion adaptation. ACM Trans. Graph. 29, 4, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Ho, E. S. L. and Komura, T. 2011. A finite state machine based on topology coordinates for wrestling games. Comput. Animat. Virtual Worlds 22, 5, 435--443. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Hsu, E., Gentry, S., and Popovic, J. 2004. Example-based control of human motion. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'04). Eurographics Association, 69--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ikemoto, L., Arikan, O., and Forsyth, D. 2007. Quick transitions with cached multi-way blends. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D'07). ACM Press, New York, 145--151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Inamura, T., Nakamura, Y., Toshima, I., and Tanie, H. 2004. Embodied symbol emergence based on mimesis theory. Int. J. Robotics Res. 23, 4--5, 363--377.Google ScholarGoogle ScholarCross RefCross Ref
  16. Kovar, L. and Gleicher, M. 2004. Automated extraction and parameterization of motions in large data sets. ACM Trans. Graph. 23, 3, 559--568. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. ACM Trans. Graph. 21, 3, 473--482. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Krüger, B., Tautges, J., Weber, A., and Zinke, A. 2010. Fast local and global similarity searches in large motion capture databases. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'10). Eurographics Association, 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lee, D., Ott, C., and Nakamura, Y. 2009. Mimetic communication with impedance control for physical human-robot interaction. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'09). 1535--1542. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3, 491--500. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Lee, J. and Lee, K. H. 2004. Precomputing avatar behavior from human motion data. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'04). Eurographics Association, 79--87. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Liu, C. K., Hertzmann, A. and Popovic, Z. 2006. Composition of complex optimal multi-character motions. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'06). Eurographics Association, 215--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Magnenat-Thalmann, N., Protopsaltou, D., and Kavakli, E. 2008. Learning how to dance using a web 3d platform. In Proceedings of the 6th International Conference on Advances in Web Based Learning (ICWL'07). H. Leung, F. Li, R. Lau, and Q. Li, Eds., Lecture Notes in Computer Science, vol. 4823, Springer, 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Mount, D. M. and Arya, S. 2006. ANN: A library for approximate nearest neighbor searching. Programming manual. College Park, Maryland. http://www.cs.umd.edu/∼mount/ANN/.Google ScholarGoogle Scholar
  25. Nakamura and Yamane Laboratory. 2005. Animatronic humanoid robot project in prototype robot exhibition, Aichi EXPO. http://www.ynl.t.u-tokyo.ac.jp/research/expo2005/expo2005-e.html.Google ScholarGoogle Scholar
  26. Nakayama, D., Kosuge, K., and Hirata, Y. 2009. Human-adaptive step estimation method for a dance partner robot. In Proceedings of the IEEE International Conference on Automation and Logistics (ICAL'09). 191--196.Google ScholarGoogle Scholar
  27. Sakai, Y., Takeda, T., Hirata, Y., and Kosuge, K. 2007. Collision avoidance based on estimated step of other dance couples for male-type dance partner robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'07). 3264--3269.Google ScholarGoogle Scholar
  28. Shum, H. P. H., Komura, T., and Yamazaki, S. 2007. Simulating competitive interactions using singly captured motions. In Proceedings of the ACM Symposium on Virtual Reality Software Technology. 65--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Shum, H. P. H., Komura, T., and Yamazaki, S. 2008. Simulating interactions of avatars in high dimensional state space. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D'08). ACM Press, New York, 131--138. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Si, H. and Grtner, K. 2005. Meshing piecewise linear complexes by constrained delaunay tetrahedralizations. In Proceedings of the 14th International Meshing Roundtable, B. W. Hanks, Ed. Springer, 147--163.Google ScholarGoogle Scholar
  31. Sugihara, T. and Nakamura, Y. 2005. A fast online gait planning with boundary condition relaxation for humanoid robots. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'05). 305--310.Google ScholarGoogle Scholar
  32. Sugihara, T., Yamamoto, K., and Nakamura, Y. 2005. Architectural design of miniature anthropomorphic robots towards high mobility. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'05). 2869--2874.Google ScholarGoogle Scholar
  33. Takano, W., Imagawa, H., and Nakamura, Y. 2011. Prediction of human behaviors in the future through symbolic inference. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'11). 1970--1975.Google ScholarGoogle Scholar
  34. Takeda, T., Hirata, Y., and Kosuge, K. 2007a. Dance partner robot cooperative motion generation with adjustable length of dance step stride based on physical interaction. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'07). 3258--3263.Google ScholarGoogle Scholar
  35. Takeda, T., Hirata, Y., and Kosuge, K. 2007b. Dance step estimation method based on hmm for dance partner robot. IEEE Trans. Indust. Electron. 54, 2, 699--706.Google ScholarGoogle ScholarCross RefCross Ref
  36. Takeda, T., Kosuge, K., and Hirata, Y. 2005. HMM-based dance step estimation for dance partner robot -ms dancer. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'05). 3245--3250.Google ScholarGoogle Scholar
  37. Tang, J. K. T., Chan, J. C. P., and Leung, H. 2011. Interactive dancing game with real-time recognition of continuous dance moves from 3D human motion capture. In Proceedings of the 5th International Conference on Ubiquitous Information Management and Communication (ICUIMC'11). ACM Press, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Treuille, A., Lee, Y., and Popovic, Z. 2007. Near-optimal character animation with continuous control. In ACM SIGGRAPH Papers. ACM Press, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Tsuruta, S., Kawauchi, Y., Choi, W., and Hachimura, K. 2007. Real-time recognition of body motion for virtual dance collaboration system. In Proceedings of the International Conference on Artificial Reality and Telexistence. IEEE Computer Society, 23--30. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Interactive partner control in close interactions for real-time applications

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            • Published in

              cover image ACM Transactions on Multimedia Computing, Communications, and Applications
              ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 9, Issue 3
              June 2013
              121 pages
              ISSN:1551-6857
              EISSN:1551-6865
              DOI:10.1145/2487268
              Issue’s Table of Contents

              Copyright © 2013 ACM

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 3 July 2013
              • Accepted: 1 April 2013
              • Revised: 1 January 2013
              • Received: 1 April 2012
              Published in tomm Volume 9, Issue 3

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • research-article
              • Research
              • Refereed

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!