Eyecatch: simulating visuomotor coordination for object interception

Abstract

We present a novel framework for animating human characters performing fast visually guided tasks, such as catching a ball. The main idea is to consider the coordinated dynamics of sensing and movement. Based on experimental evidence about such behaviors, we propose a generative model that constructs interception behavior online, using discrete submovements directed by uncertain visual estimates of target movement. An important aspect of this framework is that eye movements are included as well, and play a central role in coordinating movements of the head, hand, and body. We show that this framework efficiently generates plausible movements and generalizes well to novel scenarios.

References

1. Abe, Y., and Popović, J. 2006. Interactive animation of dynamic manipulation. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, Eurographics Association, 195--204.
2. Bauml, B., Schmidt, F., Wimbock, T., Birbach, O., Dietrich, A., Fuchs, M., Friedl, W., Frese, U., Borst, C., Grebenstein, M., et al. 2011. Catching flying balls and preparing coffee: Humanoid rollin'justin performs dynamic and sensitive tasks. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE, 3443--3444.
3. Beek, P., and Lewbel, A. 1995. The science of juggling. Scientific American 273, 5, 92--97.
4. Berthoz, A. 2000. The brain's sense of movement. Harvard Univ Pr.
5. Birbach, O., Frese, U., and Bauml, B. 2011. Realtime perception for catching a flying ball with a mobile humanoid. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE, 5955--5962.
6. Carpenter, R., and Williams, M. 1995. Neural computation of log likelihood in control of saccadic eye movements. Nature 377, 6544, 59--62.
7. Cooper, S., Hertzmann, A., and Popović, Z. 2007. Active learning for real-time motion controllers. In ACM Transactions on Graphics (TOG), vol. 26, ACM, 5.
8. Crawford, J., Martinez-Trujillo, J., and Klier, E. 2003. Neural control of three-dimensional eye and head movements. Current opinion in neurobiology 13, 6, 655--662.
9. Dessing, J., Peper, C., Bullock, D., and Beek, P. 2005. How position, velocity, and temporal information combine in the prospective control of catching: Data and model. Journal of cognitive neuroscience 17, 4, 668--686.
10. Dipietro, L., Krebs, H., Fasoli, S., Volpe, B., and Hogan, N. 2009. Submovement changes characterize generalization of motor recovery after stroke. Cortex 45, 3, 318--324.
11. Flash, T., and Hogan, N. 1985. The coordination of arm movements: an experimentally confirmed mathematical model. The journal of Neuroscience 5, 7, 1688--1703.
12. Francik, J., and Szarowicz, A. 2005. Character animation with decoupled behaviour and smart objects. In 6th International Conference on Computer Games CGAIMS, Louisville, Kentucky, USA.
13. Freedman, E., Stanford, T., and Sparks, D. 1996. Combined eye-head gaze shifts produced by electrical stimulation of the superior colliculus in rhesus monkeys. Journal of neurophysiology 76, 2, 927--952.
14. Freedman, E. 2001. Interactions between eye and head control signals can account for movement kinematics. Biological cybernetics 84, 6, 453--462.
15. Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., and Sasse, M. 2003. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. in Proceedings of the SIGCHI conference on Human factors in computing systems, ACM, 529--536.
16. Gillies, M., and Dodgson, N. 1999. Ball catching: An example of psychologically-based behavioural animation. Eurographics UK.
17. Grillon, H., and Thalmann, D. 2009. Simulating gaze attention behaviors for crowds. Computer Animation and Virtual Worlds 20, 2-3, 111--119.
18. Grochow, K., Martin, S. L., Hertzmann, A., and Popović, Z. 2004. Style-based inverse kinematics. In ACM SIGGRAPH 2004 Papers, ACM, New York, NY, USA, SIGGRAPH '04, 522--531.
19. Gu, E., and Badler, N. 2006. Visual attention and eye gaze during multiparty conversations with distractions. In Intelligent Virtual Agents, Springer, 193--204.
20. Hayhoe, M., and Ballard, D. 2005. Eye movements in natural behavior. Trends in cognitive sciences 9, 4, 188--194.
21. Hebb, D. 1949. The organization of behavior: A neuropsychological theory. Lawrence Erlbaum.
22. Hove, B., and Slotine, J. 1991. Experiments in robotic catching. In American Control Conference, 1991, IEEE, 380--386.
23. Itti, L. 2003. Realistic avatar eye and head animation using a neurobiological model of visual attention. Tech. rep., DTIC Document.
24. Itti, L. 2006. Quantitative modelling of perceptual salience at human eye position. Visual cognition 14, 4--8, 959--984.
25. Johansson, R., Westling, G., Bäckström, A., and Flanagan, J. 2001. Eye-hand coordination in object manipulation. the Journal of Neuroscience 21, 17, 6917--6932.
26. Lanman, J., Bizzi, E., and Allum, J. 1978. The coordination of eye and head movement during smooth pursuit. Brain Research 153, 1, 39--53.
27. Lee, S., and Terzopoulos, D. 2006. Heads up!: biomechanical modeling and neuromuscular control of the neck. In ACM Transactions on Graphics (TOG), vol. 25, ACM, 1188--1198.
28. Lee, S., Badler, J., and Badler, N. 2002. Eyes alive. In ACM Transactions on Graphics (TOG), vol. 21, ACM, 637--644.
29. Lee, S., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics (TOG) 28, 4, 99.
30. Leigh, R., and Zee, D. 1999. The neurology of eye movements. No. 55. Oxford Univ Pr.
31. Liu, C. 2009. Dextrous manipulation from a grasping pose. In ACM Transactions on Graphics (TOG), vol. 28, ACM, 59.
32. McIntyre, J., Zago, M., Berthoz, A., Lacquaniti, F., et al. 2001. Does the brain model newton's laws? Nature Neuroscience 4, 7, 693--694.
33. McKee, S. 1981. A local mechanism for differential velocity detection. Vision Research 21, 4, 491--500.
34. Meyer, C., Lasker, A., and Robinson, D. 1985. The upper limit of human smooth pursuit velocity. Vision Research 25, 4, 561--563.
35. Novak, K., Miller, L., and Houk, J. 2002. The use of overlapping submovements in the control of rapid hand movements. Experimental Brain Research 144, 3, 351--364.
36. Orban de Xivry, J., and Lefèvre, P. 2007. Saccades and pursuit: two outcomes of a single sensorimotor process. The Journal of Physiology 584, 1, 11--23.
37. Pelachaud, C., and Bilvi, M. 2003. Modelling gaze behavior for conversational agents. In Intelligent Virtual Agents, Springer, 93--100.
38. Peters, C., and Qureshi, A. 2010. A head movement propensity model for animating gaze shifts and blinks of virtual characters. Computers & Graphics 34, 6, 677--687.
39. Plamondon, R. 1995. A kinematic theory of rapid human movements. Biological Cybernetics 72, 4, 295--307.
40. Pollard, N., and Zordan, V. 2005. Physically based grasping control from example. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM, 311--318.
41. Riley, M., and Atkeson, C. 2002. Robot catching: Towards engaging human-humanoid interaction. Autonomous Robots 12, 1, 119--128.
42. Robinson, D., Gordon, J., and Gordon, S. 1986. A model of the smooth pursuit eye movement system. Biological Cybernetics 55, 1, 43--57.
43. Robinson, D. 1965. The mechanics of human smooth pursuit eye movement. The Journal of Physiology 180, 3, 569.
44. Rohrer, B., and Hogan, N. 2003. Avoiding spurious submovement decompositions: a globally optimal algorithm. Biological cybernetics 89, 3, 190--199.
45. Shao, W., and Terzopoulos, D. 2005. Autonomous pedestrians. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM, 19--28.
46. Soechting, J., and Terzuolo, C. 1987. Organization of arm movements. motion is segmented. Neuroscience 23, 1, 39--51.
47. Starkes, J., Helsen, W., and Elliott, D. 2002. A menage a trois: the eye, the hand and on-line processing. Journal of sports sciences 20, 3, 217--224.
48. Sternad, D., and Schaal, S. 1999. Segmentation of end-point trajectories does not imply segmented control. Experimental Brain Research 124, 1, 118--136.
49. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3, 83:1--83:8.
50. Terzopoulos, D., and Rabie, T. 1997. Animat vision: Active vision in artificial animals. Videre. Journal of Computer Vision Research 1, 1, 2--19.
51. Tsang, W., Singh, K., and Fiume, E. 2005. Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM, 319--328.
52. Tu, X., and Terzopoulos, D. 1994. Artificial fishes: Physics, locomotion, perception, behavior. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, ACM, 43--50.
53. Urtasun, R., Fleet, D. J., Geiger, A., Popović, J., Darrell, T. J., and Lawrence, N. D. 2008. Topologically-constrained latent variable models. In Proceedings of the 25th international conference on Machine learning, ACM, New York, NY, USA, ICML '08, 1080--1087.
54. Vallbo, A., and Wessberg, J. 1993. Organization of motor output in slow finger movements in man. The Journal of physiology 469, 1, 673.
55. Wang, J. M., Fleet, D. J., and Hertzmann, A. 2007. Multifactor gaussian process models for style-content separation. In Proceedings of the 24th international conference on Machine learning, ACM, New York, NY, USA, ICML '07, 975--982.
56. Wolpert, D., Ghahramani, Z., and Jordan, M. 1995. An internal model for sensorimotor integration. Science 269, 5232, 1880.
57. Woodworth, R. 1899. Accuracy of voluntary movement. The Psychological Review: Monograph Supplements 3, 3, i.
58. Yamane, K., Kuffner, J., and Hodgins, J. 2004. Synthesizing animations of human manipulation tasks. In ACM Transactions on Graphics (TOG), vol. 23, ACM, 532--539.
59. Young, L., and Stark, L. 1963. Variable feedback experiments testing a sampled data model for eye tracking movements. Human Factors in Electronics, IEEE Transactions on, 1, 38--51.
60. Zago, M., McIntyre, J., Senot, P., and Lacquaniti, F. 2009. Visuo-motor coordination and internal models for object interception. Experimental Brain Research 192, 4, 571--604.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation