Synthesis of detailed hand manipulations using contact sampling

Abstract

Capturing human activities that involve both gross full-body motion and detailed hand manipulation of objects is challenging for standard motion capture systems. We introduce a new method for creating natural scenes with such human activities. The input to our method includes motions of the full-body and the objects acquired simultaneously by a standard motion capture system. Our method then automatically synthesizes detailed and physically plausible hand manipulation that can seamlessly integrate with the input motions. Instead of producing one "optimal" solution, our method presents a set of motions that exploit a wide variety of manipulation strategies. We propose a randomized sampling algorithm to search for as many as possible visually diverse solutions within the computational time budget. Our results highlight complex strategies human hands employ effortlessly and unconsciously, such as static, sliding, rolling, as well as finger gaits with discrete relocation of contact points.

References

1. Albrecht, I., Haber, J., and Seidel, H.-P. 2003. Construction and animation of anatomically based human hand models. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 98--109.
2. Andersen, E. D., Jensen, B., Jensen, J., and Sandvik, R. 2009. Mosek version 6. Tech. Rep. TR-2009-3, Ulf Worsøe, October.
3. Aydin, Y., and Nakajima, M. 1999. Database guided computer animation of human grasping using forward and inverse kinematics. Computers and Graphics 23, 1, 145--154.
4. Cai, C., and Roth, B. 1987. On the spatial motion of a rigid body with point contact. In Proc. IEEE Int. Con. Robotics and Automation, 686--695.
5. Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. In SISGGRAPH, 219--228.
6. Cherif, M., and Gupta, K. K. 1999. Planning quasi-static fingertip manipulations for reconfiguring objects. IEEE Trans. on Robotics and Automation 15, 5, 837--848.
7. Choi, M. G., Lee, J., and Shin, S. Y. 2003. Planning biped locomotion using motion capture data and probabilistic roadmaps. ACM Transactions on Graphics 22, 182--203.
8. Ciocarlie, M. T., and Allen, P. K. 2008. On-line interactive dexterous grasping. Eurohaptics (June).
9. Ciocarlie, M. T., Lackner, C., and Allen, P. K. 2007. Soft finger model with adaptive contact geometry for grasping and manipulation tasks. IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (March).
10. Cole, A., Hsu, P., and Sastry, S. 1992. Dynamic control of sliding by robot hands for regrasping. IEEE Trans. on Robotics and Automation 8, 1, 42--52.
11. Coumans, E., 2005. Bullet physics engine. http://bulletphysics.org.
12. Cyberglove Systems, http://www.cyberglovesystems.com/.
13. Deshpande, A., Ko, J., Fox, D., and Matsuoka, Y. 2009. Anatomically correct testbed hand control: Muscle and joint control strategies. In Robotics and Automation, 2009. ICRA '09. IEEE International Conference on, 4416--4422.
14. ElKoura, G., and Singh, K. 2003. Handrix: Animating the human hand. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 110--119.
15. Gill, P., Saunders, M., and Murray, W. 1996. Snopt: An sqp algorithm for large-scale constrained optimization. Tech. Rep. NA 96-2, University of California, San Diego.
16. Gleicher, M. 1998. Retargeting motion to new characters. In SIGGRAPH, 33--42.
17. Hamer, H., Gall, J., Urtasun, R., and Gool, L. V. 2011. Data-driven animation of hand-object interactions. In IEEE Conference on Automatic Face and Gesture Recognition, 360--367.
18. Han, L., and Trinkle, J. 1998. Dextrous manipulation by rolling and finger gaiting. In ICRA, IEEE, 730--735.
19. Ho, E. S., Komura, T., and Tai, C.-L. 2010. Spatial relationship preserving character motion adaptation. ACM Trans. Graph 29, 3.
20. Hong, J., Lafferriere, G., Mishra, B., and Tang, X. 1990. Fine manipulation with multifinger hand. In ICRA, IEEE, 1568--1573.
21. Huang, Z., Boulic, R., and Thalmann, D. 1995. A multi-sensor approach for grasping and 3-D interaction. In Computer Graphics International '95.
22. Jain, S., and Liu, C. K. 2009. Interactive synthesis of human-object interaction. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 173--182.
23. Jain, S., and Liu, C. K. 2011. Controlling physics-based characters using soft contacts. ACM Trans. Graph. (SIGGRAPH Asia) 30 (Dec.), 163:1--163:10.
24. Joerg, S., Hodgins, J., and Sullivan, C. 2010. The perception of finger motions. Applied Perception in Graphics and Visualization (APGV).
25. Kavraki, L., Svestka, P., Latombe, J.-C., and Overmars, M. H. 1996. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. on Robotics and Automation 12, 4, 566--580.
26. Kim, J., Cordier, F., and Magnenat-Thalmann, N. 2000. Neural network-based violinistâs hand animation. In Conference on Computer Graphics International, 37--44.
27. Koga, Y., Kondo, K., Kuffner, J., and Latombe, J.-C. 1994. Planning motions with intentions. In SIGGRAPH, 395--408.
28. Kry, P. G., and Pai, D. K. 2006. Interaction capture and synthesis. ACM Trans. on Graphics 25, 3 (Aug.), 872--880.
29. Lavalle, S. M., and Kuffner, J. J. 2000. Rapidly-exploring random trees: Progress and prospects.
30. Liu, L., Yin, K., van de Panne, M., Shao, T., and Xu, W. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph.(SIGGRAPH) 29, 4, 1--10.
31. Liu, C. K. 2009. Dextrous manipulation from a single grasping pose. ACM Transactions on Graphics (SIGGRAPH) 28, 3 (Aug.).
32. Majkowska, A., Zordan, V. B., and Faloutsos, P. 2006. Automatic splicing for hand and body animations. In Eurographics/SIGGRAPH Symposium on Computer Animation.
33. Miller, A., and Allen, P. 1999. Examples of 3d grasp quality computations. In Robotics and Automation, 1999. Proceedings. 1999 IEEE International Conference on, vol. 2, 1240--1246 vol.2.
34. Ngo, J. T., and Marks, J. 1993. Spacetime constraints revisited. In SIGGRAPH, vol. 27, 343--350.
35. Pollard, N. S., and Hodgins, J. K. 2002. Generalizing demonstrated manipualtion tasks. In Workshop on the Algorithmic Foundations of Robotics (WAFR '02).
36. Pollard, N. S., and Wolf, A. 2004. Grasp synthesis from example: tuning the example to a task or object. In Workshop on Multi-point Interaction in Robotics and Virtual Reality, IEEE International Conference on Robotics and Automation, 77--90.
37. Pollard, N. S., and Zordan, V. B. 2005. Physically based grasping control from example. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 311--318.
38. Pollard, N. S. 2004. Closure and quality equivalence for efficient synthesis of grasps from examples. International Journal of Robotics Research 23, 6 (June), 595--614.
39. Sims, K. 1994. Evolving virtual creatures. In SIGGRAPH.
40. Sok, K. W., Kim, M., and Lee, J. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. (SIGGRAPH) 26, 3, 107.
41. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. on Graphics 27, 3 (Aug.).
42. Tournassoud, P., Lozano-Perez, T., and Mazer, E. 1987. Regrasping. In Proc. IEEE Int. Con. Robotics and Automation, 1924--1928.
43. Tsang, W., Singh, K., and Fiume, E. 2005. Helping hand: An anatomically accurate inverse dynamics solution for unconstrained hand motion. In Eurographics/SIGGRAPH Symposium on Computer Animation, 1--10.
44. Twigg, C. D., and James, D. L. 2007. Many-worlds browsing for control of multibody dynamics. ACM Trans. Graph 26, 3.
45. van de Panne, M., and Fiume, E. 1993. Sensor-actuator networks. In SIGGRAPH, vol. 27, 335--342.
46. Xu, J., Koo, T.-K. J., and Li, Z. 2007. Finger gaits planning for multifingered manipulation. In IROS, IEEE, 2932--2937.
47. Yamane, K., Kuffner, J. J., and Hodgins, J. K. 2004. Synthesizing animations of human manipulation tasks. ACM Trans. Graph 23, 3, 532--539.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation