Learning to use chopsticks in diverse gripping styles

Abstract

Learning dexterous manipulation skills is a long-standing challenge in computer graphics and robotics, especially when the task involves complex and delicate interactions between the hands, tools and objects. In this paper, we focus on chopsticks-based object relocation tasks, which are common yet demanding. The key to successful chopsticks skills is steady gripping of the sticks that also supports delicate maneuvers. We automatically discover physically valid chopsticks holding poses by Bayesian Optimization (BO) and Deep Reinforcement Learning (DRL), which works for multiple gripping styles and hand morphologies without the need of example data. Given as input the discovered gripping poses and desired objects to be moved, we build physics-based hand controllers to accomplish relocation tasks in two stages. First, kinematic trajectories are synthesized for the chopsticks and hand in a motion planning stage. The key components of our motion planner include a grasping model to select suitable chopsticks configurations for grasping the object, and a trajectory optimization module to generate collision-free chopsticks trajectories. Then we train physics-based hand controllers through DRL again to track the desired kinematic trajectories produced by the motion planner. We demonstrate the capabilities of our framework by relocating objects of various shapes and sizes, in diverse gripping styles and holding positions for multiple hand morphologies. Our system achieves faster learning speed and better control robustness, when compared to vanilla systems that attempt to learn chopstick-based skills without a gripping pose optimization module and/or without a kinematic motion planner. Our code and models are available at this link.¹

References

1. Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. 2019. Solving rubik's cube with a robot hand. arXiv preprint arXiv:1910.07113 (2019).
2. Sheldon Andrews and Paul G Kry. 2013. Goal directed multi-finger manipulation: Control policies and analysis. Computers & Graphics 37, 7 (2013), 830--839.
3. Yahya Aydin and Masayuki Nakajima. 1999. Database guided computer animation of human grasping using forward and inverse kinematics. Computers & Graphics 23, 1 (1999), 145--154.
4. Kevin Bergamin, Simon Clavet, Daniel Holden, and James Richard Forbes. 2019. DReCon: Data-driven responsive control of physics-based characters. ACM Transactions on Graphics (TOG) 38, 6, Article 206 (2019).
5. Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian interactive optimization approach to procedural animation design. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 103--112.
6. Eric Brochu, Abhijeet Ghosh, and Nando de Freitas. 2007. Preference galleries for material design. SIGGRAPH Posters 105, 10.1145 (2007), 1280720--1280834.
7. Bao-Chi Chang, Biing-Shiun Huang, Ching-Kong Chen, and Shyh-Jen Wang. 2007. The pincer chopsticks: The investigation of a new utensil in pinching function. Applied ergonomics 38, 3 (2007), 385--390.
8. Tao Chen, Jie Xu, and Pulkit Agrawal. 2021. A system for general in-hand object re-orientation. In Conference on Robot Learning. PMLR.
9. George ElKoura and Karan Singh. 2003. Handrix: animating the human hand. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 110--119.
10. Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. 2020a. Graspnet-1billion: A large-scale benchmark for general object grasping. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). 11444--11453.
11. Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, and Silvio Savarese. 2020b. Learning task-oriented grasping for tool manipulation from simulated self-supervision. The International Journal of Robotics Research 39, 2--3 (2020), 202--216.
12. Guillermo Garcia-Hernando, Edward Johns, and Tae-Kyun Kim. 2020. Physics-based dexterous manipulations with estimated hand poses and residual reinforcement learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 9561--9568.
13. GPy. since 2012. GPy: A Gaussian process framework in python. http://github.com/SheffieldML/GPy.
14. Philipp Hennig and Christian J Schuler. 2012. Entropy search for information-efficient global optimization. The Journal of Machine Learning Research 13, 1 (2012), 1809--1837.
15. Daniel Holden, Jun Saito, Taku Komura, and Thomas Joyce. 2015. Learning motion manifolds with convolutional autoencoders. In SIGGRAPH Asia 2015 Technical Briefs. 1--4.
16. Sha Hu, Zeshi Yang, and Greg Mori. 2021. Neural fidelity warping for efficient robot morphology design. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 7079--7086.
17. Zhiyong Huang, Ronan Boulic, Nadia Magnenat Thalmann, and Daniel Thalmann. 1995. A multi-sensor approach for grasping and 3D interaction. In Computer graphics. Elsevier, 235--253.
18. Sumit Jain and C Karen Liu. 2011. Controlling physics-based characters using soft contacts. ACM Transactions on Graphics (TOG) 30, 6 (2011), 1--10.
19. Rohan A Joseph, Alvin C Goh, Sebastian P Cuevas, Michael A Donovan, Matthew G Kauffman, Nilson A Salas, Brian Miles, Barbara L Bass, and Brian J Dunkin. 2010. "Chopstick" surgery: A novel technique improves surgeon performance and eliminates arm collision in robotic single-incision laparoscopic surgery. Surgical endoscopy 24, 6 (2010), 1331--1335.
20. Korrawe Karunratanakul, Jinlong Yang, Yan Zhang, Michael J Black, Krikamol Muandet, and Siyu Tang. 2020. Grasping field: Learning implicit representations for human grasps. In 2020 International Conference on 3D Vision (3DV). IEEE, 333--344.
21. Liyiming Ke, Ajinkya Kamat, Jingqiang Wang, Tapomayukh Bhattacharjee, Christoforos Mavrogiannis, and Siddhartha S Srinivasa. 2020. Telemanipulation with chopsticks: Analyzing human factors in user demonstrations. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 11539--11546.
22. Liyiming Ke, Jingqiang Wang, Tapomayukh Bhattacharjee, Byron Boots, and Siddhartha Srinivasa. 2021. Grasping with chopsticks: Combating covariate shift in model-free imitation learning for fine manipulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 6185--6191.
23. Junhwan Kim, Frederic Cordier, and Nadia Magnenat-Thalmann. 2000. Neural network-based violinist's hand animation. In Proceedings Computer Graphics International 2000. IEEE, 37--41.
24. Seungsu Kim, Ashwini Shukla, and Aude Billard. 2014. Catching objects in flight. IEEE Transactions on Robotics 30, 5 (2014), 1049--1065.
25. Uikyum Kim, Dawoon Jung, Heeyoen Jeong, Jongwoo Park, Hyun-Mok Jung, Joono Cheong, Hyouk Ryeol Choi, Hyunmin Do, and Chanhun Park. 2021. Integrated linkage-driven dexterous anthropomorphic robotic hand. Nature Communications 12, 1 (2021), 1--13.
26. Yoshihito Koga, Koichi Kondo, James Kuffner, and Jean-Claude Latombe. 1994. Planning motions with intentions. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques. 395--408.
27. Yuki Koyama, Issei Sato, and Masataka Goto. 2020. Sequential gallery for interactive visual design optimization. ACM Transactions on Graphics (TOG) 39, 4, Article 88 (2020).
28. Paul G Kry and Dinesh K Pai. 2006. Interaction capture and synthesis. ACM Transactions on Graphics (TOG) 25, 3 (2006), 872--880.
29. Shutao Li, Mingkui Tan, Ivor W Tsang, and James Tin-Yau Kwok. 2011. A hybrid PSO-BFGS strategy for global optimization of multimodal functions. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 41, 4 (2011), 1003--1014.
30. C Karen Liu. 2008. Synthesis of interactive hand manipulation. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 163--171.
31. C Karen Liu. 2009. Dextrous manipulation from a grasping pose. ACM Transactions on Graphics (TOG) 28, 3, Article 59 (2009).
32. Libin Liu and Jessica Hodgins. 2017. Learning to schedule control fragments for physics-based characters using deep Q-learning. ACM Transactions on Graphics (TOG) 36, 3, Article 42a (2017).
33. Macro. since 2021. Ten thousand ways to use chopsticks. https://marcosticks.org/poster-ten-thousand-ways-to-use-chopsticks/.
34. Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, and Nicolas Heess. 2020. Catch & Carry: Reusable neural controllers for vision-guided whole-body tasks. ACM Transactions on Graphics (TOG) 39, 4, Article 39 (2020).
35. Lester James V. Miranda. 2018. PySwarms, a research-toolkit for particle swarm optimization in Python. Journal of Open Source Software 3 (2018).
36. Igor Mordatch, Zoran Popović, and Emanuel Todorov. 2012. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 137--144.
37. Yukiko Mukai and Keiko Hashimoto. 1978. A study on ways of holding chopsticks. Journal of Home Economics of Japan 29, 7 (1978), 467--473.
38. Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. 2020. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning. PMLR, 1101--1112.
39. Tomoko Osera, Chihiro Yamamoto, Rika Senke, Misako Kobayashi, Setsuko Tsutie, and Nobutaka Kurihara. 2018. Relationship between mothers and children on how to hold chopsticks and concerns about chopsticks in japanese kindergarten. Journal of Japanese Society of Shokuiku 12, 1 (2018), 19--25.
40. Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and Jehee Lee. 2019. Learning predict-and-simulate policies from unorganized human motion data. ACM Transactions on Graphics (TOG) 38, 6, Article 205 (2019).
41. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).
42. Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018a. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics (TOG) 37, 4, Article 143 (2018).
43. Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. 2018b. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 3803--3810.
44. Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel Van De Panne. 2017. Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Transactions on Graphics (TOG) 36, 4, Article 41 (2017).
45. Xue Bin Peng, Michael Chang, Grace Zhang, Pieter Abbeel, and Sergey Levine. 2019. MCP: Learning composable hierarchical control with multiplicative compositional policies. Advances in Neural Information Processing Systems 32 (2019).
46. Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Edward Lee, Jie Tan, and Sergey Levine. 2020. Learning Agile Robotic Locomotion Skills by Imitating Animals. In Proceedings of Robotics: Science and Systems.
47. Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. 2021. AMP: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (TOG) 40, 4, Article 144 (2021).
48. Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073 (2017).
49. Madhu Ragupathi, Diego I Ramos-Valadez, Rodrigo Pedraza, and Eric M Haas. 2010. Robotic-assisted single-incision laparoscopic partial cecectomy. The International Journal of Medical Robotics and Computer Assisted Surgery 6, 3 (2010), 362--367.
50. Akshara Rai, Rika Antonova, Seungmoon Song, William Martin, Hartmut Geyer, and Christopher Atkeson. 2018. Bayesian optimization using domain knowledge on the ATRIAS biped. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 1771--1778.
51. Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. 2018. Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. In Proceedings of Robotics: Science and Systems.
52. Ahmed A Ramadan, Tomohito Takubo, Yasushi Mae, Kenichi Oohara, and Tatsuo Arai. 2009. Developmental process of a chopstick-like hybrid-structure two-fingered micromanipulator hand for 3-D manipulation of microscopic objects. IEEE Transactions on Industrial Electronics 56, 4 (2009), 1121--1135.
53. Carl Edward Rasmussen. 2003. Gaussian processes in machine learning. In Summer School on Machine Learning. Springer, 63--71.
54. Haruka Sakurai, Takahiro Kanno, and Kenji Kawashima. 2016. Thin-diameter chopsticks robot for laparoscopic surgery. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4122--4127.
55. John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy optimization. In International conference on machine learning. PMLR, 1889--1897.
56. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).
57. Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. 2010. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th annual international conference on machine learning. 1015--1022.
58. Omid Taheri, Nima Ghorbani, Michael J Black, and Dimitrios Tzionas. 2020. GRAB: A dataset of whole-body human grasping of objects. In European conference on computer vision (ECCV). Springer, 581--600.
59. Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. 2018. Sim-to-Real: Learning Agile Locomotion For Quadruped Robots. In Proceedings of Robotics: Science and Systems.
60. Emanuel Todorov. 2014. Convex and analytically-invertible dynamics with contacts and constraints: Theory and implementation in MuJoCo. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 6054--6061.
61. Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 5026--5033.
62. Deepak Tolani and Norman I. Badler. 1996. Real-time inverse kinematics of the human arm. Presence: Teleoperators and Virtual Environments 5, 4 (1996), 393--401.
63. Marc A Toussaint, Kelsey Rebecca Allen, Kevin A Smith, and Joshua B Tenenbaum. 2018. Differentiable physics and stable modes for tool-use and manipulation planning. (2018).
64. Yangang Wang, Jianyuan Min, Jianjie Zhang, Yebin Liu, Feng Xu, Qionghai Dai, and Jinxiang Chai. 2013. Video-based hand manipulation capture through composite motion control. ACM Transactions on Graphics (TOG) 32, 4, Article 43 (2013).
65. Nkenge Wheatland, Yingying Wang, Huaguang Song, Michael Neff, Victor Zordan, and Sophie Jörg. 2015. State of the art in hand and finger modeling and animation. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 735--760.
66. Jungdam Won and Jehee Lee. 2019. Learning body shape variation in physics-based characters. ACM Transactions on Graphics (TOG) 38, 6, Article 207 (2019).
67. Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, and Pieter Abbeel. 2019. Learning to manipulate deformable objects without demonstrations. arXiv preprint arXiv:1910.13439 (2019).
68. Kazuki Yamakawa, Yuko Tashiro, Mizuki Nakajima, and Tsuyoshi Saitoh. 2018. Development of a support system for holding chopsticks correctly. In 2018 International Workshop on Advanced Image Technology (IWAIT). IEEE, 1--2.
69. Tomoko Yamauchi, Atsumi Koide, Atsuko Yamamoto, and Kazuko Oba. 2010. Effect of parental training on table manners and the way of holding chopsticks. Journal of Cookery Science of Japan 43, 4 (2010), 260--264.
70. Akira Yamazaki and Ryosuke Masuda. 2012. Autonomous foods handling by chopsticks for meal assistant robot. In ROBOTIK 2012; 7th German Conference on Robotics. VDE, 1--6.
71. Zeshi Yang and Zhiqi Yin. 2021. Efficient hyperparameter optimization for physics-based character animation. Proceedings of the ACM on Computer Graphics and Interactive Techniques 4, 1 (2021), 1--19.
72. Yuting Ye and C Karen Liu. 2012. Synthesis of detailed hand manipulations using contact sampling. ACM Transactions on Graphics (TOG) 31, 4, Article 41 (2012).
73. Zhiqi Yin, Zeshi Yang, Michel Van de Panne, and Kangkang Yin. 2021. Discovering Diverse Athletic Jumping Strategies. ACM Transactions on Graphics (TOG) 40, 4, Article 91.
74. Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. 2020. Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics 36, 4 (2020), 1307--1319.
75. Wenyuan Zeng, Wenjie Luo, Simon Suo, Abbas Sadat, Bin Yang, Sergio Casas, and Raquel Urtasun. 2019. End-to-end interpretable neural motion planner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8660--8669.
76. He Zhang, Yuting Ye, Takaaki Shiratori, and Taku Komura. 2021. ManipNet: Neural manipulation synthesis with a hand-object spatial representation. ACM Transactions on Graphics (TOG) 40, 4, Article 121 (2021).
77. Yunbo Zhang, Wenhao Yu, C Karen Liu, Charlie Kemp, and Greg Turk. 2020. Learning to manipulate amorphous materials. ACM Transactions on Graphics (TOG) 39, 6, Article 189 (2020).
78. Wenping Zhao, Jianjie Zhang, Jianyuan Min, and Jinxiang Chai. 2013. Robust realtime physics-based motion control for human grasping. ACM Transactions on Graphics (TOG) 32, 6, Article 207 (2013).

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

      • Physical simulation

  • Machine learning

    • Learning paradigms

      • Reinforcement learning