DiffAqua: a differentiable computational design pipeline for soft underwater swimmers with shape interpolation

Abstract

The computational design of soft underwater swimmers is challenging because of the high degrees of freedom in soft-body modeling. In this paper, we present a differentiable pipeline for co-designing a soft swimmer's geometry and controller. Our pipeline unlocks gradient-based algorithms for discovering novel swimmer designs more efficiently than traditional gradient-free solutions. We propose Wasserstein barycenters as a basis for the geometric design of soft underwater swimmers since it is differentiable and can naturally interpolate between bio-inspired base shapes via optimal transport. By combining this design space with differentiable simulation and control, we can efficiently optimize a soft underwater swimmer's performance with fewer simulations than baseline methods. We demonstrate the efficacy of our method on various design problems such as fast, stable, and energy-efficient swimming and demonstrate applicability to multi-objective design.

References

1. Melinos Averkiou, Vladimir G Kim, Youyi Zheng, and Niloy J Mitra. 2014. Shapesynth: Parameterizing model collections for coupled shape exploration and synthesis. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 125--134.
2. Seung-Yeob Baek, Jeonghun Lim, and Kunwoo Lee. 2015. Isometric shape interpolation. Computers & Graphics 46 (2015), 257--263.
3. Jernej Barbič, Marco da Silva, and Jovan Popović. 2009. Deformable object animation using reduced optimal control. ACM Transactions on Graphics (TOG) 28, 3 (2009), 1--9.
4. Jernej Barbič and Jovan Popović. 2008. Real-time control of physically based simulations using gentle forces. ACM transactions on graphics (TOG) 27, 5 (2008), 1--10.
5. Filipe de A Belbute-Peres, Kevin A Smith, Kelsey R Allen, Joshua B Tenenbaum, and J Zico Kolter. 2018. End-to-end differentiable physics for learning and control. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 7178--7189.
6. Florian Berlinger, Mihai Duduta, Hudson Gloria, David Clarke, Radhika Nagpal, and Robert Wood. 2018. A modular dielectric elastomer actuator to drive miniature autonomous underwater vehicles. In 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3429--3435.
7. Nicolas Bonneel, Gabriel Peyré, and Marco Cuturi. 2016. Wasserstein barycentric coordinates: histogram regression using optimal transport. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1--10.
8. Alexander M Bronstein, Michael M Bronstein, Leonidas J Guibas, and Maks Ovsjanikov. 2011. Shape google: Geometric words and expressions for invariant shape retrieval. ACM Transactions on Graphics (TOG) 30, 1 (2011), 1--20.
9. Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. 2018. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 6572--6583.
10. Nick Cheney, Robert MacCurdy, Jeff Clune, and Hod Lipson. 2014. Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. ACM SIGEVOlution 7, 1 (2014), 11--23.
11. Francesco Corucci, Nick Cheney, Hod Lipson, Cecilia Laschi, and Josh Bongard. 2016. Evolving swimming soft-bodied creatures. In ALIFE XV, The Fifteenth International Conference on the Synthesis and Simulation of Living Systems, Late Breaking Proceedings, Vol. 6.
12. Jonas Degrave, Michiel Hermans, Joni Dambre, and Francis wyffels. 2019. A Differentiable Physics Engine for Deep Learning in Robotics. Frontiers in Neurorobotics 13 (2019), 6.
13. Cosimo Della Santina, Robert K Katzschmann, Antonio Biechi, and Daniela Rus. 2018. Dynamic control of soft robots interacting with the environment. In 2018 IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 46--53.
14. Tao Du, Adriana Schulz, Bo Zhu, Bernd Bickel, and Wojciech Matusik. 2016. Computational multicopter design. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--10.
15. Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. 2021. DiffPD: Differentiable Projective Dynamics with Contact. arXiv preprint arXiv:2101.05917 (2021).
16. FE Fish and George V Lauder. 2006. Passive and active flow control by swimming fishes and mammals. Annu. Rev. Fluid Mech. 38 (2006), 193--224.
17. Noa Fish, Melinos Averkiou, Oliver Van Kaick, Olga Sorkine-Hornung, Daniel Cohen-Or, and Niloy J Mitra. 2014. Meta-representation of shape families. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--11.
18. Thomas Geijtenbeek, Michiel Van De Panne, and A Frank Van Der Stappen. 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1--11.
19. Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski, and Stelian Coros. 2020. ADD: analytically differentiable dynamics for multi-body systems with frictional contact. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--15.
20. Markus Giftthaler, Michael Neunert, Markus Stäuble, Marco Frigerio, Claudio Semini, and Jonas Buchli. 2017. Automatic differentiation of rigid body dynamics for optimal control and estimation. Advanced Robotics 31, 22 (2017), 1225--1237.
21. Radek Grzeszczuk and Demetri Terzopoulos. 1995. Automated learning of muscle-actuated locomotion through control abstraction. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. 63--70.
22. David Ha. 2019. Reinforcement learning for improving agent design. Artificial life 25, 4 (2019), 352--365.
23. Sehoon Ha, Stelian Coros, Alexander Alspach, James M Bern, Joohyung Kim, and Katsu Yamane. 2018. Computational design of robotic devices from high-level motion specifications. IEEE Transactions on Robotics 34, 5 (2018), 1240--1251.
24. Sehoon Ha, Stelian Coros, Alexander Alspach, Joohyung Kim, and Katsu Yamane. 2017. Joint Optimization of Robot Design and Motion Parameters using the Implicit Function Theorem.. In Robotics: Science and systems, Vol. 8.
25. David Hahn, Pol Banzet, James M Bern, and Stelian Coros. 2019. Real2sim: Visco-elastic parameter estimation from dynamic motion. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--13.
26. Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. 2003. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary computation 11, 1 (2003), 1--18.
27. Chris Hecker, Bernd Raabe, Ryan W Enslow, John DeWeese, Jordan Maynard, and Kees van Prooijen. 2008. Real-time motion retargeting to highly varied user-created morphologies. ACM Transactions on Graphics (TOG) 27, 3 (2008), 1--11.
28. Philipp Holl, Nils Thuerey, and Vladlen Koltun. 2020. Learning to Control PDEs with Differentiable Physics. In International Conference on Learning Representations.
29. Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Frédo Durand. 2020. DiffTaichi: Differentiable Programming for Physical Simulation. International Conference on Learning Representations (2020).
30. Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B Tenenbaum, William T Freeman, Jiajun Wu, Daniela Rus, and Wojciech Matusik. 2019. ChainQueen: A real-time differentiable physical simulator for soft robotics. In 2019 International conference on robotics and automation (ICRA). IEEE, 6265--6271.
31. Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B. Tenenbaum, and Chuang Gan. 2021. PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics. In International Conference on Learning Representations.
32. Robert K Katzschmann, Cosimo Della Santina, Yasunori Toshimitsu, Antonio Bicchi, and Daniela Rus. 2019a. Dynamic motion control of multi-segment soft robots using piecewise constant curvature matched with an augmented rigid body model. In 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 454--461.
33. Robert K Katzschmann, Joseph DelPreto, Robert MacCurdy, and Daniela Rus. 2018. Exploration of underwater life with an acoustically controlled soft robotic fish. Science Robotics 3, 16 (2018).
34. Robert K Katzschmann, Maxime Thieffry, Olivier Goury, Alexandre Kruszewski, Thierry-Marie Guerra, Christian Duriez, and Daniela Rus. 2019b. Dynamically closed-loop controlled soft robotic arm using a reduced order finite element model with state observer. In 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 717--724.
35. Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations.
36. John P Lewis, Ken Anjyo, Taehyun Rhee, Mengjie Zhang, Frederic H Pighin, and Zhigang Deng. 2014. Practice and Theory of Blendshape Facial Models. Eurographics (State of the Art Reports) 1, 8 (2014), 2.
37. Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua Tenenbaum, and Antonio Torralba. 2019. Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids. In International Conference on Learning Representations.
38. Junbang Liang, Ming Lin, and Vladlen Koltun. 2019. Differentiable Cloth Simulation for Inverse Problems. In Advances in Neural Information Processing Systems. 772--781.
39. Michael James Lighthill. 1971. Large-amplitude elongated-body theory of fish locomotion. Proceedings of the Royal Society of London. Series B. Biological Sciences 179, 1055 (1971), 125--138.
40. Pingchuan Ma, Yunsheng Tian, Zherong Pan, Bo Ren, and Dinesh Manocha. 2018. Fluid directed rigid body control using deep reinforcement learning. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--11.
41. Andrew D Marchese, Cagdas D Onal, and Daniela Rus. 2014. Autonomous soft robotic fish capable of escape maneuvers using fluidic elastomer actuators. Soft robotics 1, 1 (2014), 75--87.
42. Andrew D Marchese, Russ Tedrake, and Daniela Rus. 2016. Dynamics and trajectory optimization for a soft spatial fluidic elastomer manipulator. The International Journal of Robotics Research 35, 8 (2016), 1000--1019.
43. Antoine McNamara, Adrien Treuille, Zoran Popović, and Jos Stam. 2004. Fluid control using the adjoint method. ACM Transactions On Graphics (TOG) 23, 3 (2004), 449--456.
44. Vittorio Megaro, Bernhard Thomaszewski, Maurizio Nitti, Otmar Hilliges, Markus Gross, and Stelian Coros. 2015. Interactive design of 3d-printable robotic creatures. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1--9.
45. Sehee Min, Jungdam Won, Seunghwan Lee, Jungnam Park, and Jehee Lee. 2019. SoftCon: simulation and control of soft-bodied animals with biomimetic actuators. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--12.
46. Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy J Mitra, and Leonidas J Guibas. 2019. StructureNet: hierarchical graph networks for 3D shape generation. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--19.
47. Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. 2012. Functional maps: a flexible representation of maps between shapes. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--11.
48. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Love-grove. 2019. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 165--174.
49. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32 (2019), 8026--8037.
50. Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei A Efros. 2019. Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity. Advances in Neural Information Processing Systems 32 (2019), 2295--2305.
51. Jovan Popović, Steven M Seitz, and Michael Erdmann. 2003. Motion sketching for control of rigid-body simulations. ACM Transactions on Graphics (TOG) 22, 4 (2003), 1034--1054.
52. Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. 2020. Scalable Differentiable Physics for Learning and Control. In International Conference on Machine Learning.
53. Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 2000. The earth mover's distance as a metric for image retrieval. International journal of computer vision 40, 2 (2000), 99--121.
54. Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. 2020. Learning to Simulate Complex Physics with Graph Networks. In International Conference on Machine Learning.
55. Charles Schaff, David Yunis, Ayan Chakrabarti, and Matthew R Walter. 2019. Jointly learning to construct and control agents using deep reinforcement learning. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 9798--9805.
56. Connor Schenck and Dieter Fox. 2018. SPNets: Differentiable Fluid Dynamics for Deep Neural Networks. Conference on Robot Learning (CoRL) (2018).
57. Christophe Schlick. 1994. Fast Alternatives to Perlin's Bias and Gain Functions. Graphics Gems 4 (1994), 401--404.
58. Adriana Schulz, Ariel Shamir, Ilya Baran, David IW Levin, Pitchaya Sitthi-Amorn, and Wojciech Matusik. 2017a. Retrieval on parametric shape collections. ACM Transactions on Graphics (TOG) 36, 1 (2017), 1--14.
59. Adriana Schulz, Cynthia Sung, Andrew Spielberg, Wei Zhao, Robin Cheng, Eitan Grinspun, Daniela Rus, and Wojciech Matusik. 2017b. Interactive robogami: An end-to-end system for design of robots with ground locomotion. The International Journal of Robotics Research 36, 10 (2017), 1131--1147.
60. Michael Sfakiotakis, David M Lane, and J Bruce C Davies. 1999. Review of fish swimming modes for aquatic locomotion. IEEE Journal of oceanic engineering 24, 2 (1999), 237--252.
61. Karl Sims. 1994. Evolving virtual creatures. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques. 15--22.
62. Justin Solomon, Fernando De Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, and Leonidas Guibas. 2015. Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1--11.
63. Andrew Spielberg, Brandon Araki, Cynthia Sung, Russ Tedrake, and Daniela Rus. 2017. Functional co-optimization of articulated robots. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5035--5042.
64. Andrew Spielberg, Allan Zhao, Yuanming Hu, Tao Du, Wojciech Matusik, and Daniela Rus. 2019. Learning-in-the-loop optimization: End-to-end control and co-design of soft robots through learned deep latent representations. Advances in Neural Information Processing Systems 32 (2019), 8284--8294.
65. Jie Tan, Yuting Gu, Greg Turk, and C Karen Liu. 2011. Articulated swimming creatures. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1--12.
66. Maxime Thieffry, Alexandre Kruszewski, Christian Duriez, and Thierry-Marie Guerra. 2018. Control Design for Soft Robots Based on Reduced-Order Model. IEEE Robotics and Automation Letters 4, 1 (2018), 25--32.
67. Michael S Triantafyllou and George S Triantafyllou. 1995. An efficient swimming machine. Scientific american 272, 3 (1995), 64--70.
68. Merel Van Diepen and Kristina Shea. 2019. A spatial grammar method for the computational design synthesis of virtual soft locomotion robots. Journal of Mechanical Design 141, 10 (2019).
69. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6000--6010.
70. Kevin Wampler and Zoran Popović. 2009. Optimal gait and form for animal locomotion. ACM Transactions on Graphics (TOG) 28, 3 (2009), 1--8.
71. Tingwu Wang, Yuhao Zhou, Sanja Fidler, and Jimmy Ba. 2018. Neural Graph Evolution: Towards Efficient Automatic Robot Design. In International Conference on Learning Representations.
72. Jungdam Won and Jehee Lee. 2019. Learning body shape variation in physics-based characters. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--12.
73. Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. 2019. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4541--4550.
74. Allan Zhao, Jie Xu, Mina Konaković-Luković, Josephine Hughes, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. 2020. RoboGrammar: graph grammar for terrain-optimized robot design. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--16.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

      • Physical simulation

    • Shape modeling

      • Volumetric models

  • Modeling and simulation