Crossing the reality gap in evolutionary robotics by promoting transferable controllers

ABSTRACT

The reality gap, that often makes controllers evolved in simulation inefficient once transferred onto the real system, remains a critical issue in Evolutionary Robotics (ER); it prevents ER application to real-world problems. We hypothesize that this gap mainly stems from a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: best solutions in simulation often rely on bad simulated phenomena (e.g. the most dynamic ones). This hypothesis leads to a multi-objective formulation of ER in which two main objectives are optimized via a Pareto-based Multi-Objective Evolutionary Algorithm: (1) the fitness and (2) the transferability. To evaluate this second objective, a simulation-to-reality disparity value is approximated for each controller. The proposed method is applied to the evolution of walking controllers for a real 8-DOF quadrupedal robot. It successfully finds efficient and well-transferable controllers with only a few experiments in reality.

References

1. A. Boeing and T. Bräunl. Evaluation of real-time physics simulation systems. In Proc. of the 5th international conference on Computer graphics and interactive techniques in Australia and Southeast Asia, pages 281--288. ACM New York, NY, USA, 2007.
2. J. Bongard and H. Lipson. Once more unto the breach: Co-evolving a robot and its simulator. In Proc. of Artificial life IX, page 57. MIT Press, 2004.
3. C. Coello. Constraint-handling using an evolutionary multiobjective optimization technique. Civil Engineering and Environmental Systems, 17(4):319--346, 2000.
4. K. Deb. Multi-objective optimization using evolutionary algorithms. Wiley, 2001.
5. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182--197, 2002.
6. D. Floreano and F. Mondada. Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks, 11(7-8):1461--1478, 1998.
7. D. Floreano and J. Urzelai. Evolution of plastic control networks. Autonomous Robots, 11(3):311--317, 2001.
8. C. Grand, F. BenAmar, F. Plumet, and P. Bidaud. Decoupled control of posture and trajectory of the hybrid wheel-legged robot Hylos. In IEEE International Conference on Robotics and Automation, volume 5, pages 5111--5116, 2004.
9. S. Hanford, L. Long, and J. Horn. A small semi-autonomous rotary-wing unmanned air vehicle (UAV). In 2003 AIAA Atmospheric Flight Mechanics Conference & Exhibit, 2003.
10. C. Hartland and N. Bredèche. Evolutionary Robotics, Anticipation and the Reality Gap. In Proc. of ROBIO-06, pages 1640--1645, 2006.
11. N. Jakobi. Minimal Simulations for Evolutionary Robotics. PhD thesis, University of Sussex, 1998.
12. N. Jakobi. Running across the reality gap: Octopod locomotion evolved in a minimal simulation. Lecture Notes in Computer Science, 1468:39--58, 1998.
13. S. Koos, J. Mouret, and S. Doncieux. Automatic system identification based on coevolution of models and tests. In Proc. of IEEE CEC, volume 2009, 2009.
14. J. Lehman and K. Stanley. Exploiting open-endedness to solve problems through the search for novelty. In Proc. of ALIFE XI, volume 54, 2008.
15. O. Miglino, H. Lund, and S. Nolfi. Evolving Mobile Robots in Simulated and Real Environments. Artificial Life, 2(4):417--434, 1995.
16. J.-B. Mouret and S. Doncieux. Using behavioral exploration objectives to solve deceptive problems in neuro-evolution. In Proc. of GECCO-09, pages 627--634. ACM, 2009.
17. J.-B. Mouret and S. Doncieux. Sferes v2: Evolvin- in the multicore world. In Proc. of IEEE CEC-2010, 2010.
18. S. Nolfi and D. Floreano. Evolutionary robotics. MIT Press, 2000.
19. M. Palmer, D. Miller, and T. Blackwell. An Evolved Neural Controller for Bipedal Walking: Transitioning from Simulator to Hardware. In Proc. of IROS 2009 Workshop on Exploring new horizons in Evolutionary Design of Robots, 2009.
20. J. Pollack, H. Lipson, S. Ficici, P. Funes, G. Hornby, and R. Watson. Evolutionary techniques in physical robotics. In Evolvable systems: from biology to hardware: third international conference, ICES 2000, Proceedings, pages 175--186. Springer Verlag, 2000.
21. A. Thompson, P. Layzell, and R. Zebulum. Explorations in design space: Unconventional electronics design through artificial evolution. IEEE Transactions on Evolutionary Computation, 3(3):167--196, 1999.
22. J. Zagal and J. Ruiz-del Solar. Combining simulation and reality in evolutionary robotics. Journal of Intelligent and Robotic Systems, 50(1):19--39, 2007.

Index Terms

• Computer systems organization

  • Embedded and cyber-physical systems

    • Robotics

• Computing methodologies

  • Artificial intelligence

    • Control methods

      • Robotic planning

    • Planning and scheduling

      • Robotic planning