skip to main content
research-article

Applying dynamic reconfiguration in the mobile robotics domain: A case study on computer vision algorithms

Published:22 August 2011Publication History
Skip Abstract Section

Abstract

Mobile robots are widely used in industrial environments and are expected to be widely available in human environments in the near future, for example, in the area of care and service robots. This article proposes an implementation for a highly customizable color recognition module based on Field Programmable Gate Array (FPGA) hardware to accomplish tasks like real-time frame processing for image streams. In comparison to a pure software solution on a CPU, an attached FPGA-based hardware accelerator enables real-time image processing and significantly reduces the required computing power of the CPU. Instead, the CPU can be used for tasks that cannot be efficiently implemented on FPGAs, for example, because of a large control overhead. We concentrate on a multirobot scenario where a group of robots follows a human team member by keeping a specific formation in order to support the human in exploration and object detection. Additionally, the robots provide a communication infrastructure to maintain a stable multihop communication network between the human and a base station recording all actions and evaluating the captured images and transmitted data. Depending on the current operating conditions, the robot system has to be able to execute a wide variety of different tasks. Since only a small number of tasks have to be executed concurrently, dynamic reconfiguration of the FPGA can be used to avoid the parallel implementation of all tasks on the FPGA. Within this context, this article discusses application fields where dynamic reconfiguration of FPGA-based coprocessors significantly reduces the CPU load and presents examples of how dynamic reconfiguration can be used in exploration.

References

  1. Ali, M. I. A. H., Sitte, J., and Witkowski, U. 2007. Parallel early vision algorithms for mobile robots. In Proceedings of the 4th International Symposium on Autonomous Mini-Robots for Research and Edutainment (AMiRE). 133--140.Google ScholarGoogle Scholar
  2. Athanas, P. M. and Abbott, A. L. 1994. Image processing on a custom computing platform. In Proceedings of the 4th International Workshop on Field-Programmable Logic and Applications (FPL'94). Springer, 156--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Athanas, P. M. and Abbott, A. L. 1995. Real-Time image processing on a custom computing platform. Comput. 28, 2, 16--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Berthelot, F., Nouvel, F., and Houzet, D. 2006. Partial and dynamic reconfiguration of FPGAs: A top down design methodology for an automatic implementation. In Proceedings of the 20th International Parallel and Distributed Processing Symposium (IPDPS'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Braitenberg, V. 1984. Vehicles: Experiments in Synthetic Psychology. MIT Press.Google ScholarGoogle Scholar
  6. Braun, L., Göhringer, D., Perschke, T., Schatz, V., Hübner, M., and Becker, J. 2009. Adaptive real-time image processing exploiting two dimensional reconfigurable architecture. J. Real-Time Image Process. 4, 109--125.Google ScholarGoogle ScholarCross RefCross Ref
  7. Bräunl, T. 2006. Embedded Robotics: Mobile Robot Design and Applications with Embedded Systems. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Brodley, C. E. and Utgoff, P. E. 1995. Multivariate decision trees. Mach. Learn. 19, 1, 45--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Brown, T. A. and Koplowitz, J. 1979. The weighted nearest neighbor rule for class dependent sample sizes. IEEE Trans. Inf. Theory 25, 5, 617--619.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Cervera, E., Witkowski, U., Gancet, J., Gazi, V., Penders, J., Marques, L., Bureau, P., and Guzman, R. 2007. GUARDIANS: Robot assistants in hazardous emergencies. In Proceedings of the Workshop (Robotics in Challenging and Hazardous Environments) at IEEE International Conference on Robotics (ICRA07).Google ScholarGoogle Scholar
  11. Chinapirom, T., Kaulmann, T., Witkowski, U., and Rueckert, U. 2004. Visual object recognition by 2D-color camera and on-board information processing for minirobots. FIRA Robot World Congress.Google ScholarGoogle Scholar
  12. Claus, C., Ahmed, R., Altenried, F., and Stechele, W. 2010. Towards rapid dynamic partial reconfiguration in video-based driver assistance systems. In Proceedings of the 6th International Symposium on Applied Reconfigurable Computing (ARC'10). Lecture Notes in Computer Science. Springer, 55--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Commuri, S., Tadigotla, V., and Sliger, L. 2007. Task-Based hardware reconfiguration in mobile robots using FPGAs. J. Intell. Robot. Syst. 49, 2, 111--134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Cox, B. and Krichmar, J. 2009. Neuromodulation as a robot controller. Robot. Autom. Mag. 16, 3, 72--80.Google ScholarGoogle ScholarCross RefCross Ref
  15. Dittmann, F. and Frank, S. 2007. Hard real-time reconfiguration port scheduling. In Proceedings of the Conference on Design, Automation and Test in Europe (DATE'07). 123--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Dye, D. 2010. Partial reconfiguration of virtex FPGAs in ISE 12. Xilinx white paper WP374.Google ScholarGoogle Scholar
  17. Ebied, H. M., Witkowski, U., Rueckert, U., and Abdel-Wahab, M. S. 2008. Robot localization based on visual landmarks. In Proceedings of the 5th IEEE International Conference on Informatics in Control, Automation and Robotics (ICINCO). J. Filipe, J. Andrade-Cetto, and J.-L. Ferrier, Eds., INSTICC Press, 49--53.Google ScholarGoogle Scholar
  18. Fife, W. S. and Archibald, J. K. 2007. Reconfigurable on-board vision processing for small autonomous vehicles. EURASIP J. Embed. Syst. 1, 33--33. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Griese, B., Oberthür, S., and Porrmann, M. 2005. Component case study of a self-optimizing RCOS/RTOS system: A reconfigurable network service. In Proceedings of the International Embedded Systems Symposium (IESS). From Specification to Embedded Systems Application, A. Rettberg, M. C. Zanella, and F. J. Rammig, Eds. 267--277.Google ScholarGoogle Scholar
  20. Hagemeyer, J., Kettelhoit, B., Koester, M., and Porrmann, M. 2007. A design methodology for communication infrastructures on partially reconfigurable FPGAs. In Proceedings of the 17th International Conference on Field Programmable Logic and Applications (FPL). 331--338.Google ScholarGoogle Scholar
  21. Hinaga, S., Yamaguchi, Y., Yao, T., and Kawabe, T. 2009. Dynamic reconfiguration system for real-time video processing. In Proceedings of the 19th International Conference on Field Programmable Logic and Applications, (FPL'09). 691--694.Google ScholarGoogle Scholar
  22. Horn, B. K. P. and Schunck, B. G. 1981. Determining optical flow. Art. Intell. 17, 185--203.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Hsiung, P.-A., Santambrogio, M. D., and Huang, C.-H. 2009. Reconfigurable System Design and Verification. CRC Press, Boca Raton, FL. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Bruce, J., and Tucker, B. M. V. 2000. Fast and inexpensive color image segmentation for interactive robots. In Proceedings of IROS-2000. Vol. 3. 2061--2066.Google ScholarGoogle Scholar
  25. Johnston, C. T., Lyons, P., and Bailey, D. G. 2009. User evaluation and overview of a visual language for real time image processing on FPGAs. In Proceedings of the 10th International Conference NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction (CHINZ'09). ACM, New York, 85--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Kaiser, I., Kaulmann, T., Gausemeier, J., and Witkowski, U. 2007. Miniaturization of autonomous robots by the new technology molded interconnected devices (MID). In Proceedings of the 4th International Symposium on Autonomous Minirobots for Research and Edutainment (AMiRE'07). 107--114.Google ScholarGoogle Scholar
  27. Khan, J., Handa, M., and Vemuri, R. 2002. ipace-v1: A portable adaptive computing engine for real time applications. In Proceedings of the Reconfigurable Computing Is Going Mainstream, 12th International Conference on Field-Programmable Logic and Applications (FPL'02). Springer, 69--78. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Kim, J.-H., Jeong, I.-B., Park, I.-W., and Lee, K.-H. 2009. Multi-Layer architecture of ubiquitous robot system for integrated services. Int. J. Social Robot. 1, 1, 19--28.Google ScholarGoogle ScholarCross RefCross Ref
  29. Koester, M., Luk, W., Hagemeyer, J., and Porrmann, M. 2009. Design optimizations to improve placeability of partial reconfiguration modules. In Proceedings of the Conference on Design, Automation and Test in Europe (DATE'09). 976--981. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Koester, M., Luk, W., Hagemeyer, J., Porrmann, M., and Ruckert, U. 2010. Design optimizations for tiled partially reconfigurable systems. IEEE Trans. VLSI Syst. 19, 6, 1048--1061. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Manet, P., Maufroid, D., Tosi, L., Gailliard, G., Mulertt, O., Di Ciano, M., Legat, J.-D., Aulagnier, D., Gamrat, C., Liberati, R., La Barba, V., Cuvelier, P., Rousseau, B., and Gelineau, P. 2008. An evaluation of dynamic partial reconfiguration for signal and image processing in professional electronics applications. EURASIP J. Embed. Syst. 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mitchell, H. B. 2007. Multi-Sensor Data Fusion: An Introduction. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Moctezuma Eugenio, J. C. and Arias Estrada, M. 2009. Hardware/Software FPGA architecture for robotics applications. In Proceedings of the 5th International Workshop on Reconfigurable Computing: Architectures, Tools and Applications (ARC'09). Springer, 27--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Sedcole, P., Blodget, B., Becker, T., Anderson, J., and Lysaght, P. 2006. Modular dynamic reconfiguration in virtex FPGAs. IEE Proc. Comput. Digital Techn. 153, 3, 157--164.Google ScholarGoogle ScholarCross RefCross Ref
  35. Sinha, A., Chen, H., Danu, D. G., Kirubarajan, T., and Farooq, M. 2008. Estimation and decision fusion: A survey. Neurocomput. 71, 13-15, 2650--2656. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Sironi, F., Triverio, M., Hoffmann, H., Maggio, M., and Santambrogio, M. D. 2010. Self-Aware adaptation in FPGA-based systems. In Proceedings of the 20th International Conference on Field Programmable Logic and Applications (FPL'10). 187--192. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Witkowski, U., El Habbal, M. A. M., Herbrechtsmeier, S., Tanoto, A., Penders, J., Alboul, L., and Gazi, V. 2008. Ad-Hoc network communication infrastructure for multi-robot systems in disaster scenarios. In Proceedings of IARP/EURON Workshop on Robotics for Risky Interventions and Environmental Surveillance (RISE'08).Google ScholarGoogle Scholar

Index Terms

  1. Applying dynamic reconfiguration in the mobile robotics domain: A case study on computer vision algorithms

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Reconfigurable Technology and Systems
      ACM Transactions on Reconfigurable Technology and Systems  Volume 4, Issue 3
      August 2011
      204 pages
      ISSN:1936-7406
      EISSN:1936-7414
      DOI:10.1145/2000832
      Issue’s Table of Contents

      Copyright © 2011 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 22 August 2011
      • Accepted: 1 November 2010
      • Revised: 1 October 2010
      • Received: 1 November 2009
      Published in trets Volume 4, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!