Implementation of artificial neural networks on a reconfigurable hardware accelerator

ABSTRACT

The hardware implementation of three different artificial neural networks is presented. The basis for the implementation is the reconfigurable hardware accelerator RAPTOR2000, which is based on FPGAs. The investigated neural network architectures are neural associative memories, self-organizing feature maps and basis function networks. Some of the key implementational issues are considered. Especially resource-efficiency and performance of the presented realizations are discussed.

References

1. Pintaske, C. System- und Schaltungstechnik neuronaler Assoziativspeicher. PhD thesis, University of Paderborn, Heinz Nixdorf Institut, HNI-Verlagsschriftenreiche, no 29, 1998.
2. Rückert, U.: "ULSI Implementations for Artificial Neural Networks", 9th Euromicro Workshop on Parallel and Distributed Processing, Feb. 2001, Mantova, Italien, pp. 436-442.
3. Kalte, H., Porrmann, M., Rückert, U.: "Rapid Prototyping System für dynamisch rekonfigurierbare Hardwarestrukturen", AES2000, Karlsruhe, 18.-19. Jan. 2000, pp. 150-157.
4. H. Kalte, M. Porrmann, U. Rückert, "Using a Dynamically Reconfigurable System to Accelerate Octree Based 3D Graphics", Int. Conf. on Parallel and Distributed Processing Techniques and Applications (PDPTA), June 26-29, 2000, Las Vegas, Nevada, USA, pp. 2819-2824.
5. Palm G., Schwenker F., Sommer F.T., Strey A. Neural Associative Memories. in Associative Processing and Processors. Krikelis A. Weems C.C. IEEE Computer Society, pp 307- 326, Press Los Alamitos, 1997
6. Rückert U., Kreuzer I., Goser K. A VLSI Concept for an Associative Matrix Based on Neural Networks. in Proebster W.E, Reiner H. VLSI and Computer. Washington: Computer Society Press, pp 31-34, 1987.
7. Kohonen T. Self-Organizing Maps. Springer, Berlin, 1995.
8. Porrmann M., Heittmann A., Rüping S., Rückert U. A Hybrid Knowledge Processing System. NEURAP 98, Marseilles, France, pp 177-184, 11-13 March 1998.
9. Rüping S., Porrmann M., Rückert U. SOM Hardware-Accelerator. WSOM'97: Workshop on Self-Organizing Maps. June 4-6, pp 136-141, Espoo, Finnland, 1997.
10. Rüping S., Porrmann M., Rückert U. A High Performance SOFM Hardware-System. IWANN'97: International Work-Conference on Artificial and Natural Neural Networks, June 4-6, pp 772-781, Lanzarote, Spain, 1997.
11. Rüping S., Goser K., Rückert U. A Chip for Selforganizing Feature Maps. IEEE MICRO, vol. 15, No. 3, pp 57-59, June 1995.
12. Rüping S., Rückert U. A Scalable Processor Array for Self-Organizing Feature Maps. Fifth Int. Conf. on Microelectronics for Neural Networks and Fuzzy Systems, MicroNeuro'96, February 12-14, pp 285-291, Lausanne, Switzerland, 1996.
13. Porrmann, M., Rüping, S., Rückert, U., "SOM Hardware with Acceleration Module for Graphical Representation of the Learning Process", Proceedings of the Seventh Int. Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems, pp. 380-386, Granada, April 1999.
14. Rüping S. VLSI-gerechte Umsetzung von selbstorganisierenden Karten und ihre Einbindung in ein Neuro-Fuzzy Analysesystem. Fortschritt-Berichte VDI, Reihe 9: Elektronik, Düsseldorf: VDI Verlag, 1995.
15. Rüping, S., Porrmann, M., Rückert, U., "SOM Accelerator System", Neurocomputing 21, pp. 31-50, 1998.
16. Porrmann, M., Rüping, S. and Rückert, U., "The Impact of Communication on Hardware Accelerators for Neural Networks", Proceedings of The 5th World Multi-Conference on Systemics, Cybernetics and Informatics, SCI 2001, Volume 3, pages 248-253, Orlando, Florida, USA, July 2001.
17. Porrmann, M., Kalte, H., Witkowski, U., Niemann, J.-C. and Rückert, U., "A Dynamically Reconfigurable Hardware Accelerator for Self-Organizing Feature Maps", 5th World Multi-Conf. on Systemics, Cybernetics and Informatics, SCI, Volume 3, pages 242-247, Orlando, Florida, USA, July 2001.
18. Kohonen T. Associative Memory: A system-theoretical approach. Springer, Berlin, 1977.
19. Willshaw D.J., Buneman O.P., Longuet-Higgins H.C. A Non-Holographic Model of Associative Memory. Nature 222, no. 5197, pp. 960-962, 1969.
20. Palm G. On Associative Memory. Biol. Cybern. 36, pp 19-31, 1980.
21. Schmidt, M., Rückert, U.: "Content-Based Information Retrieval using an embedded Neural Associative Memory", 9th Euromicro Workshop on Parallel and Distributed Processing 2001, Mantova, Italien, 07.-09. Feb. 2001, S. 443-450.
22. Rückert U. VLSI Design of an Associative Memory based on Distributed Storage of Information. in Ramacher U., Rückert U. VLSI Design of Neural Networks. Bosten Kluwer Academic Publ., pp 153-168, 1991.
23. Rückert U., Funke A., Pintaske C. Acceleratorboard for Neural Associative Memories. Neurocomputing 5, Elsevier, pp 39-49, 1993.
24. Rückert U. An Associative Memory with Neural Architecture and its VLSI Implementation. HICSS-24, Hawaii, Jan. 1990.
25. Heittmann A., Malin J., Pintaske C., Rückert U. Digital VLSI Implementation of a Neural Associative Memory. 6th Internat. Conf. on Microelectronics for Neural Networks, Evolutionary & Fuzzy Systems, pp 280-285, Dresden, Sep. 1997.
26. Poggio T., Girosi F. "A Theory of Networks for Approximation and Learning", AI Memo 1140/CBIP Paper 31, Massachusetts Institute of Technology, Cambridge, MA, July 1989.
27. Beineke S., Schütte F., Grotstollen H., "Comparison of Methods for State Estimation and On-line Identification in Speed and Position Control Loops", 7th Europ. Conf. on Power Electronics and Applications, Trondheim, Norway, vol. 3, pp. 364-369, Sept. 1997.
28. Krüger L., Naunin D., Hidiroglu A., Neumerkel D., Franz J., "Neural Model Predective Control of an Induction Motor", Proc. of the Conf. on Power Electronics and Applications, Sevilla, Spain, vol. 1, pp. 409-414, 1995.
29. Fogelman-Soulie F, "Applications of Neural Networks", The Handbook of Brain Theory and Neural Networks, M.A. Arbib (ed.), Bradford Books / MIT Press, pp. 94-98, 1995.
30. Lowe D., "Radial Basis Function Networks", The Handbook of Brain Theory and Neural Networks, M.A. Arbib (ed.), Bradford Books / MIT Press, pp. 779-782, 1995.
31. Katayama R. et Al., "Performance Evaluation of Self Generating Radial Basis Function for Function Approximation", Proceedings of 1993 International Joint Conference on Neural Networks, pp. 471-474, 1993.
32. Hammerstrom D. A highly parallel digital architecture for neural network emulation. In Delgado-Frias J.G., Moore W.R. VLSI for Artificial Intelligence and Neural Networks. chapter 5.1, pp 357-366. Plenum press, New York, 1991.
33. Ramacher U. SYNAPSE - A neurocomputer that synthesizes neural algorithms on a parallel systolic engine. Journal of Parallel and Distributed Computing, 14(3), pp 306-318, 1992

Index Terms

• Computer systems organization

  • Architectures

    • Other architectures

      • Neural networks

• Computing methodologies

  • Machine learning

    • Machine learning approaches

      • Neural networks

• Hardware

  • Emerging technologies

    • Circuit substrates

      • Cellular neural networks

• Theory of computation

  • Theory and algorithms for application domains

    • Machine learning theory