skip to main content
research-article

A novel software framework for embedded multiprocessor smart cameras

Published:22 April 2009Publication History
Skip Abstract Section

Abstract

Distributed smart cameras (DSC) are an emerging technology for a broad range of important applications including smart rooms, surveillance, entertainment, tracking, and motion analysis. By having access to many views and through cooperation among the individual cameras, these DSCs have the potential to realize many more complex and challenging applications than single-camera systems.

This article focuses on the system-level software required for efficient streaming applications on single smart cameras as well as on networks of DSCs. Embedded platforms with limited resources do not provide middleware services well known on general-purpose platforms. Our software framework supports transparent intra- and interprocessor communication while keeping the memory and computation overhead very low. The software framework is based on a publisher--subscriber architecture and provides mechanisms for dynamically loading and unloading software components as well as for graceful degradation in case of software- and hardware-related faults. The software framework has been completely implemented and tested on our embedded smart cameras consisting of an ARM-based network processor and several digital signal processors. Two case studies demonstrate the feasibility of our approach.

References

  1. Aghajan, H. and Kleihorst, R., Eds. 2007. Proceedings of the ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC'07). ACM, New York.Google ScholarGoogle Scholar
  2. Akyildiz, I. F., Su, W., Sankarasubramaniam, Y., and Cayirci, E. 2002. Wireless sensor networks: a survey. Comput. Netw. 38, 4, 393--422. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Balasubramanian, K., Wang, N., Gill, C., and Schmidt, D. C. 2003. Towards composable distributed real-time and embedded software. In Proceedings of the 8th IEEE International Workshop on Object-Oriented Real-Time Dependable Systems. IEEE, Los Almitos, CA, 226--233.Google ScholarGoogle Scholar
  4. Becker, C., Schiele, G., Gubbles, H., and Rothermel, K. 2003. BASE—a micro-brokerbased middleware for pervasive computing. In Proceedings of the 1st IEEE International Conference on Pervasive Computing and Communications. IEEE, Los Alamitos, CA, 443--451. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Bramberger, M. 2005. Distributed dynamic task allocation in clusters of embedded smart cameras. Ph.D. thesis, Institute for Technical Informatics, Graz University of Technology, Graz, Austria.Google ScholarGoogle Scholar
  6. Bramberger, M., Brunner, J., Rinner, B., and Schwabach, H. 2004. Real-Time video analysis on an embedded smart camera for traffic surveillance. In Proceedings of the 10th IEEE Real-Time and Embedded Technology and Applications Symposium. IEEE, Los Alamitos, CA, 174--181. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bramberger, M., Doblander, A., Maier, A., Rinner, B., and Schwabach, H. 2006. Distributed smart cameras for surveillance applications. Computer 39, 2, 68--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Bramberger, M., Rinner, B., and Schwabach, H. 2004. An embedded smart Camera on a scalable heterogeneous multi-DSP system. In Proceedings of the European DSP Education and Research Symposium.Google ScholarGoogle Scholar
  9. Bramberger, M., Rinner, B., and Schwabach, H. 2005. A method for dynamic allocation of tasks in clusters of embedded smart cameras. In Proceedings of the International Conference on Systems, Man and Cybernetics. IEEE, Los Alamitos, CA, 2595--2600.Google ScholarGoogle Scholar
  10. Clarke, M., Blair, G. S., Coulson, G., and Parlavantzas, N. 2001. An efficient component model for the construction of adaptive middleware. In Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms, R. Guerraoui, Ed. Lecture Notes in Computer Science. Vol. 2218. Springer, 160--178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. DeMichiel, L. G. 1995. The component object model specification. Tech. rep., Microsoft Corporation.Google ScholarGoogle Scholar
  12. DeMichiel, L. G. 2002. Enterprise JavaBeans specification version 2.1. Tech. rep., SUN Microsystems.Google ScholarGoogle Scholar
  13. Doblander, A., Rinner, B., Trenkwalder, N., and Zoufal, A. 2006a. A light-weight publisher-subscriber middleware for dynamic reconfiguration in networks of embedded smart cameras. In Proceedings of the 5th World Scientific and Engineering Academy and Society International Conference on Software Engineering, Parallel and Distributed Systems. ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Doblander, A., Rinner, B., Trenkwalder, N., and Zoufal, A. 2006b. A middleware framework for dynamic reconfiguration and component composition in embedded smart cmeras. WSEAS Trans. Comput. 5, 3, 574--581.Google ScholarGoogle Scholar
  15. Fraga, J., Siqueira, F., and Favarim, F. 2003. An adaptive fault-tolerant component model. In Proceedings of the 9th IEEE International Workshop on Object-Oriented Real-Time Dependable Systems. IEEE, Los Alamitos, CA, 179--186.Google ScholarGoogle Scholar
  16. Hansson, H., Åkerholm, M., Crnkovic, I., and Törngren, M. 2004. SaveCCM—a component model for safety-critical real-time systems. In Proceedings of the 30th EUROMICRO Conference. IEEE, Los Alamitos, CA, 627--635. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Karsai, G., Sztipanovits, J., Ledeczi, A., and Bapty, T. 2003. Model-integrated development of embedded software. Proc. IEEE 91, 1, 145--164.Google ScholarGoogle ScholarCross RefCross Ref
  18. Lin, C. H., Wolf, W., Dixon, A., Koutsoukos, X., and Sztipanovits, J. 2006. Design and implementation of ubiquitous smart cameras. In Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing. IEEE, Los Alamitos, CA, 32--39. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Maier, A. 2006. Dynamic power-aware camera configuration in distributed embedded surveillance clusters. Ph.D. thesis, Institute for Technical Informatics, Graz University of Technology, Graz, Austria.Google ScholarGoogle Scholar
  20. Maier, A., Rinner, B., and Schwabach, H. 2005. A hierarchical approach for energy-aware distributed embedded intelligent video surveillance. In Proceedings of the IEEE/IFIP International Workshop on Parallel and Distributed Embedded Systems. IEEE, Los Alamitos, CA, 12--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Mascolo, C., Capra, L., and Emmerich, W. 2002. Mobile computing middleware. In Advanced Lectures on Networking: NETWORKING 2002 Tutorials, E. Gregori, G. Anastasi, and S. Basagni, Eds. Lecture Notes in Computer Science, vol. 2497. Springer, Berlin, Germany, 20--52. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Microsoft. 2005. .Net Home Page. http://www.microsoft.com/net.Google ScholarGoogle Scholar
  23. Mody, M. 2006. XDAIS-DM (XDM): A step towards the “plug and play” architecture for multimedia codecs. TI Developer Conference. http://www.s.ti.com/sc/techlit/sprp496.pdf.Google ScholarGoogle Scholar
  24. Molla, M. M. and Ahamed, S. I. 2006. A survey of middleware for sensor Networks and Challenges. In Proceedings of the International Conference on Parallel Processing Workshops (ICPPW'06). IEEE, Los Alamitos, CA, 223--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Object Management Group. 2001. Real-Time CORBA 2.0. http://www.omg.org.Google ScholarGoogle Scholar
  26. Object Management Group. 2002. Minimum CORBA 1.0. http://www.omg.org.Google ScholarGoogle Scholar
  27. Object Management Group. 2005. http://www.omg.org/technology/documents/formal/components. htm.Google ScholarGoogle Scholar
  28. Pitt, E. and McNiff, K. 2001. Java.rmi: The Remote Method Invocation Guide. Addison Wesley, Upper Saddle River, NJ. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Pope, A. 1998. The CORBA Reference Guide: Understanding the Common Oject Request Broker Architecture. Addison Wesley, Upper Saddle River, NJ. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Rinner, B., Jovanovic, M., and Quaritsch, M. 2007. Embedded middleware on distributed smart cameras. In Proceedings of the IEEE International Conference on Acoustics, Speech, Signal Processing (ICASSP'07). IEEE, Los Alamitos, CA, 1381--1384.Google ScholarGoogle Scholar
  31. Rinner, B., Schriebl, W., Winkler, T., Quaritsch, M., and Wolf, W. 2008. The evolution from single to pervasive smart sameras. In Proceedings of the ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC'08). ACM, New York.Google ScholarGoogle Scholar
  32. Rinner, B. and Wolf, W. 2008a. A bright future for distributed smart cameras (guest editor's introduction). Proc. IEEE 96, 10, 1562--1564.Google ScholarGoogle ScholarCross RefCross Ref
  33. Rinner, B. and Wolf, W. 2008b. An introduction to distributed smart cameras. Proc. IEEE 96, 10, 1565--1575.Google ScholarGoogle ScholarCross RefCross Ref
  34. Schmidt, D. C. 2002. Middleware for real-time and embedded systems. Comm. ACM 45, 6, 43--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sessions, R. 1997. COM and DCOM: Microsoft's Vision for Distributed Objects. John Wiley & Sons, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Systems, M. C. and Thales. 2003. Light Weight CORBA Component Model. Tech. rep., Object Management Group.Google ScholarGoogle Scholar
  37. Texas Instruments. 2002. TMS320 Algorithm Standard—Rules and Guidelines. Literature Number: SPRU352E.Google ScholarGoogle Scholar
  38. Wolf, W., Ozer, B., and Lv, T. 2002. Smart cameras as embedded systems. Computer 35, 9, 48--53. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A novel software framework for embedded multiprocessor smart cameras

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!