skip to main content
research-article

Distributed Spatiotemporal Gesture Recognition in Sensor Arrays

Published:08 September 2015Publication History
Skip Abstract Section

Abstract

We present algorithms for gesture recognition using in-network processing in distributed sensor arrays embedded within systems such as tactile input devices, sensing skins for robotic applications, and smart walls. We describe three distributed gesture-recognition algorithms that are designed to function on sensor arrays with minimal computational power, limited memory, limited bandwidth, and possibly unreliable communication. These constraints cause storage of gesture templates within the system and distributed consensus algorithms for recognizing gestures to be difficult. Building up on a chain vector encoding algorithm commonly used for gesture recognition on a central computer, we approach this problem by dividing the gesture dataset between nodes such that each node has access to the complete dataset via its neighbors. Nodes share gesture information among each other, then each node tries to identify the gesture. In order to distribute the computational load among all nodes, we also investigate an alternative algorithm, in which each node that detects a motion will apply a recognition algorithm to part of the input gesture, then share its data with all other motion nodes. Next, we show that a hybrid algorithm that distributes both computation and template storage can address trade-offs between memory and computational efficiency.

References

  1. C. Bahlmann, B. Haasdonk, and H. Burkhardt. 2002. Online handwriting recognition with support vector machines-a kernel approach. In Proceedings of the 8th International Workshop on Frontiers in Handwriting Recognition, 2002. IEEE, 49--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. W. Butera. 2002. Programming a Paintable Computer. Ph.D. dissertation. Program in Media Arts and Sciences, School of Architecture and Planning, MIT, Cambridge, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. J. M. Carmona and J. Climent. 2012. A performance evaluation of HMM and DTW for gesture recognition (Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications), L. lvarez, M. Mejail, L. Gmez Dniz, and J. C. Jacobo (Eds.). Springer, New York, NY, 236--243.Google ScholarGoogle Scholar
  4. W. J. Confer and R. O. Chapman. 2004. System and method of handwritten character recognition. (April 2004). Patent No. 6,721,452. Filed September 12, 2002. Issued April 13, 2004.Google ScholarGoogle Scholar
  5. R. H. Davis and J. Lyall. 1986. Recognition of handwritten characters—a review. Image and Vision Computing 4, 4, 208--218. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. M. Elmezain, A. Al-hamadi, and B. Michaelis. 2009. A hidden Markov model-based isolated and meaningful hand gesture recognition. International Journal of Electronical, Computer, and Systems Engineering 3, 3.Google ScholarGoogle Scholar
  7. N. Farrow, N. Sivagnanadasan, and N. Correll. 2014. Gesture based distributed user interaction system for a reconfigurable self-organizing smart wall. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI). ACM, New York, NY, 245--246. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. J. Hawkins. 2002. Graffiti (Palm OS). https://en.wikipedia.org/wiki/Graffiti_(Palm_OS).Google ScholarGoogle Scholar
  9. H. Hosseinmardi, N. Correll, and R. Han. 2012. Bloom filter-based ad hoc multicast communication in cyber-physical systems and computational materials. Wireless Algorithms, Systems, and Applications 595--606.Google ScholarGoogle Scholar
  10. D. Hughes and N. Correll. 2014. A soft, amorphous skin that can sense and localize texture. In IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, 1844--1851. Retrieved July 28, 2015 from http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber==6907101.Google ScholarGoogle Scholar
  11. H. K. Lee and J. H. Kim. 1999. An HMM-based threshold model approach for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 10, 961--973. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. Lifton, D. Seetharam, M. Broxton, and J. Paradiso. 2002. Pushpin computing system overview: A platform for distributed, embedded, ubiquitous sensor networks. Pervasive Computing 139--151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. V. J. Lumelsky, M. S. Shur, and S. Wagner. 2001. Sensitive skin. IEEE Sensors Journal 1, 1, 41--51.Google ScholarGoogle ScholarCross RefCross Ref
  14. S. Ma, H. Hosseinmardi, N. Farrow, R. Han, and N. Correll. 2012. Establishing multi-cast groups in computational robotic materials. In IEEE International Conference on Cyber, Physical and Social Computing. Besancon, France. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. M. A. McEvoy and N. Correll. 2014. Thermoplastic variable stiffness composites with embedded, networked sensing, actuation, and control. Journal of Composite Materials. Retrieved July 28, 2015 from http://jcm.sagepub.com/content/early/2014/03/07/0021998314525982.Google ScholarGoogle Scholar
  16. M. A. McEvoy and N. Correll. 2015. Materials that couple sensing, actuation, computation and communication. Science 347, 6228. Retrieved July 28, 2015 from http://www.sciencemag.org/lookup/doi/10.1126/science.1261689.Google ScholarGoogle ScholarCross RefCross Ref
  17. M. A. McEvoy, N. Farrow, and N. Correll. 2013. Robotic materials with controllable stiffness. In Proceedings of the 19th International Conference on Composite Materials (ICCM).Google ScholarGoogle Scholar
  18. S. Mitra and T. Acharya. 2007. Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37, 3, 311--324. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. K. Murakami and H. Taguchi. 1991. Gesture recognition using recurrent neural networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching through Technology. ACM, 237--242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. J. Nagi, H. Ngo, A. Giusti, Luca M. Gambardella, J. Schmidhuber, and G. A. Di Caro. 2012. Incremental learning using partial feedback for gesture-based human-swarm interaction. In RO-MAN. IEEE, 898--905.Google ScholarGoogle Scholar
  21. Z. Nakad, M. Jones, and T. Martin. 2003. Communications in electronic textile systems. In Proceedings of the 2003 International Conference on Communications in Computing. 37--43.Google ScholarGoogle Scholar
  22. O. F. Ozer, O. Ozun, C. O. Tuzel, V. Atalay, and A. E. Cetin. 2001. Vision-based single-stroke character recognition for wearable computing. Intelligent Systems, IEEE 16, 3, 33--37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. A. Paradiso, J. Lifton, and M. Broxton. 2004. Sensate media multimodal electronic skins as dense sensor networks. BT Technology Journal 22, 4, 32--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. H. Profita, N. Farrow, and N. Correll. 2015. Flutter: An exploration of an assistive garment using distributed sensing, computation and actuation. In Proceedings of the 9th International Conference on Tangible, Embedded, and Embodied Interaction (TEI’15). ACM, New York, NY, 359--362. DOI:http://dx.doi.org/10.1145/2677199.2680586 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. H. Sakoe and S. Chiba. 1990. Dynamic Programming Algorithm Optimization for Spoken Word Recognition. In A. Waibel and K.-F. Lee (Eds.). Readings in Speech Recognition. Morgan Kaufmann Publishers Inc., San Francisco, CA. 159--165. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. T. Schlömer, B. Poppinga, N. Henze, and S. Boll. 2008. Gesture recognition with a Wii controller. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction. ACM, New York, NY, 11--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. A. D. Wilson and A. F. Bobick. 1999. Parametric hidden Markov models for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 9, 884--900. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Y. Wu and T. Huang. 1999. Vision-based gesture recognition: A review. Gesture-based Communication in Human-Computer Interaction, Lecture Notes in Computer Science, Vol. 1739, Springer, Berlin, 103--115. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Distributed Spatiotemporal Gesture Recognition in Sensor Arrays

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Autonomous and Adaptive Systems
      ACM Transactions on Autonomous and Adaptive Systems  Volume 10, Issue 3
      October 2015
      204 pages
      ISSN:1556-4665
      EISSN:1556-4703
      DOI:10.1145/2819320
      Issue’s Table of Contents

      Copyright © 2015 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 8 September 2015
      • Accepted: 1 March 2015
      • Revised: 1 June 2014
      • Received: 1 December 2013
      Published in taas Volume 10, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!