skip to main content
research-article

Human perception of haptic-to-video and haptic-to-audio skew in multimedia applications

Published:10 May 2013Publication History
Skip Abstract Section

Abstract

The purpose of this research is to assess the sensitivity of humans to perceive asynchrony among media signals coming from a computer application. Particularly we examine haptic-to-video and haptic-to-audio skew. For this purpose we have designed an experimental setup, where users are exposed to a basic multimedia presentation resembling a ping-pong game. For every collision between a ball and a racket, the user is able to perceive auditory, visual, and haptic cues about the collision event. We artificially introduce negative and positive delay to the auditory and visual cues with respect to the haptic stream. We subjectively evaluate the perception of inter-stream asynchrony perceived by the users using two types of haptic devices. The statistical results of our evaluation show perception rates of around 100 ms regardless of modality and type of device.

References

  1. Adelstein, B. D., Begault, D. R., Anderson, M. R., and Wenzel, E. M. 2003. Sensitivity to haptic-audio asynchrony. In Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI'03). ACM Press, New York, 73--76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Adams, R. and Hannaford, B. 1998. A two port framework for the design of unconditionally stable haptic interfaces. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 1254--1259.Google ScholarGoogle Scholar
  3. Adams, R. J., Moreyra, M. R., and Hannaford, B. 1998. Stability and performance of haptic displays: Theory and experiments. In Proceedings of the ASME International. Mechanical Engineering Congress and Exhibition. 227--234.Google ScholarGoogle Scholar
  4. Allan, L. G. and Kristofferson, A. B. 1974. Successiveness discrimination: Two models. Percept. Psychophys. 15, 1, 37--46.Google ScholarGoogle ScholarCross RefCross Ref
  5. Colgate, J. E., Stanley, M. C., and Brown, J. M. 1995. Issues in haptic display of tool use. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 140--145. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Conti, F., Barbagli, F., Morris, D., and Sewell, C. 2005.CHAI 3D: An open-source library for the rapid development of haptic scenes. http://www.chai3d.org/project.html.Google ScholarGoogle Scholar
  7. Diolaiti, N., Niemeyer, G., Barbagli, F., and Salisbury, J. K. 2006. Stability of haptic rendering: Discretization, quantization, time delay and coulomb effects. IEEE Trans. Robot. 22, 2, 256--268. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Dixon, N. F. and Spitz, L. 1980. The detection of auditory visual desynchrony. Percept. 9, 6.Google ScholarGoogle ScholarCross RefCross Ref
  9. Dixon, W. J. and Mood, A. M. 1948. A method for obtaining and analyzing sensitivity data. J. Amer. Statist. Assoc. 43, 241, 109--126.Google ScholarGoogle ScholarCross RefCross Ref
  10. El Saddik, A. 2007. The potential of haptics technologies. IEEE Instrument. Measur. 10, 1, 10--17.Google ScholarGoogle ScholarCross RefCross Ref
  11. Eid, M., Orozco, M., and El Saddik, A. 2007. A guided tour in haptic audio visual environment and applications. Int. J. Advanced Media Comm. 1, 3, 265--297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Fujimoto, T., Ishibashi, Y., and Sugawara, S. 2008. Influences of inter-stream synchronization error on collaborative work in haptic and visual environments. In Proceedings of the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS'08). 113--119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Ishibashi, Y., Kanbara, T., and Tasaka, S. 2004. Inter-Stream synchronization between haptic media and voice in collaborative virtual environments. In Proceedings of the 12th Annual ACM International Conference on Multimedia (MULTIMEDIA'04). 604--611. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hinterseer, P., Hirche, S., Chaudhuri, S., Steinbach, E., and Buss, M. 2007. Perception-Based data reduction and transmission of haptic data in telepresence and teleaction systems. IEEE Trans. Signal Process. 55, 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jay, C., Glencross, M., and Hubbold, R. 2007. Modeling the effects of delayed haptic and visual feedback in a collaborative virtual environment. ACM Trans. Comput.-Hum. Interact. 14, 2, 8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Kameyama, S. and Ishibashi, Y. 2006. Perception of synchronization errors in haptic and visual communications. Proc. SPIE Optics East Multimedia Syst. Appl. IX 6391, 8.Google ScholarGoogle Scholar
  17. Levitt, H. 1970. Transformed up-down methods in psychoacoustics. J. Acoust. Soc. Amer. 49, 2, 467--477.Google ScholarGoogle ScholarCross RefCross Ref
  18. Levitin, D. J., Maclean, K., Mathews, M., Chu L., and Jensen, E. 2000.The perception of cross-modal simultaneity. AIP Conf. Proc. Comput. Anticipatory Syst. 517, 1, 323--329.Google ScholarGoogle ScholarCross RefCross Ref
  19. Mcgrath, M. and Summerfield, Q. 1985. Intermodal timing relations and audio-visual speech recognition by normalhearing adults. J. Acoust. Soc. Amer. 77, 2, 678--685.Google ScholarGoogle ScholarCross RefCross Ref
  20. Miyasato, T., Noma, H., and Kishino, F. 1996. Subjective evaluation of perception of delay time between visual information and tactile information. IEICE Trans. Fundam. Electron. Comm. Comput. Sci. E79-A, 5, 655--657.Google ScholarGoogle Scholar
  21. Novint Falcon Hapticdevice. Novint technologieses. http://home.novint.com/products/novint_falcon.php.Google ScholarGoogle Scholar
  22. Phantomhapticdevice. Sensable technologies. http://www.sensable.com/index.htm.Google ScholarGoogle Scholar
  23. Steinmetz, R. 1996. Human perception of jitter and media synchronization. IEEE J. Select. Areas Comm. 14, 1, 61--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Tatematsu, A., Ishibashi, Y., Fukushima, N., and Sugawara, S. 2010. QoE assessment in haptic media, sound and video transmission: Influences of network latency. In Proceedings of the IEEE International Communications Quality and Reliability Workshop (CQR'10).Google ScholarGoogle Scholar
  25. Vogels, I. M. L. C. 2004. Detection of temporal delays in visual-haptic interfaces. Hum. Factors 46, 1, 118--134.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Human perception of haptic-to-video and haptic-to-audio skew in multimedia applications

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 9, Issue 2
            May 2013
            144 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/2457450
            Issue’s Table of Contents

            Copyright © 2013 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 10 May 2013
            • Accepted: 1 June 2012
            • Revised: 1 April 2012
            • Received: 1 April 2011
            Published in tomm Volume 9, Issue 2

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!