skip to main content
research-article
Open Access

Ultrasound-driven Curveball in Table Tennis: Human Activity Support via Noncontact Remote Object Manipulation

Published:05 November 2021Publication History
Skip Abstract Section

Abstract

Augmented Human (AH) is a research field enhancing human physical abilities or supporting human activity using advanced technologies. As one of the AH approaches, previous studies have attached an actuator to a human body or tools used for an activity. The attached actuators are used to control their movements to support an activity. In this study, instead of attaching actuators, we propose to directly apply noncontact ultrasound force to a lightweight tool to manipulate it. The advantage of using noncontact force is that users do not need to wear a specific device and to process tools used for the activity. As a proof-of-concept system, we developed an ultrasound-based curveball system by which table tennis players can shoot a curveball regardless of their physical ability. In the system, a moving ping-pong ball (PPB) is a target tool for remote manipulation. The system curves the trajectory of a moving PPB by continuously focusing ultrasound on it. Users can control the curve timing and the curve direction (left or right) using a racket-shaped controller. In the user study, we conducted an actual table tennis match using the curveball system and qualitatively confirmed that the player using the system had the upper hand. Another user study using a ball dispenser quantitatively showed that the ultrasound-driven curveball increased the number of mistakes of the opponent player 2.95 times. These results indicate that the proposed concept is feasible.

Skip Supplemental Material Section

Supplemental Material

V5iss503sVF.mp4

Supplemental video

References

  1. Kentaro Ariga, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda. 2020. Midair Haptic Presentation Using Concave Reflector. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 307--315.Google ScholarGoogle Scholar
  2. Aaron Becker, Robert Sandheinrich, and Timothy Bretl. 2009. Automated manipulation of spherical objects in three dimensions using a gimbaled air jet. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 781--786.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 505--514.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bioeffects Committee et almbox. 1977. AIUM: Statement on mammalian in vivo ultrasonic biological effects. JCU , Vol. 5 (1977), 2.Google ScholarGoogle Scholar
  5. Piyum Fernando, Roshan Lalintha Peiris, and Suranga Nanayakkara. 2014. I-Draw: towards a freehand drawing assistant. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design . 208--211.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Takuro Furumoto, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda. 2020. Balloon Interface for Midair Haptic Interaction. In SIGGRAPH Asia 2020 Emerging Technologies. 1--2.Google ScholarGoogle Scholar
  7. Takuro Furumoto, Keisuke Hasegawa, Yasutoshi Makino, and Hiroyuki Shinoda. 2019 a. Three-Dimensional Manipulation of a Spherical Object Using Ultrasound Plane Waves. IEEE Robotics and Automation Letters , Vol. 4, 1 (2019), 81--88.Google ScholarGoogle ScholarCross RefCross Ref
  8. Takuro Furumoto, Mitsuru Ito, Masahiro Fujiwara, Yasutoshi Makino, Hiroyuki Shinoda, and Takaaki Kamigaki. 2019 b. Three-dimensional Interaction Technique Using an Acoustically Manipulated Balloon. In SIGGRAPH Asia 2019 Emerging Technologies. 51--52.Google ScholarGoogle Scholar
  9. Tatsuki Fushimi, Asier Marzo, Bruce W Drinkwater, and Thomas L Hill. 2019. Acoustophoretic volumetric displays using a fast-moving levitated particle. Applied Physics Letters , Vol. 115, 6 (2019), 064101.Google ScholarGoogle Scholar
  10. Guillaume Gourmelen, Adrien Verhulst, Benjamin Navarro, Tomoya Sasaki, Ganesh Gowrishankar, and Masahiko Inami. 2019. Co-Limbs: An Intuitive Collaborative Control for Wearable Robotic Arms. In SIGGRAPH Asia 2019 Emerging Technologies. 9--10.Google ScholarGoogle Scholar
  11. Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics , Vol. 3, 3 (2010), 155--165.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Jian Huang, Weiguang Huo, Wenxia Xu, Samer Mohammed, and Yacine Amirat. 2015. Control of upper-limb power-assist exoskeleton using a human-robot interface based on motion intention recognition. IEEE transactions on automation science and engineering , Vol. 12, 4 (2015), 1257--1270.Google ScholarGoogle ScholarCross RefCross Ref
  13. Seki Inoue, Yasutoshi Makino, and Hiroyuki Shinoda. 2016. Scalable architecture for airborne ultrasound tactile display. In International AsiaHaptics conference. Springer, 99--103.Google ScholarGoogle Scholar
  14. Seki Inoue, Shinichi Mogami, Tomohiro Ichiyama, Akihito Noda, Yasutoshi Makino, and Hiroyuki Shinoda. 2019. Acoustical boundary hologram for macroscopic rigid-body levitation. The Journal of the Acoustical Society of America , Vol. 145, 1 (2019), 328--337.Google ScholarGoogle Scholar
  15. Hiroshi Ishii, Craig Wisneski, Julian Orbanes, Ben Chun, and Joe Paradiso. 1999. PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 394--401.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016a. Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectors. In ACM SIGGRAPH 2016 Emerging Technologies. ACM, 13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016b. Laplacian vision: Augmenting motion prediction via optical see-through head-mounted displays. In Proceedings of the 7th Augmented Human International Conference 2016. ACM, 16.Google ScholarGoogle Scholar
  18. Takayuki Iwamoto, Mari Tatezono, and Hiroyuki Shinoda. 2008. Non-contact method for producing tactile sensation using airborne ultrasound. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications . Springer, 504--513.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. BV Jayawant. 1981. Electromagnetic suspension and levitation. Reports on Progress in Physics , Vol. 44, 4 (1981), 411.Google ScholarGoogle ScholarCross RefCross Ref
  20. Soheil Kianzad, Yuxiang Huang, Robert Xiao, and Karon E MacLean. 2020. Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. ZeroN: mid-air tangible interaction enabled by computer controlled magnetic levitation. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 327--336.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Azumi Maekawa, Seito Matsubara, Sohei Wakisaka, Daisuke Uriu, Atsushi Hiyama, and Masahiko Inami. 2020. Dynamic Motor Skill Synthesis with Human-Machine Mutual Actuation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Azumi Maekawa, Shota Takahashi, MHD Yamen Saraiji, Sohei Wakisaka, Hiroyasu Iwata, and Masahiko Inami. 2019. Naviarm: Augmenting the Learning of Motor Skills using a Backpack-type Robotic Arm System. In Proceedings of the 10th Augmented Human International Conference 2019. 1--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Mark Marshall, Thomas Carter, Jason Alexander, and Sriram Subramanian. 2012. Ultra-tangibles: creating movable tangible objects on interactive tables. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2185--2188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Asier Marzo, Mihai Caleap, and Bruce W Drinkwater. 2018. Acoustic virtual vortices with tunable orbital angular momentum for trapping of mie particles. Physical review letters , Vol. 120, 4 (2018), 044301.Google ScholarGoogle Scholar
  26. Asier Marzo and Bruce W Drinkwater. 2019. Holographic acoustic tweezers. Proceedings of the National Academy of Sciences , Vol. 116, 1 (2019), 84--89.Google ScholarGoogle ScholarCross RefCross Ref
  27. Asier Marzo, Richard McGeehan, Jess McIntosh, Sue Ann Seah, and Sriram Subramanian. 2015a. Ghost touch: Turning surfaces into interactive tangible canvases with focused ultrasound. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces. 137--140.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Asier Marzo, Sue Ann Seah, Bruce W Drinkwater, Deepak Ranjan Sahoo, Benjamin Long, and Sriram Subramanian. 2015b. Holographic acoustic elements for manipulation of levitated objects. Nature communications , Vol. 6 (2015), 8661.Google ScholarGoogle Scholar
  29. Tao Morisaki, Ryoma Mori, Ryosuke Mori, Yasutoshi Makino, Yuta Itoh, Yuji Yamakawa, and Hiroyuki Shinoda. 2019. Hopping-Pong: Changing Trajectory of Moving Object Using Computational Ultrasound Force. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces. 123--133.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Tao Morisaki, Ryoma Mori, Ryosuke Mori, Kohki Serizawa, Yasutoshi Makino, Yuta Itoh, Yuji Yamakawa, and Hiroyuki Shinoda. 2020. Hopping-Pong: Computational Curveball in Table Tennis by Noncontact Ultrasound Force. In ACM SIGGRAPH 2020 Emerging Technologies. 1--2.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Florian'Floyd' Mueller and Martin R Gibbs. 2007 a. Building a table tennis game for three players. In Proceedings of the international conference on Advances in computer entertainment technology. ACM, 179--182.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Florian'Floyd' Mueller and Martin R Gibbs. 2007 b. Evaluating a distributed physical leisure game for three players. In Proceedings of the 19th Australasian Conference on Computer-Human Interaction: Entertaining User Interfaces. ACM, 143--150.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kei Nitta, Keita Higuchi, and Jun Rekimoto. 2014. HoverBall: augmented sports with a flying ball. In Proceedings of the 5th Augmented Human International Conference. ACM, 13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kei Nitta, Keita Higuchi, Yuichi Tadokoro, and Jun Rekimoto. 2015. Shepherd pass: ability tuning for augmented sports using ball-shaped quadcopter. In Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology. 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yoichi Ochiai, Takayuki Hoshi, and Jun Rekimoto. 2014a. Pixie dust: graphics generated by levitated and animated objects in computational acoustic-potential field. ACM Transactions on Graphics (TOG) , Vol. 33, 4 (2014), 85.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Yoichi Ochiai, Takayuki Hoshi, and Jun Rekimoto. 2014b. Three-dimensional mid-air acoustic manipulation by ultrasonic phased arrays. PloS one , Vol. 9, 5 (2014), e97590.Google ScholarGoogle ScholarCross RefCross Ref
  37. Tomoya Ohta, Shumpei Yamakawa, Takashi Ichikawa, and Takuya Nojima. 2014. TAMA: development of trajectory changeable ball for future entertainment. In Proceedings of the 5th Augmented Human International Conference. 1--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. MHD Yamen Saraiji, Tomoya Sasaki, Kai Kunze, Kouta Minamizawa, and Masahiko Inami. 2018. Metaarms: Body remapping using feet-controlled artificial arms. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology . 65--74.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Yuta Sugiura, Koki Toda, Takayuki Hoshi, Youichi Kamiyama, Takeo Igarashi, and Masahiko Inami. 2014. Graffiti fur: turning your carpet into a computer display. In Proceedings of the 27th annual ACM symposium on User interface software and technology . 149--156.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Shun Suzuki, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda. 2020. Reducing Amplitude Fluctuation by Gradual Phase Shift in Midair Ultrasound Haptics. IEEE Transactions on Haptics , Vol. 13, 1 (2020), 87--93.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Shun Suzuki, Seki Inoue, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda. 2021. AUTD3: Scalable Airborne Ultrasound Tactile Display. IEEE Transactions on Haptics (2021).Google ScholarGoogle Scholar
  42. Chi Thanh Vi, Asier Marzo, Damien Ablart, Gianluca Memoli, Sriram Subramanian, Bruce Drinkwater, and Marianna Obrist. 2017. Tastyfloats: A contactless food delivery system. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. 161--170.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. RR Whymark. 1975. Acoustic field positioning for containerless processing. Ultrasonics , Vol. 13, 6 (1975), 251--261.Google ScholarGoogle ScholarCross RefCross Ref
  44. Erwin Wu and Hideki Koike. 2020. Futurepong: Real-time table tennis trajectory forecasting using pose prediction network. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Junichi Yamaoka and Yasuaki Kakehi. 2013. dePENd: augmented handwriting system using ferromagnetism of a ballpoint pen. In Proceedings of the 26th annual ACM symposium on User interface software and technology . 203--210.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Hong Kai Yap, Jeong Hoon Lim, Fatima Nasrallah, James CH Goh, and Raye CH Yeow. 2015. A soft exoskeleton for hand assistive and rehabilitation application using pneumatic actuators with variable stiffness. In 2015 IEEE international conference on robotics and automation (ICRA). IEEE, 4967--4972.Google ScholarGoogle ScholarCross RefCross Ref
  47. K Yosioka and Y Kawasima. 1955. Acoustic radiation pressure on a compressible sphere. Acta Acustica united with Acustica , Vol. 5, 3 (1955), 167--173.Google ScholarGoogle Scholar

Index Terms

  1. Ultrasound-driven Curveball in Table Tennis: Human Activity Support via Noncontact Remote Object Manipulation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Article Metrics

      • Downloads (Last 12 months)96
      • Downloads (Last 6 weeks)8

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!