Ruofan Liu
Erwin Wu
Hideki Koike
ABSTRACT
When learning a dexterous skill such as playing the piano, people commonly watch videos of a teacher. However, this conventional way has some downsides such as limited information to be retrieved and less intuitive instructions. We propose a virtual training system by visualizing differences between hands to provide intuitive feedback for skill acquisition. After synchronizing the data, two visual cues are proposed including a hand-overlay manner and a two-keyboards visualization. A pilot study confirm the superiority of the proposed methods over conventional video-viewing.
Supplemental Material
- Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. 2019. Temporal Cycle-Consistency Learning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1801–1810. https://doi.org/10.1109/CVPR.2019.00190Google Scholar
- Werner Goebl and Caroline Palmer. 2013. Temporal Control and Hand Movement Efficiency in Skilled Music Performance. PLOS ONE 8, 1 (01 2013), 1–10. https://doi.org/10.1371/journal.pone.0050901Google Scholar
- Seita Kayukawa, Keita Higuchi, Ryo Yonetani, Masanori Nakamura, Yoichi Sato, and Shigeo Morishima. 2018. Dynamic Object Scanning: Object-Based Elastic Timeline for Quickly Browsing First-Person Videos. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–4. https://doi.org/10.1145/3170427.3186501Google ScholarDigital Library
- Takayuki Nozawa, Erwin Wu, Florian Perteneder, and Hideki Koike. 2019. Visualizing Expert Motion for Guidance in a VR Ski Simulator. In ACM SIGGRAPH 2019 Posters (Los Angeles, California) (SIGGRAPH ’19). Association for Computing Machinery, New York, NY, USA, Article 64, 2 pages. https://doi.org/10.1145/3306214.3338561Google ScholarDigital Library
Recommendations
PianoHandSync: An Alignment-based Hand Pose Discrepancy Visualization System for Piano Learning
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing SystemsVideo-based lessons are becoming a popular way for distance piano education. However, limited by the fixed camera angle, a video is difficult to tell precise 3D hand posture, which is one of the most essential factors for learning piano. This paper ...
Following the Master’s Hands: Capturing Piano Performances for Mixed Reality Piano Learning Applications
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing SystemsPiano learning applications in Mixed Reality (MR) are a promising substitute for physical instruction when a piano teacher is absent. Existing piano learning applications that use visual indicators to highlight the notes to be played on the keyboard or ...
Piano&Dancer: Interaction Between a Dancer and an Acoustic Instrument
MOCO '17: Proceedings of the 4th International Conference on Movement ComputingPiano&Dancer is an interactive piece for a dancer and an electromechanical acoustic piano. The piece presents the dancer and the piano as two performers on stage whose bodily movements are mutually interdependent. This interdependence reveals a close ...
Comments