“Writing with music”: Exploring the use of auditory feedback in gesture interfaces
|
Tools and Resources
Share: |
||||||||||||||||||||||
ABSTRACTWe investigate the use of auditory feedback in pen-gesture interfaces in a series of informal and formal experiments. Initial iterative exploration showed that gaining performance advantage with auditory feedback was possible using absolute cues and state feedback after the gesture was produced and recognized. However, gaining learning or performance advantage from auditory feedback tightly coupled with the pen-gesture articulation and recognition process was more difficult. To establish a systematic baseline, Experiment 1 formally evaluated gesture production accuracy as a function of auditory and visual feedback. Size of gestures and the aperture of the closed gestures were influenced by the visual or auditory feedback, while other measures such as shape distance and directional difference were not, supporting the theory that feedback is too slow to strongly influence the production of pen stroke gestures. Experiment 2 focused on the subjective aspects of auditory feedback in pen-gesture interfaces. Participants' rating on the dimensions of being wonderful and stimulating was significantly higher with musical auditory feedback. Several lessons regarding pen gestures and auditory feedback are drawn from our exploration: a few simple functions such as indicating the pen-gesture recognition results can be achieved, gaining performance and learning advantage through tightly coupled process-based auditory feedback is difficult, pen-gesture sets and their recognizers can be designed to minimize visual dependence, and people's subjective experience of gesture interaction can be influenced using musical auditory feedback. These lessons may serve as references and stepping stones toward future research and development in pen-gesture interfaces with auditory feedback.
AUTHORS
|
|
||||||||||||||||||||||||||||||||||||||||
| View colleagues of Tue Haste Andersen | |||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
| View colleagues of Shumin Zhai | ||||||||||||||||||||||||||||||||||||||||
REFERENCESNote: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
|
1
|
Algazi, V. R., Duda, R. O., Thompson, D. M., and Avendano, C. 2001. The cipic hrtf database. In Proceedings of the Workshop on Applications of Signal Processing to Audio and Electroacoustics. IEEE, Los Alamitos, CA, 99--102.
|
|
|
2
|
Blattner, M., Sumikawa, D., and Greenberg, R. 1990. Earcons and icons: Their structure and common design principles. Hum. Comput. Interact. 16, 523--531.
|
|
|
3
|
||
| |
4
|
|
|
5
|
Broadbent, D. 1977. The hidden pre-attentive processes. Am. Psychol. 32, 2, 109--118.
|
|
|
6
|
Buxton, W. 1995. Readings in Human Computer Interaction: Towards the Year 2000. Morgan Kaufmann Publishers, San Francisco, 525.
|
|
| |
7
|
|
|
8
|
Clarke, E. 1999. Rhythm and timing in music. In The Psychology of Music 2nd Ed. Deutsch, D. Ed., Academic Press, San Diego, CA, 473--500.
|
|
|
9
|
Deatherage, B. H. 1972. Human Engineering Guide to Equipment Design. Revised Ed. John Wiley and Sons, New York, NJ, 123--160.
|
|
|
10
|
Fletcher, H., Blackham, E., and Stratton, R. 1962. Quality of piano tones. J. Acoust. Soc. Am. 34, 6, 749--761.
|
|
|
11
|
||
|
12
|
Ghez, C., Rikakis, T., DuBois, R. L., and Cook, P. 2000. An auditory display system for aiding interjoint coordination. In Proceedings of the International Conference on Auditory Display. ACM, New York.
|
|
| |
13
|
|
|
14
|
Järveläinen, H., Välimäki, V., and Karjalainen, M. 2001. Audibility of the timbral effects of inharmonicity in stringed instrument tones. Acoust. Res. Lett. Online 2, 79--84.
|
|
|
15
|
||
| |
16
|
|
| |
17
|
|
|
18
|
Lacuaniti, F., Terzuolo, C., and Viviani, P. 1983. The law relating the kinematic and figural aspects of drawing movements. Acta Psychologica 54, 115--130.
|
|
|
19
|
Legge, D., Steinberg, H., and Summerfield, A. 1964. Simple measures of handwriting as indices of drug effects. Percept. Motor Skills 18, 549--558.
|
|
|
20
|
Loeb, R. and Fitch, W. 2002. A laboratory evaluation of an auditory display designed to enhance intraoperative monitoring. Anesthesia Analgesia 94, 362--368.
|
|
|
21
|
Luschei, E., Saslow, C., and Glickstein, M. 1967. Muscle potentials in reaction time. Exp. Neurol. 18, 429--442.
|
|
|
22
|
||
|
23
|
MacMillan, K., Droettboom, M., and Fujinaga, I. 2001. Audio latency measurements of desktop operating systems. In Proceedings of the International Computer Music Conference. 259--262.
|
|
|
24
|
Norman, D. 2004. Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, New York.
|
|
|
25
|
Olive, T. and Piolat, A. 2002. Suppressing visual feedback in written composition: Effects on processing demands and coordination of the writing processes. Int. J. Psych. 37, 4, 209--218.
|
|
|
26
|
Patterson, R. 1989. Guidelines for the design of auditory warning sounds. In Proceedings of the Institute of Acoustics Spring Conference. Vol. 2. Institute of Acoustics, 17--24.
|
|
| |
27
|
|
|
28
|
Shea, C., Wulf, G., Park, J., and Gaunt, B. 2001. Effects of an auditory model on the learning of relative and absolute timing. J. Motor Behav. 33, 2, 127--138.
|
|
|
29
|
Singh, P. 1987. Perceptual organization of complex-tone sequences: A tradeoff between pitch and timbre? J. Acoust. Soc. Am. 82, 886--899.
|
|
|
30
|
Smyth, M. and Silvers, G. 1987. Functions of vision in the control of handwriting. Acta Psychologica 66, 47--64.
|
|
|
31
|
Teulings, H. L. and Schomaker, L. 1993. Invariant properties between stroke features in handwriting. Acta Psychologica 82, 69--88.
|
|
|
32
|
Tractinsky, N., Katz, A., and Ikar, D. 2000. What is beautiful is usable. Interact. Comput. 13, 127--145.
|
|
|
33
|
van Doorn, R. and Keuss, P. 1992. The role of vision in the temporal and spatial control of handwriting. Acta Psychologica 81, 269--286.
|
|
|
34
|
van Doorn, R. and Keuss, P. 1993. Does production of letter strokes in handwriting benefit from vision? Acta Psychologica 82, 275--290.
|
|
|
35
|
Welch, R. 1999. Meaning, attention, and the “unity assumption” in the intersensory bias of spatial and temporal perceptions. In Cognitive Contributions to the Perception of Spatial and Temporal Events, Elsevier.
|
|
| |
36
|
Jacob O. Wobbrock , Brad A. Myers , John A. Kembel, EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.61-70, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964703]
|
|
37
|
Woods, D. 1995. The alarm problem and directed attention in dynamic fault management. Ergonomics 38, 11, 2371--2394.
|
|
|
38
|
Wright, C. E. 1990. Generalized motor programs: Reexamining claims of effector independence in writing. In Attention and Performance XIII, Erlbaum, Hillsdale, NJ, 294--320.
|
|
|
39
|
Yost, W. A. 1994. Fundamentals of Hearing: An Introduction 3rd Ed. Academic Press, New York.
|
|
| |
40
|
CITED BY13 Citations
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
INDEX TERMSThe ACM Computing Classification System (CCS rev.2012)
PUBLICATION| Title | ACM Transactions on Applied Perception (TAP) TAP Homepage table of contents archive |
| Volume 7 Issue 3, June 2010 | |
| Article No. | 17 |
| Publication Date | 2010-06-01 (yyyy-mm-dd) |
| Publisher | ACM New York, NY, USA |
| ISSN: 1544-3558 EISSN: 1544-3965 doi>10.1145/1773965.1773968 |
REVIEWS
COMMENTSBe the first to comment To Post a comment please sign in or create a free Web account
Table of ContentsVolume 7 Issue 3, June 2010
| Identifying the role of proprioception in upper-limb prosthesis control: Studies on targeted motion | |
| Amy Blank, Allison M. Okamura, Katherine J. Kuchenbecker | |
| Article No.: 15 | |
| doi>10.1145/1773965.1773966 | |
Full text: PDF
|
|
|
Proprioception plays a crucial role in enabling humans to move purposively and interact with their physical surroundings. Current technology in upper-limb prostheses, while beginning to incorporate some haptic feedback, does not provide amputees with ...
expand
|
|
| Evaluating the multivariate visual quality performance of image-processing components | |
| Jenni Radun, Tuomas Leisti, Toni Virtanen, Jukka Häkkinen, Tero Vuori, Göte Nyman | |
| Article No.: 16 | |
| doi>10.1145/1773965.1773967 | |
Full text: PDF
|
|
|
The estimation of image quality is a demanding task, especially when estimating different high-quality imaging products or their components. The challenge is the multivariate nature of image quality as well as the need to use naïve observers as ...
expand
|
|
| “Writing with music”: Exploring the use of auditory feedback in gesture interfaces | |
| Tue Haste Andersen, Shumin Zhai | |
| Article No.: 17 | |
| doi>10.1145/1773965.1773968 | |
Full text: PDF
|
|
|
We investigate the use of auditory feedback in pen-gesture interfaces in a series of informal and formal experiments. Initial iterative exploration showed that gaining performance advantage with auditory feedback was possible using absolute cues and ...
expand
|
|
| Pilot gaze and glideslope control | |
| Juno Kim, Stephen A. Palmisano, April Ash, Robert S. Allison | |
| Article No.: 18 | |
| doi>10.1145/1773965.1773969 | |
Full text: PDF
|
|
|
We examined the eye movements of pilots as they carried out simulated aircraft landings under day and night lighting conditions. Our five students and five certified pilots were instructed to quickly achieve and then maintain a constant 3-degree glideslope ...
expand
|
|
| Evaluating 2D and 3D visualizations of spatiotemporal information | |
| Andreas Kjellin, Lars Winkler Pettersson, Stefan Seipel, Mats Lind | |
| Article No.: 19 | |
| doi>10.1145/1773965.1773970 | |
Full text: PDF
|
|
|
Time-varying geospatial data presents some specific challenges for visualization. Here, we report the results of three experiments aiming at evaluating the relative efficiency of three existing visualization techniques for a class of such data. The class ...
expand
|
|
| Neural modeling of flow rendering effectiveness | |
| Daniel Pineo, Colin Ware | |
| Article No.: 20 | |
| doi>10.1145/1773965.1773971 | |
Full text: PDF
|
|
|
It has been previously proposed that understanding the mechanisms of contour perception can provide a theory for why some flow rendering methods allow for better judgments of advection pathways than others. In this article, we develop this theory through ...
expand
|