Abstract

Software engineers routinely use sketches (informal, ad-hoc drawings) to visualize and communicate complex ideas for colleagues or themselves. We hypothesize that sketching could also be used as a novel interaction modality in integrated software development environments (IDEs), allowing developers to express desired source code manipulations by sketching right on top of the IDE, rather than remembering keyboard shortcuts or using a mouse to navigate menus and dialogs. For an initial assessment of the viability of this idea, we conducted an elicitation study that prompted software developers to express a number of common IDE commands through sketches. For many of our task prompts, we observed considerable agreement in how developers would express the respective commands through sketches, suggesting that further research on a more formal sketch-based visual command language for IDEs would be worthwhile.
- Robin Andersson, Jonas Berglund, Aykut Coskun, Morten Fjeld, and Mohammad Obaid. 2017. Defining Gestural Interactions for Large Vertical Touch Displays. In Human-Computer Interaction - INTERACT 2017 - 16th IFIP TC 13 International Conference, Mumbai, India, September 25--29, 2017, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 10513), Regina Bernhaupt, Girish Dalvi, Anirudha Joshi, Devanuj K. Balkrishan, Jacki O'Neill, and Marco Winckler (Eds.). Springer, 36--55. https://doi.org/10.1007/978--3--319--67744--6_3Google Scholar
- Caroline Appert and Shumin Zhai. 2009. Using Strokes As Command Shortcuts: Cognitive Benefits and Toolkit Support. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI '09). ACM, New York, NY, USA, 2289--2298. https://doi.org/10.1145/1518701.1519052Google Scholar
Digital Library
- Uday Athavankar. 1997. Mental Imagery as a Design Tool. Cybernetics and Systems, Vol. 28, 1 (1997), 25--42. https://doi.org/10.1080/019697297126236 https://doi.org/10.1561/1100000012Google Scholar
Cross Ref
Index Terms
Eliciting Sketched Expressions of Command Intentions in an IDE
Recommendations
Towards Sketch-based User Interaction with Integrated Software Development Environments
ICSEW'20: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering WorkshopsPowerful software tools, such as software development environments, often have complex graphical user interfaces (GUIs) that are not intuitive to handle, especially when performing complex, multi-step operations. We hypothesize that sketching could be a ...
Learn to Sketch (Even if You Can't Draw): Hands-on Sketching Course
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsSketching as a technique to quickly draw something on a piece of paper can be used to explore and communicate ideas. Practitioners can make use of their ability to draw sketches from an early phase of a project on. It can be valuable not only for the ...
Sketch-sketch revolution: an engaging tutorial system for guided sketching and application learning
UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technologyWe describe Sketch-Sketch Revolution, a new tutorial system that allows any user to experience the success of drawing content previously created by an expert artist. Sketch-Sketch Revolution not only guides users through the application user interface, ...






Comments