Abstract

Touch-surface multi-stroke gestures, as well as freehand drawings, are typically acquired by devices and sensors as a suite of timestamped points on a plane. This Cartesian coordinate system, although useful for computation like complexity analysis, gesture classification and recognition, becomes complex and inefficient when gestures need to be visualized and directly manipulated for editing. To address these challenges, a new mathematical representation of these gestures via a Bézier curve is defined to initiate a model-based approach for gesture direct manipulation (e.g., cut, copy, paste, translate, scale, rotate, deform, crop, compose, decompose). SketchADoodle, an Android-based mobile application for drawing, gesturing, demonstrates how the pseudo-code of the Bézier-based operations are engineered for real-time direct manipulation. We release the programming code for further development of gesture-based user interfaces based on Bézier curves.
- Adobe. 2019. PhotoShop. (2019).https://www.adobe.com/products/photoshop.htmlGoogle Scholar
- Caroline Appert and Shumin Zhai. 2009. Using Strokes as Command Shortcuts: Cognitive Benefits and Toolkit Support. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '09). Association for Computing Machinery, New York, NY, USA, 2289--2298. https://doi.org/10.1145/1518701.1519052Google Scholar
Digital Library
- Daniel Ashbrook and Thad Starner. 2010. MAGIC: A Motion Gesture Design Tool. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2159--2168. https://doi.org/10.1145/1753326.1753653Google Scholar
Digital Library
- Oscar Kin-Chung Au, Chiew-Lan Tai, and Hongbo Fu. 2012. Multitouch Gestures for Constrained Transformation of3D Objects.Computer Graphics Forum 31, 2pt3 (May 2012), 651--660. https://doi.org/10.1111/j.1467--8659.2012.03044.xGoogle Scholar
- Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. 2002. Models and Issues in Data Stream Systems. In Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS '02). Association for Computing Machinery, New York, NY, USA, 1--16.https://doi.org/10.1145/543613.543615Google Scholar
Digital Library
- P.J. Barry and R.N. Goldman. 1988. de Casteljau-Type Subdivision is Peculiar to Bézier Curves. Computer-Aided Design 20 (1988). Issue 3. https://doi.org/10.1016/0010--4485(88)90018--8Google Scholar
- Pierre Bézier. 1978. General distortion of an ensemble of biparametric surfaces. Computer-Aided Design 10, 2 (1978), 116--120. https://doi.org/10.1016/0010--4485(78)90088-XGoogle Scholar
Cross Ref
- Birgit Bomsdorf, Rainer Blum, and Daniel Künkel. 2015. Towards Pro Gesture, a Tool Supporting Early Prototyping of 3D-Gesture Interaction.International Journal of People-Oriented Programming 4, 2 (July 2015), 54--70.https://doi.org/10.4018/IJPOP.2015070103Google Scholar
- Ugo Braga Sangiorgi, Vivian Genaro Motti, François Beuvens, and Jean Vanderdonckt. 2012. Assessing Lag Perception in Electronic Sketching. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12). Association for Computing Machinery, New York, NY, USA, 153--161. https://doi.org/10.1145/2399016.2399040Google Scholar
Digital Library
- Ashley Bush and Sandeep Purao. 2011. Mapping UML Techniques to Design Activities. In Information Modeling in the New Millennium, Matti Rossi and Keng Siau (Eds.). IDEA Publishing Group, Chapter 12, 199--217.Google Scholar
- Alessandro Carcangiu and Lucio Davide Spano. 2018. G-Gene: A Gene Alignment Method for Online Partial Stroke Gestures Recognition. Proceedings of ACM Human-Computer Interaction 2, EICS, Article 13 (June 2018), 17 pages.https://doi.org/10.1145/3229095Google Scholar
Digital Library
- Hung-Hsin Chang and Hong Yan. 1998. Vectorization of Hand-Drawn Image using Piecewise Cubic Bézier Curves Fitting. Pattern Recognition 31, 11 (1998), 1747 -- 1755. https://doi.org/10.1016/S0031--3203(98)00045--4Google Scholar
Cross Ref
- Fuhua Cheng and Chi-Cheng Lin. 1985. Clipping of Bézier Curves. In Proceedings of the 1985 ACM Annual Conference on The Range of Computing: Mid-80's Perspective (ACM '85). ACM, New York, NY, USA, 74--84. https://doi.org/10.1145/320435.320462Google Scholar
Digital Library
- E. Cohen, R. F. Riesenfeld, and G. Elber. 2001. Geometric Modeling with Splines: An Introduction. A. K. Peters, Ltd., Natick, MA, United States.Google Scholar
Digital Library
- Corel Corporation. 2019. CorelDRAW Graphics Suite. (2019). https://www.coreldraw.com/Google Scholar
- Victor A. Debelov and Aleksandr M. Matsokin. 2000. Implementation of Set Operations and Intersection of Bézier Curves. Computers & Graphics 24 (2000). Issue 1. https://doi.org/10.1016/s0097--8493(99)00137--5Google Scholar
- Qiulei Dong, Yihong Wu, and Zhanyi Hu. 2006. Gesture Recognition using Quadratic Curves. In Computer Vision(ACCV '06), P. J. Narayanan, Shree K. Nayar, and Heung-Yeung Shum (Eds.). Springer, Berlin, Heidelberg, 817--825.Google Scholar
- Arno Eigenwillig, Lutz Kettner, and Nicola Wolpert. 2007. Snap Rounding of Bézier Curves. In Proceedings of the 23rd ACM Annual Symposium on Computational Geometry (SCG '07). ACM, New York, NY, USA, 158--167. https://doi.org/10.1145/1247069.1247101Google Scholar
Digital Library
- Gerald Farin. 1993. Curves and Surfaces for Computer Aided Geometric Design: A Practical Guide(3 ed.). Academic Press Professional, Inc., San Diego, CA, USA.Google Scholar
- Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. Simplicity: A Global Approach to Line Drawing Vectorization. ACM Transactions on Graphics 35, 4, Article 120 (July 2016), 10 pages. https://doi.org/10.1145/2897824.2925946Google Scholar
Digital Library
- Sebastian Feuerstack, Mauro Dos Santos Anjo, and Ednaldo B. Pizzolato. 2011. Model-based Design and Generation of a Gesture-based User Interface Navigation Control. In Proceedings of the 10th Brazilian Symposium on Human Factors in Computing Systems and the 5th Latin American Conference on Human-Computer Interaction (IHC+CLIHC '11). Brazilian Computer Society, Porto Alegre, Brazil, 227--231. http://dl.acm.org/citation.cfm?id=2254436.2254475Google Scholar
Digital Library
- Leah Findlater, Karyn Moffatt, Joanna McGrenere, and Jessica Dawson. 2009. Ephemeral Adaptation: The Use of Gradual Onset to Improve Menu Selection Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). Association for Computing Machinery, New York, NY, USA, 1655-1664. https://doi.org/10.1145/1518701.1518956Google Scholar
Digital Library
- Erich Gamma, Richard Helm, Ralph Johnson, and John M. Vlissides. 1994.Design Patterns: Elements of Reusable Object-Oriented Software(1 ed.). Addison-Wesley Professional. http://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612/ref=ntt_at_ep_dpi_1Google Scholar
Digital Library
- Bogdan-Florin Gheran, Jean Vanderdonckt, and Radu-Daniel Vatavu. 2018. Gestures for Smart Rings: Empirical Results,Insights, and Design Implications. In Proceedings of the ACM International Conference on Designing Interactive Systems(DIS '18). ACM, New York, NY, USA, 623--635.https://doi.org/10.1145/3196709.3196741Google Scholar
Digital Library
- Yves Mineur; Tony Lichah; Jean Marie Castelain; Henri Giaume. 1998. A shape controlled fitting method for Bãzier curves. Computer Aided Geometric Design 15 (1998). Issue 9. https://doi.org/10.1016/s0167--8396(98)00025--9Google Scholar
- Donatien Grolaux. 2020. Java Script version of Sketch A Doodle algorithms. (2020). https://github.com/GrolauxDonatien/SketchADoodleJSGoogle Scholar
- L. J. Guibas and J. Stolfi. 1982. A Language for Bitmap Manipulation.ACM Trans. Graph.1, 3 (July 1982), 191--214. https://doi.org/10.1145/357306.357308Google Scholar
Digital Library
- Kenrick Kin, Björn Hartmann, Tony DeRose, and Maneesh Agrawala. 2012. Proton++: A Customizable Declarative Multitouch Framework. In Proceedings of the 25th ACM Symposium on User Interface Software and Technology (UIST'12). ACM, New York, NY, USA, 477--486. https://doi.org/10.1145/2380116.2380176Google Scholar
Digital Library
- P. P. Korovkin. 2001. Encyclopedia of Mathematics. Springer Science+Business Media/ Kluwer Academic Publishers, Berlin, Germany, Chapter Bernstein polynomials. https://www.encyclopediaofmath.org/index.php/Bernstein_polynomialsGoogle Scholar
- Gordon Kurtenbach, George Fitzmaurice, Thomas Baudel, and Bill Buxton. 1997. The Design of a GUI Paradigm Basedon Tablets, Two-hands, and Transparency. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '97). ACM, New York, NY, USA, 35--42. https://doi.org/10.1145/258549.258574Google Scholar
Digital Library
- Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers' Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, Article Paper 31, 12 pages. https://doi.org/10.1145/3173574.3173605Google Scholar
Digital Library
- Luis A. Leiva, Daniel Martín-Albo, and Réjean Plamondon. 2015. Gestures à Go Go: Authoring Synthetic Human-Like Stroke Gestures Using the Kinematic Theory of Rapid Movements. ACM Trans. Intell. Syst. Technol.7, 2, Article Article 15 (Nov. 2015), 29 pages. https://doi.org/10.1145/2799648Google Scholar
Digital Library
- James R. Lewis. 1995. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7, 1 (1995), 57--78. https://doi.org/10.1080/10447319509526110arXiv:https://doi.org/10.1080/10447319509526110Google Scholar
Digital Library
- James R. Lewis. 2002. Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies. International Journal of Human-Computer Interaction 14, 3--4 (2002), 463--488.https://doi.org/10.1080/10447318.2002.9669130arXiv:https://doi.org/10.1080/10447318.2002.9669130Google Scholar
Cross Ref
- James R. Lewis. 2006. Sample Sizes for Usability Tests: Mostly Math, Not Magic. interactions 13, 6 (Nov. 2006), 29--33.https://doi.org/10.1145/1167948.1167973Google Scholar
Digital Library
- Rensis Likert. 1932. A Technique for the Measurement of Attitudes.Archives of Psychology22, 140 (1932), 55--.http://psycnet.apa.org/record/1933-01885-001Google Scholar
- Jingbo Liu, Oscar Kin-Chung Au, Hongbo Fu, and Chiew-Lan Tai. 2012. Two-Finger Gestures for 6DOF Manipulation of 3D Objects. Computer Graphics Forum 31, 7pt1 (sep 2012), 2047--2055.https://doi.org/10.1111/j.1467--8659.2012.03197.xGoogle Scholar
Digital Library
- Yingbin Liu and Stephen Mann. 2008. Parametric Triangular Bézier Surface Interpolation with Approximate Continuity. In Proceedings of the ACM Symposium on Solid and Physical Modeling (June 2008)(SPM '08). Association for Computing Machinery, New York, NY, USA, 381--387.https://doi.org/10.1145/1364901.1364956Google Scholar
Digital Library
- Hao Lü, James A. Fogarty, and Yang Li. 2014. Gesture Script: Recognizing Gestures and Their Structure Using Rendering Scripts and Interactively Trained Parts. In Proceedings of the 32nd ACM International Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1685--1694. https://doi.org/10.1145/2556288.2557263Google Scholar
- Hao Lü and Yang Li. 2013. Gesture Studio: Authoring Multi-touch Interactions Through Demonstration and Declaration. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 257--266.https://doi.org/10.1145/2470654.2470690Google Scholar
- Otto Parra, Sergio Espana, Jose Ignacio Panach, and Oscar Pastor. 2019. An empirical comparative evaluation of gestUI to include gesture-based interaction in user interfaces. Science of Computer Programming 172 (2019), 232 -- 263. https://doi.org/10.1016/j.scico.2018.12.001Google Scholar
Cross Ref
- Hartmut Prautzsch. 1989. A Round Trip to B-Splines via de Casteljau. ACM Trans. Graph. 8, 3 (July 1989), 243--254.https://doi.org/10.1145/77055.77061Google Scholar
Digital Library
- O. Rashid and A. Al-Hamadi. 2015. Utilizing the Bézier Descriptors for Hand Gesture Recognition. In Proceedings of the IEEE International Conference on Image Processing (ICIP '15). 3525--3529. https://doi.org/10.1109/ICIP.2015.7351460Google Scholar
- Micha Sapek and Szczepan Paszkiel. 2017. Detection of Gestures Without Begin and End Markers by Fitting into Bézier Curves with Least Squares Method. Pattern Recognition Letters 100, C (dec 2017), 83--88. https://doi.org/10.1016/j.patrec.2017.10.006Google Scholar
- Jeff Sauro and James R. Lewis. 2016. Quantifying the User Experience. Practical Statistics for User Re-search(2 ed.). Morgan Kaufmann, Burlington, MA, USA.https://www.sciencedirect.com/book/9780123849687/quantifying-the-user-experienceGoogle Scholar
Digital Library
- Ovidiu Andrei Schipor, Radu-Daniel Vatavu, and Jean Vanderdonckt. 2019. Euphoria: A Scalable, event-driven architecture for designing interactions across heterogeneous devices in smart environments.Information & Software Technology 109 (2019), 43--59. https://doi.org/10.1016/j.infsof.2019.01.006Google Scholar
- L. Shen, S. Huo, M. Chen, F. Li, F. Yao, and H. Shen. 2017. Multi-touch Gesture Recognition Algorithm of Vehicle Electronic Devices-based on Bézier Curve Optimization Strategy. In Proceedings of the 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC '17). 720--723. https://doi.org/10.1109/YAC.2017.7967503Google Scholar
Cross Ref
- Min C. Shin, V. Tsap, Leonid, and Dmitry B. Goldgof. 2004. Gesture Recognition using Bézier Curves for Visualization Navigation from Registered 3-D Data.Pattern Recognition37, 5 (2004), 1011--1024. https://doi.org/10.1016/j.patcog.2003.11.007Google Scholar
- Lucio Davide Spano, Antonio Cisternino, Fabio Paternò, and Gianni Fenu. 2013. GestIT: A Declarative and Com-positional Framework for Multiplatform Gesture Definition. In Proceedings of the 5th ACM International Symposium on Engineering Interactive Computing Systems (EICS '13). ACM, New York, NY, USA, 187--196. https://doi.org/10.1145/2494603.2480307Google Scholar
- Eugene M. Taranta, Andres N. Vargas, and Joseph J. LaViola. 2016. Streamlined and accurate gesture recognition with Penny Pincher. Computers and Graphics 55 (2016), 130 -- 142. https://doi.org/10.1016/j.cag.2015.10.011Google Scholar
Digital Library
- Inkscape Team. 2019. Inkscape - Draw Freely. (2019).https://inkscape.org/Google Scholar
- The GIMP Team. 2019. GIMP - GNU Image Manipulation. (2019).https://www.gimp.org/Google Scholar
- Jean Vanderdonckt, Paolo Roselli, and Jorge Luis Pérez-Medina. 2018. !FTL, an Articulation-Invariant Stroke Gesture Recognizer with Controllable Position, Scale, and Rotation Invariances. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI '18). ACM, New York, NY, USA, 125--134.https://doi.org/10.1145/3242969.3243032Google Scholar
Digital Library
- Radu-Daniel Vatavu. 2017. Improving Gesture Recognition Accuracy on Touch Screens for Users with Low Vision. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(May 2017)(CHI '17). Association for Computing Machinery, New York, NY, USA, 4667--4679.https://doi.org/10.1145/3025453.3025941Google Scholar
Digital Library
- Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2018. $Q: A Super-quick, Articulation-invariant Stroke-gesture Recognizer for Low-resource Devices. In Proceedings of the 20th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '18). ACM, New York, NY, USA, Article 23, 12 pages.https://doi.org/10.1145/3229434.3229465Google Scholar
- Julie R. Williamson, Stephen Brewster, and Rama Vennelakanti. 2013. Mo!Games: Evaluating Mobile Gestures in the Wild. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI '13). ACM, New York,NY, USA, 173--180. https://doi.org/10.1145/2522848.2522874Google Scholar
Digital Library
- Atsushi Yamada and Fujio Yamaguchi. 1996. Homogeneous Bounding Boxes as Tools for Intersection Algorithms of Rational Bézier Curves and Surfaces.The Visual Computer 12, 4 (01 Apr 1996), 202--214.https://doi.org/10.1007/BF01782323Google Scholar
- Chee K. Yap. 2006. Complete Subdivision Algorithms, I: Intersection of Bézier Curves. In Proceedings of the 22nd ACM Symposium on Computational Geometry (SCG '06). ACM, New York, NY, USA, 217--226.https://doi.org/10.1145/1137856.1137890Google Scholar
Digital Library
- Shumin Zhai, Per Kristensson, Caroline Appert, Tue Andersen, and Xiang Cao. 2012. Foundational Issues in Touch-Surface Stroke Gesture Design: An Integrative Review. Foundations of Trends in Human-Computer Interaction 5, 2 (feb 2012), 97--205. https://doi.org/10.1561/1100000012Google Scholar
Digital Library
- Lejun Shao; Hao Zhou. 1996. Curve Fitting with Bézier Cubics. Graphical Models and Image Processing58 (1996). Issue3. https://doi.org/10.1006/gmip.1996.0019Google Scholar
Index Terms
SketchADoodle: Touch-surface Multi-stroke Gesture Handling by Bézier Curves
Recommendations
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
A comparative evaluation of finger and pen stroke gestures
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThis paper reports an empirical investigation in which participants produced a set of stroke gestures with varying degrees of complexity and in different target sizes using both the finger and the pen. The recorded gestures were then analyzed according ...
Foundational Issues in Touch-Surface Stroke Gesture Design: An Integrative Review
The potential for using stroke gestures to enter, retrieve and select commands and text has been recently unleashed by the popularity of touchscreen devices. This monograph provides a state-of-the-art integrative review of a body of human–computer ...






Comments