skip to main content
research-article

SketchADoodle: Touch-surface Multi-stroke Gesture Handling by Bézier Curves

Authors Info & Claims
Published:18 June 2020Publication History
Skip Abstract Section

Abstract

Touch-surface multi-stroke gestures, as well as freehand drawings, are typically acquired by devices and sensors as a suite of timestamped points on a plane. This Cartesian coordinate system, although useful for computation like complexity analysis, gesture classification and recognition, becomes complex and inefficient when gestures need to be visualized and directly manipulated for editing. To address these challenges, a new mathematical representation of these gestures via a Bézier curve is defined to initiate a model-based approach for gesture direct manipulation (e.g., cut, copy, paste, translate, scale, rotate, deform, crop, compose, decompose). SketchADoodle, an Android-based mobile application for drawing, gesturing, demonstrates how the pseudo-code of the Bézier-based operations are engineered for real-time direct manipulation. We release the programming code for further development of gesture-based user interfaces based on Bézier curves.

References

  1. Adobe. 2019. PhotoShop. (2019).https://www.adobe.com/products/photoshop.htmlGoogle ScholarGoogle Scholar
  2. Caroline Appert and Shumin Zhai. 2009. Using Strokes as Command Shortcuts: Cognitive Benefits and Toolkit Support. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '09). Association for Computing Machinery, New York, NY, USA, 2289--2298. https://doi.org/10.1145/1518701.1519052Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Daniel Ashbrook and Thad Starner. 2010. MAGIC: A Motion Gesture Design Tool. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2159--2168. https://doi.org/10.1145/1753326.1753653Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Oscar Kin-Chung Au, Chiew-Lan Tai, and Hongbo Fu. 2012. Multitouch Gestures for Constrained Transformation of3D Objects.Computer Graphics Forum 31, 2pt3 (May 2012), 651--660. https://doi.org/10.1111/j.1467--8659.2012.03044.xGoogle ScholarGoogle Scholar
  5. Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. 2002. Models and Issues in Data Stream Systems. In Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS '02). Association for Computing Machinery, New York, NY, USA, 1--16.https://doi.org/10.1145/543613.543615Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. P.J. Barry and R.N. Goldman. 1988. de Casteljau-Type Subdivision is Peculiar to Bézier Curves. Computer-Aided Design 20 (1988). Issue 3. https://doi.org/10.1016/0010--4485(88)90018--8Google ScholarGoogle Scholar
  7. Pierre Bézier. 1978. General distortion of an ensemble of biparametric surfaces. Computer-Aided Design 10, 2 (1978), 116--120. https://doi.org/10.1016/0010--4485(78)90088-XGoogle ScholarGoogle ScholarCross RefCross Ref
  8. Birgit Bomsdorf, Rainer Blum, and Daniel Künkel. 2015. Towards Pro Gesture, a Tool Supporting Early Prototyping of 3D-Gesture Interaction.International Journal of People-Oriented Programming 4, 2 (July 2015), 54--70.https://doi.org/10.4018/IJPOP.2015070103Google ScholarGoogle Scholar
  9. Ugo Braga Sangiorgi, Vivian Genaro Motti, François Beuvens, and Jean Vanderdonckt. 2012. Assessing Lag Perception in Electronic Sketching. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12). Association for Computing Machinery, New York, NY, USA, 153--161. https://doi.org/10.1145/2399016.2399040Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Ashley Bush and Sandeep Purao. 2011. Mapping UML Techniques to Design Activities. In Information Modeling in the New Millennium, Matti Rossi and Keng Siau (Eds.). IDEA Publishing Group, Chapter 12, 199--217.Google ScholarGoogle Scholar
  11. Alessandro Carcangiu and Lucio Davide Spano. 2018. G-Gene: A Gene Alignment Method for Online Partial Stroke Gestures Recognition. Proceedings of ACM Human-Computer Interaction 2, EICS, Article 13 (June 2018), 17 pages.https://doi.org/10.1145/3229095Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Hung-Hsin Chang and Hong Yan. 1998. Vectorization of Hand-Drawn Image using Piecewise Cubic Bézier Curves Fitting. Pattern Recognition 31, 11 (1998), 1747 -- 1755. https://doi.org/10.1016/S0031--3203(98)00045--4Google ScholarGoogle ScholarCross RefCross Ref
  13. Fuhua Cheng and Chi-Cheng Lin. 1985. Clipping of Bézier Curves. In Proceedings of the 1985 ACM Annual Conference on The Range of Computing: Mid-80's Perspective (ACM '85). ACM, New York, NY, USA, 74--84. https://doi.org/10.1145/320435.320462Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. E. Cohen, R. F. Riesenfeld, and G. Elber. 2001. Geometric Modeling with Splines: An Introduction. A. K. Peters, Ltd., Natick, MA, United States.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Corel Corporation. 2019. CorelDRAW Graphics Suite. (2019). https://www.coreldraw.com/Google ScholarGoogle Scholar
  16. Victor A. Debelov and Aleksandr M. Matsokin. 2000. Implementation of Set Operations and Intersection of Bézier Curves. Computers & Graphics 24 (2000). Issue 1. https://doi.org/10.1016/s0097--8493(99)00137--5Google ScholarGoogle Scholar
  17. Qiulei Dong, Yihong Wu, and Zhanyi Hu. 2006. Gesture Recognition using Quadratic Curves. In Computer Vision(ACCV '06), P. J. Narayanan, Shree K. Nayar, and Heung-Yeung Shum (Eds.). Springer, Berlin, Heidelberg, 817--825.Google ScholarGoogle Scholar
  18. Arno Eigenwillig, Lutz Kettner, and Nicola Wolpert. 2007. Snap Rounding of Bézier Curves. In Proceedings of the 23rd ACM Annual Symposium on Computational Geometry (SCG '07). ACM, New York, NY, USA, 158--167. https://doi.org/10.1145/1247069.1247101Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Gerald Farin. 1993. Curves and Surfaces for Computer Aided Geometric Design: A Practical Guide(3 ed.). Academic Press Professional, Inc., San Diego, CA, USA.Google ScholarGoogle Scholar
  20. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. Simplicity: A Global Approach to Line Drawing Vectorization. ACM Transactions on Graphics 35, 4, Article 120 (July 2016), 10 pages. https://doi.org/10.1145/2897824.2925946Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Sebastian Feuerstack, Mauro Dos Santos Anjo, and Ednaldo B. Pizzolato. 2011. Model-based Design and Generation of a Gesture-based User Interface Navigation Control. In Proceedings of the 10th Brazilian Symposium on Human Factors in Computing Systems and the 5th Latin American Conference on Human-Computer Interaction (IHC+CLIHC '11). Brazilian Computer Society, Porto Alegre, Brazil, 227--231. http://dl.acm.org/citation.cfm?id=2254436.2254475Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Leah Findlater, Karyn Moffatt, Joanna McGrenere, and Jessica Dawson. 2009. Ephemeral Adaptation: The Use of Gradual Onset to Improve Menu Selection Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). Association for Computing Machinery, New York, NY, USA, 1655-1664. https://doi.org/10.1145/1518701.1518956Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Erich Gamma, Richard Helm, Ralph Johnson, and John M. Vlissides. 1994.Design Patterns: Elements of Reusable Object-Oriented Software(1 ed.). Addison-Wesley Professional. http://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612/ref=ntt_at_ep_dpi_1Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Bogdan-Florin Gheran, Jean Vanderdonckt, and Radu-Daniel Vatavu. 2018. Gestures for Smart Rings: Empirical Results,Insights, and Design Implications. In Proceedings of the ACM International Conference on Designing Interactive Systems(DIS '18). ACM, New York, NY, USA, 623--635.https://doi.org/10.1145/3196709.3196741Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Yves Mineur; Tony Lichah; Jean Marie Castelain; Henri Giaume. 1998. A shape controlled fitting method for Bãzier curves. Computer Aided Geometric Design 15 (1998). Issue 9. https://doi.org/10.1016/s0167--8396(98)00025--9Google ScholarGoogle Scholar
  26. Donatien Grolaux. 2020. Java Script version of Sketch A Doodle algorithms. (2020). https://github.com/GrolauxDonatien/SketchADoodleJSGoogle ScholarGoogle Scholar
  27. L. J. Guibas and J. Stolfi. 1982. A Language for Bitmap Manipulation.ACM Trans. Graph.1, 3 (July 1982), 191--214. https://doi.org/10.1145/357306.357308Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Kenrick Kin, Björn Hartmann, Tony DeRose, and Maneesh Agrawala. 2012. Proton++: A Customizable Declarative Multitouch Framework. In Proceedings of the 25th ACM Symposium on User Interface Software and Technology (UIST'12). ACM, New York, NY, USA, 477--486. https://doi.org/10.1145/2380116.2380176Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. P. P. Korovkin. 2001. Encyclopedia of Mathematics. Springer Science+Business Media/ Kluwer Academic Publishers, Berlin, Germany, Chapter Bernstein polynomials. https://www.encyclopediaofmath.org/index.php/Bernstein_polynomialsGoogle ScholarGoogle Scholar
  30. Gordon Kurtenbach, George Fitzmaurice, Thomas Baudel, and Bill Buxton. 1997. The Design of a GUI Paradigm Basedon Tablets, Two-hands, and Transparency. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '97). ACM, New York, NY, USA, 35--42. https://doi.org/10.1145/258549.258574Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers' Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, Article Paper 31, 12 pages. https://doi.org/10.1145/3173574.3173605Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Luis A. Leiva, Daniel Martín-Albo, and Réjean Plamondon. 2015. Gestures à Go Go: Authoring Synthetic Human-Like Stroke Gestures Using the Kinematic Theory of Rapid Movements. ACM Trans. Intell. Syst. Technol.7, 2, Article Article 15 (Nov. 2015), 29 pages. https://doi.org/10.1145/2799648Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. James R. Lewis. 1995. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7, 1 (1995), 57--78. https://doi.org/10.1080/10447319509526110arXiv:https://doi.org/10.1080/10447319509526110Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. James R. Lewis. 2002. Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies. International Journal of Human-Computer Interaction 14, 3--4 (2002), 463--488.https://doi.org/10.1080/10447318.2002.9669130arXiv:https://doi.org/10.1080/10447318.2002.9669130Google ScholarGoogle ScholarCross RefCross Ref
  35. James R. Lewis. 2006. Sample Sizes for Usability Tests: Mostly Math, Not Magic. interactions 13, 6 (Nov. 2006), 29--33.https://doi.org/10.1145/1167948.1167973Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Rensis Likert. 1932. A Technique for the Measurement of Attitudes.Archives of Psychology22, 140 (1932), 55--.http://psycnet.apa.org/record/1933-01885-001Google ScholarGoogle Scholar
  37. Jingbo Liu, Oscar Kin-Chung Au, Hongbo Fu, and Chiew-Lan Tai. 2012. Two-Finger Gestures for 6DOF Manipulation of 3D Objects. Computer Graphics Forum 31, 7pt1 (sep 2012), 2047--2055.https://doi.org/10.1111/j.1467--8659.2012.03197.xGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  38. Yingbin Liu and Stephen Mann. 2008. Parametric Triangular Bézier Surface Interpolation with Approximate Continuity. In Proceedings of the ACM Symposium on Solid and Physical Modeling (June 2008)(SPM '08). Association for Computing Machinery, New York, NY, USA, 381--387.https://doi.org/10.1145/1364901.1364956Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Hao Lü, James A. Fogarty, and Yang Li. 2014. Gesture Script: Recognizing Gestures and Their Structure Using Rendering Scripts and Interactively Trained Parts. In Proceedings of the 32nd ACM International Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1685--1694. https://doi.org/10.1145/2556288.2557263Google ScholarGoogle Scholar
  40. Hao Lü and Yang Li. 2013. Gesture Studio: Authoring Multi-touch Interactions Through Demonstration and Declaration. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 257--266.https://doi.org/10.1145/2470654.2470690Google ScholarGoogle Scholar
  41. Otto Parra, Sergio Espana, Jose Ignacio Panach, and Oscar Pastor. 2019. An empirical comparative evaluation of gestUI to include gesture-based interaction in user interfaces. Science of Computer Programming 172 (2019), 232 -- 263. https://doi.org/10.1016/j.scico.2018.12.001Google ScholarGoogle ScholarCross RefCross Ref
  42. Hartmut Prautzsch. 1989. A Round Trip to B-Splines via de Casteljau. ACM Trans. Graph. 8, 3 (July 1989), 243--254.https://doi.org/10.1145/77055.77061Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. O. Rashid and A. Al-Hamadi. 2015. Utilizing the Bézier Descriptors for Hand Gesture Recognition. In Proceedings of the IEEE International Conference on Image Processing (ICIP '15). 3525--3529. https://doi.org/10.1109/ICIP.2015.7351460Google ScholarGoogle Scholar
  44. Micha Sapek and Szczepan Paszkiel. 2017. Detection of Gestures Without Begin and End Markers by Fitting into Bézier Curves with Least Squares Method. Pattern Recognition Letters 100, C (dec 2017), 83--88. https://doi.org/10.1016/j.patrec.2017.10.006Google ScholarGoogle Scholar
  45. Jeff Sauro and James R. Lewis. 2016. Quantifying the User Experience. Practical Statistics for User Re-search(2 ed.). Morgan Kaufmann, Burlington, MA, USA.https://www.sciencedirect.com/book/9780123849687/quantifying-the-user-experienceGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  46. Ovidiu Andrei Schipor, Radu-Daniel Vatavu, and Jean Vanderdonckt. 2019. Euphoria: A Scalable, event-driven architecture for designing interactions across heterogeneous devices in smart environments.Information & Software Technology 109 (2019), 43--59. https://doi.org/10.1016/j.infsof.2019.01.006Google ScholarGoogle Scholar
  47. L. Shen, S. Huo, M. Chen, F. Li, F. Yao, and H. Shen. 2017. Multi-touch Gesture Recognition Algorithm of Vehicle Electronic Devices-based on Bézier Curve Optimization Strategy. In Proceedings of the 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC '17). 720--723. https://doi.org/10.1109/YAC.2017.7967503Google ScholarGoogle ScholarCross RefCross Ref
  48. Min C. Shin, V. Tsap, Leonid, and Dmitry B. Goldgof. 2004. Gesture Recognition using Bézier Curves for Visualization Navigation from Registered 3-D Data.Pattern Recognition37, 5 (2004), 1011--1024. https://doi.org/10.1016/j.patcog.2003.11.007Google ScholarGoogle Scholar
  49. Lucio Davide Spano, Antonio Cisternino, Fabio Paternò, and Gianni Fenu. 2013. GestIT: A Declarative and Com-positional Framework for Multiplatform Gesture Definition. In Proceedings of the 5th ACM International Symposium on Engineering Interactive Computing Systems (EICS '13). ACM, New York, NY, USA, 187--196. https://doi.org/10.1145/2494603.2480307Google ScholarGoogle Scholar
  50. Eugene M. Taranta, Andres N. Vargas, and Joseph J. LaViola. 2016. Streamlined and accurate gesture recognition with Penny Pincher. Computers and Graphics 55 (2016), 130 -- 142. https://doi.org/10.1016/j.cag.2015.10.011Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Inkscape Team. 2019. Inkscape - Draw Freely. (2019).https://inkscape.org/Google ScholarGoogle Scholar
  52. The GIMP Team. 2019. GIMP - GNU Image Manipulation. (2019).https://www.gimp.org/Google ScholarGoogle Scholar
  53. Jean Vanderdonckt, Paolo Roselli, and Jorge Luis Pérez-Medina. 2018. !FTL, an Articulation-Invariant Stroke Gesture Recognizer with Controllable Position, Scale, and Rotation Invariances. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI '18). ACM, New York, NY, USA, 125--134.https://doi.org/10.1145/3242969.3243032Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Radu-Daniel Vatavu. 2017. Improving Gesture Recognition Accuracy on Touch Screens for Users with Low Vision. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(May 2017)(CHI '17). Association for Computing Machinery, New York, NY, USA, 4667--4679.https://doi.org/10.1145/3025453.3025941Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2018. $Q: A Super-quick, Articulation-invariant Stroke-gesture Recognizer for Low-resource Devices. In Proceedings of the 20th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '18). ACM, New York, NY, USA, Article 23, 12 pages.https://doi.org/10.1145/3229434.3229465Google ScholarGoogle Scholar
  56. Julie R. Williamson, Stephen Brewster, and Rama Vennelakanti. 2013. Mo!Games: Evaluating Mobile Gestures in the Wild. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI '13). ACM, New York,NY, USA, 173--180. https://doi.org/10.1145/2522848.2522874Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Atsushi Yamada and Fujio Yamaguchi. 1996. Homogeneous Bounding Boxes as Tools for Intersection Algorithms of Rational Bézier Curves and Surfaces.The Visual Computer 12, 4 (01 Apr 1996), 202--214.https://doi.org/10.1007/BF01782323Google ScholarGoogle Scholar
  58. Chee K. Yap. 2006. Complete Subdivision Algorithms, I: Intersection of Bézier Curves. In Proceedings of the 22nd ACM Symposium on Computational Geometry (SCG '06). ACM, New York, NY, USA, 217--226.https://doi.org/10.1145/1137856.1137890Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Shumin Zhai, Per Kristensson, Caroline Appert, Tue Andersen, and Xiang Cao. 2012. Foundational Issues in Touch-Surface Stroke Gesture Design: An Integrative Review. Foundations of Trends in Human-Computer Interaction 5, 2 (feb 2012), 97--205. https://doi.org/10.1561/1100000012Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Lejun Shao; Hao Zhou. 1996. Curve Fitting with Bézier Cubics. Graphical Models and Image Processing58 (1996). Issue3. https://doi.org/10.1006/gmip.1996.0019Google ScholarGoogle Scholar

Index Terms

  1. SketchADoodle: Touch-surface Multi-stroke Gesture Handling by Bézier Curves

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!