Tracing versus freehand for evaluating computer-generated drawings

Abstract

Non-photorealistic rendering (NPR) and image processing algorithms are widely assumed as a proxy for drawing. However, this assumption is not well assessed due to the difficulty in collecting and registering freehand drawings. Alternatively, tracings are easier to collect and register, but there is no quantitative evaluation of tracing as a proxy for freehand drawing. In this paper, we compare tracing, freehand drawing, and computer-generated drawing approximation (CGDA) to understand their similarities and differences. We collected a dataset of 1,498 tracings and freehand drawings by 110 participants for 100 image prompts. Our drawings are registered to the prompts and include vector-based timestamped strokes collected via stylus input. Comparing tracing and freehand drawing, we found a high degree of similarity in stroke placement and types of strokes used over time. We show that tracing can serve as a viable proxy for freehand drawing because of similar correlations between spatio-temporal stroke features and labeled stroke types. Comparing hand-drawn content and current CGDA output, we found that 60% of drawn pixels corresponded to computer-generated pixels on average. The overlap tended to be commonly drawn content, but people's artistic choices and temporal tendencies remained largely uncaptured. We present an initial analysis to inform new CGDA algorithms and drawing applications, and provide the dataset for use by the community.

References

1. Pablo Arbeláez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour Detection and Hierarchical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 5 (May 2011), 898--916.
2. Itamar Berger, Ariel Shamir, Moshe Mahler, Elizabeth Carter, and Jessica Hodgins. 2013. Style and Abstraction in Portrait Sketching. ACM Trans. Graph. 32, 4, Article 55 (July 2013), 12 pages.
3. John Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 6 (Nov 1986), 679--698.
4. Forrester Cole, Aleksey Golovinskiy, Alex Limpaecher, Heather Stoddart Barros, Adam Finkelstein, Thomas Funkhouser, and Szymon Rusinkiewicz. 2008. Where Do People Draw Lines? ACM Trans. Graph. 27, 3, Article 88 (Aug 2008), 11 pages.
5. Richard D. De Veaux, Paul F. Velleman, and David E. Bock. 2020. Stats: Data and Models, 5th Edition. Pearson.
6. Doug DeCarlo, Adam Finkelstein, Szymon Rusinkiewicz, and Anthony Santella. 2003. Suggestive Contours for Conveying Shape. ACM Trans. Graph. 22, 3 (July 2003), 848--855.
7. Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, and Adrien Bousseau. 2018. 3D Sketching Using Multi-View Deep Volumetric Prediction. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 21 (July 2018), 22 pages.
8. Betty Edwards. 2012. Drawing on the Right Side of the Brain. Penguin Group.
9. Mathias Eitz, James Hays, and Marc Alexa. 2012. How Do Humans Sketch Objects? ACM Trans. Graph. 31, 4, Article 44 (July 2012), 10 pages.
10. Lubin Fan, Ruimin Wang, Linlin Xu, Jiansong Deng, and Ligang Liu. 2013. Modeling by Drawing with Shadow Guidance. Computer Graphics Forum 32, 7 (2013), 157--166.
11. Hongbo Fu, Shizhe Zhou, Ligang Liu, and Niloy J. Mitra. 2011. Animated Construction of Line Drawings. ACM Trans. Graph. 30, 6, Article 133 (Dec 2011), 10 pages.
12. Emma Gowen and R. Chris Miall. 2006. Eye-Hand Interactions in Tracing and Drawing Tasks. Human Movement Science 25, 4(2006), 568--585.
13. Emma Gowen and R. Chris Miall. 2007. Differentiation Between External and Internal Cuing: An fMRI Study Comparing Tracing with Drawing. NeuroImage 36, 2 (2007), 396--410.
14. Yulia Gryaditskaya, Mark Sypesteyn, Jan Willem Hoftijzer, Sylvia Pont, Frédo Durand, and Adrien Bousseau. 2019. OpenSketch: A Richly-Annotated Dataset of Product Design Sketches. ACM Trans. Graph. 38, 6, Article 232 (Nov 2019), 16 pages.
15. David Ha and Douglas Eck. 2018. A Neural Representation of Sketch Drawings. In Proceedings of the International Conference on Learning Representations.
16. Xiaoguang Han, Chang Gao, and Yizhou Yu. 2017. DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling. ACM Trans. Graph. 36, 4, Article 126 (July 2017), 12 pages.
17. Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2017. Shape Synthesis from Sketches via Procedural Models and Convolutional Networks. IEEE Transactions on Visualization and Computer Graphics 23, 8 (Aug 2017), 2003--2013.
18. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-To-Image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
19. David Jamieson. 2019. Why Learn to Draw When You Can Trace? https://vitruvianstudio.com/why-learn-to-draw-when-you-can-trace/. Accessed Aug 9, 2020.
20. Christian Mosbæk Johannessen and Theo Van Leeuwen. 2017. The Materiality of Writing: A Trace Making Perspective. Routledge.
21. Tilke Judd, Frédo Durand, and Edward Adelson. 2007. Apparent Ridges for Line Drawing. ACM Trans. Graph. 26, 3, Article 19 (July 2007), 8 pages.
22. Evangelos Kalogerakis, Derek Nowrouzezahrai, Simon Breslav, and Aaron Hertzmann. 2012. Learning Hatching for Pen-and-Ink Illustration of Surfaces. ACM Trans. Graph. 31, 1, Article 1 (Feb 2012), 17 pages.
23. Kitware. 2020. Insight Toolkit. https://itk.org/. Accessed Oct 20, 2020.
24. Yong Jae Lee, C. Lawrence Zitnick, and Michael F. Cohen. 2011. ShadowDraw: Real-Time User Guidance for Freehand Drawing. ACM Trans. Graph. 30, 4, Article 27 (July 2011), 10 pages.
25. Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2018. Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface Modeling. ACM Trans. Graph. 37, 6, Article 238 (Dec 2018), 12 pages.
26. Mengtian Li, Zhe Lin, Radomir Mech, Ersin Yumer, and Deva Ramanan. 2019. Photo-Sketching: Inferring Contour Drawings from Images. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. 1403--1412.
27. Alex Limpaecher, Nicolas Feltman, Adrien Treuille, and Michael Cohen. 2013. Real-Time Drawing Assistance through Crowdsourcing. ACM Trans. Graph. 32, 4, Article 54 (July 2013), 8 pages.
28. Difan Liu, Mohamed Nabail, Aaron Hertzmann, and Evangelos Kalogerakis. 2020. Neural Contours: Learning to Draw Lines from 3D Shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
29. Jingbo Liu, Hongbo Fu, and Chiew-Lan Tai. 2014. Dynamic Sketching: Simulating the Process of Observational Drawing. In Proceedings of the Workshop on Computational Aesthetics. ACM, New York, NY, USA, 15--22.
30. Cewu Lu, Li Xu, and Jiaya Jia. 2012. Combining Sketch and Tone for Pencil Drawing Production. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering. Eurographics Association, Goslar, Germany, 65--73.
31. Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks. In Proceedings of the International Conference on 3D Vision. 67--77.
32. James M Mahoney. 2018. The V-Sketch System: Machine Assisted Design Exploration in Virtual Reality. Master's thesis. Cornell University.
33. Yutaka Ohtake, Alexander Belyaev, and Hans-Peter Seidel. 2004. Ridge-Valley Lines on Meshes via Implicit Surface Fitting. ACM Trans. Graph. 23, 3 (Aug 2004), 609--612.
34. Justin Ostrofsky, Aaron Kozbelt, and Angelika Seidel. 2012. Perceptual Constancies and Visual Selection as Predictors of Realistic Drawing Skill. Psychology of Aesthetics, Creativity, and the Arts 6, 2 (2012), 124.
35. Emil Praun, Hugues Hoppe, Matthew Webb, and Adam Finkelstein. 2001. Real-time Hatching. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, NY, USA, 579--584.
36. Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. 2016. The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies. ACM Trans. Graph. 35, 4, Article 119 (July 2016), 12 pages.
37. Boxin Shi, Zhe Wu, Zhipeng Mo, Dinglong Duan, Sai-Kit Yeung, and Ping Tan. 2016. A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3707--3716.
38. Miroslava Slavcheva, Wadim Kehl, Nassir Navab, and Slobodan Ilic. 2018. SDF-2-SDF Registration for Real-Time 3D Reconstruction from RGB-D Data. International Journal of Computer Vision 126, 6 (June 2018), 615--636.
39. Qingkun Su, Wing Ho Andy Li, Jue Wang, and Hongbo Fu. 2014. EZ-Sketching: Three-Level Optimization for Error-Tolerant Image Tracing. ACM Trans. Graph. 33, 4, Article 54 (July 2014), 9 pages.
40. Wanchao Su, Dong Du, Xin Yang, Shizhe Zhou, and Hongbo Fu. 2018. Interactive Sketch-Based Normal Map Generation with Deep Neural Networks. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 22 (July 2018), 17 pages.
41. Masaki Suwa and Barbara Tversky. 1997. What Do Architects and Students Perceive in Their Design Sketches? A Protocol Analysis. Design Studies 18, 4 (1997), 385--403.
42. Corey Toler-Franklin, Adam Finkelstein, and Szymon Rusinkiewicz. 2007. Illustration of Complex Real-World Objects Using Images with Normals. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering. ACM, New York, NY, USA, 111-119.
43. Holger Winnemöller, Jan Eric Kyprianidis, and Sven C. Olsen. 2012. XDoG: An Extended Difference-of-Gaussians Compendium Including Advanced Image Stylization. Computers & Graphics 36, 6 (2012), 740--753.
44. Saining Xie and Zhuowen Tu. 2015. Holistically-Nested Edge Detection. In Proceedings of the IEEE International Conference on Computer Vision. 1395--1403.
45. Ying Xiong, Ayan Chakrabarti, Ronen Basri, Steven J. Gortler, David W. Jacobs, and Todd Zickler. 2015. From Shading to Local Shape. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1 (Jan 2015), 67--79.
46. Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A Benchmark for Rough Sketch Cleanup. ACM Trans. Graph. 39, 6, Article 163 (Nov 2020), 14 pages.
47. Evan You. 2020. Vue.js. https://vuejs.org/. Accessed Oct 20, 2020.
48. Li Zhang. 2012. Photometric Stereo. http://pages.cs.wisc.edu/~lizhang/courses/cs766-2012f/projects/phs/index.htm. Accessed Apr 12, 2019.

Index Terms

• Applied computing

  • Arts and humanities

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Perception

    • Rendering

      • Non-photorealistic rendering

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User studies