StrokeAggregator: consolidating raw sketches into artist-intended curve drawings

Abstract

When creating line drawings, artists frequently depict intended curves using multiple, tightly clustered, or overdrawn, strokes. Given such sketches, human observers can readily envision these intended, aggregate, curves, and mentally assemble the artist's envisioned 2D imagery. Algorithmic stroke consolidation---replacement of overdrawn stroke clusters by corresponding aggregate curves---can benefit a range of sketch processing and sketch-based modeling applications which are designed to operate on consolidated, intended curves. We propose StrokeAggregator, a novel stroke consolidation method that significantly improves on the state of the art, and produces aggregate curve drawings validated to be consistent with viewer expectations. Our framework clusters strokes into groups that jointly define intended aggregate curves by leveraging principles derived from human perception research and observation of artistic practices. We employ these principles within a coarse-to-fine clustering method that starts with an initial clustering based on pairwise stroke compatibility analysis, and then refines it by analyzing interactions both within and in-between clusters of strokes. We facilitate this analysis by computing a common 1D parameterization for groups of strokes via common aggregate curve fitting. We demonstrate our method on a large range of line drawings, and validate its ability to generate consolidated drawings that are consistent with viewer perception via qualitative user evaluation, and comparisons to manually consolidated drawings and algorithmic alternatives.

References

1. Rahul Arora, Ishan Darolia, Vinay P. Namboodiri, Karan Singh, and Adrien Bousseau. 2017. SketchSoup: Exploratory Ideation Using Design Sketches. Computer Graphics Forum (2017).
2. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: As-natural-as-possible Sketching System for Creating 3D Curve Models. In Proc. UIST. 151--160.
3. Nikhil Bansal, Avrim Blum, and Shuchi Chawla. 2004. Correlation clustering. Machine Learning 56, 1--3 (2004), 89--113.
4. Bin Bao and Hongbo Fu. 2012. Vectorizing line drawings with near-constant line width. In Proc. International Conference on Image Processing. 805--808.
5. Ilya Baran, Jaakko Lehtinen, and Jovan Popović. 2010. Sketching clothoid splines using shortest paths. In Computer Graphics Forum, Vol. 29. 655--664.
6. Pascal Barla, Joëlle Thollot, and François Sillion. 2005. Geometric Clustering for Line Drawing Simplification. In Proc. EGSR.
7. A. Bartolo, K. P. Camilleri, S. G. Fabri, J. C. Borg, and P. J. Farrugia. 2007. Scribbles to Vectors: Preparation of Scribble Drawings for CAD Interpretation. In Sketch-Based Interfaces and Modeling.
8. Thomas Baudel. 1994. A Mark-based Interaction Paradigm for Free-hand Drawing. In Proc. UIST. 185--192.
9. Mikhail Bessmeltsev, Will Chang, Nicholas Vining, Alla Sheffer, and Karan Singh. 2015. Modeling character canvases from cartoon drawings. ACM Trans. Graph. 34, 5 (2015), 162.
10. Mikhail Bessmeltsev, Nicholas Vining, and Alla Sheffer. 2016. Gesture3D: Posing 3D Characters via Gesture Drawings. ACM Trans. Graph. 35, 6 (2016).
11. Pengbo Bo, Gongning Luo, and Kuanquan Wang. 2016. A graph-based method for fitting planar B-spline curves with intersections. Journal of Computational Design and Engineering 3, 1 (2016), 14--23.
12. Ricardo J. G. B. Campello, Davoud Moulavi, Arthur Zimek, and Jörg Sander. 2015. Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection. ACM Trans. Knowl. Discov. Data 10, 1 (2015), 5:1--5:51.
13. Jiazhou Chen, Gael Guennebaud, Pascal Barla, and Xavier Granier. 2013. Non-Oriented MLS Gradient Fields. In Computer Graphics Forum, Vol. 32. 98--109.
14. Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Science Sinica 14 (1965), 1396--1400.
15. Jack Edmonds. 1967. Optimum branchings. J. Res. Nat. Bur. Standards 71B, 4 (1967), 233--240.
16. Koos Eissen and Roselien Steur. 2008. Sketching: Drawing Techniques for Product Designers. Bis Publishers.
17. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Trans. Graph. 35, 4 (2016), 120.
18. Mark Finch, John Snyder, and Hugues Hoppe. 2011. Freeform Vector Graphics with Controlled Thin-plate Splines. ACM Trans. Graph. 30, 6 (2011), 166:1--166:10.
19. Stéphane Grabli, Frédo Durand, and François Sillion. 2004. Density Measure for Line-Drawing Simplification. In Proc. Pacific Graphics.
20. Cindy Grimm and Pushkar Joshi. 2012. Just DrawIt: A 3D Sketching System. In Proc. SBIM. 121--130.
21. Robert Hess and David Field. 1999. Integration of contours: new insights. Trends in Cognitive Sciences 3, 12 (1999), 480--486.
22. Xavier Hilaire and Karl Tombre. 2006. Robust and accurate vectorization of line drawings. IEEE Trans. Pattern Anal. Mach. Intell 28, 6 (2006), 890--904.
23. Takeo Igarashi, Tomer Moscovich, and John F. Hughes. 2005. As-rigid-as-possible Shape Manipulation. ACM Trans. Graph. 24, 3 (2005), 1134--1141.
24. Pradeep Kumar Jayaraman, Chi-Wing Fu, Jianmin Zheng, Xueting Liu, and Tien-Tsin Wong. 2017. Globally Consistent Wrinkle-Aware Shading of Line Drawings. IEEE Trans. Vis. Comput. Graph (2017).
25. Robert D. Kalnins, Philip L. Davidson, Lee Markosian, and Adam Finkelstein. 2003. Coherent Stylized Silhouettes. ACM Trans. Graph. 22, 3 (2003), 856--861.
26. Margret Keuper, Evgeny Levinkov, Nicolas Bonneel, Guillaume Lavoué, Thomas Brox, and Bjorn Andres. 2015. Efficient decomposition of image and mesh graphs by lifted multicuts. In Proc. ICCV. 1751--1759.
27. In-Kwon Lee. 2000. Curve reconstruction from unorganized points. Computer aided geometric design 17, 2 (2000), 161--177.
28. David Levin. 2004. Mesh-independent surface interpolation. In Geometric modeling for scientific visualization. 37--49.
29. H Lipson and M Shpitalni. 1996. Optimization-based reconstruction of a 3D object from a single freehand line drawing. Computer-Aided Design 28, 8 (1996), 651 -- 663.
30. Xueting Liu, Tien-Tsin Wong, and Pheng-Ann Heng. 2015. Closure-aware sketch simplification. ACM Trans. Graph. 34, 6 (2015), 168.
31. James McCrae and Karan Singh. 2009. Sketching piecewise clothoid curves. Computers & Graphics 33, 4 (2009), 452--461.
32. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011a. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. Graph. 30, 6 (2011).
33. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011b. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. Graph. 30, 6 (2011). 185:1--185:10.
34. Gioacchino Noris, Alexander Hornung, Robert W Sumner, Maryann Simmons, and Markus Gross. 2013. Topology-driven vectorization of clean line drawings. ACM Trans. Graph. 32, 1 (2013), 4.
35. Gioacchino Noris, Daniel Sỳkora, A Shamir, Stelian Coros, Brian Whited, Maryann Simmons, Alexander Hornung, M Gross, and R Sumner. 2012. Smart scribbles for sketch segmentation. In Computer Graphics Forum, Vol. 31. 2516--2527.
36. Gunay Orbay and Levent Burak Kara. 2011. Beautification of design sketches using trainable stroke clustering and curve fitting. IEEE Trans. Vis. Comput. Graph 17, 5 (2011), 694--708.
37. Paul L. Rosin. 1994. Grouping Curved Lines. In Machine Graphics and Vision 7. 625--644.
38. Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh. 2012. CrossShade: Shading Concept Sketches Using Cross-section Curves. ACM Trans. Graph. 31, 4 (2012), 45:1--45:11.
39. Amit Shesh and Baoquan Chen. 2008. Efficient and dynamic simplification of line drawings. In Computer Graphics Forum, Vol. 27. 537--545.
40. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2016. Learning to simplify: fully convolutional networks for rough sketch cleanup. ACM Trans. Graph. 35, 4 (2016), 121.
41. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2017. Mastering Sketching: Adversarial Augmentation for Structured Prediction. ACM Trans. Graph. (2017).
42. O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rössl, and H.-P. Seidel. 2004. Laplacian Surface Editing. In Proc. Symposium on Geometry Processing. 175--184.
43. Johan Wagemans, James H Elder, Michael Kubovy, Stephen E Palmer, Mary A Peterson, Manish Singh, and Rüdiger von der Heydt. 2012. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological bulletin 138, 6 (2012), 1172.
44. Brett Wilson and Kwan-Liu Ma. 2004. Rendering complexity in computer-generated pen-and-ink illustrations. In Proc. NPAR. 129--137.
45. Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D Curve Networks from 2D Sketches via Selective Regularization. ACM Trans. Graph. 33, 4 (2014), 131:1--131:13.
46. Pengfei Xu, Hongbo Fu, Oscar Kin-Chung Au, and Chiew-Lan Tai. 2012. Lazy selection: a scribble-based tool for smart shape elements selection. ACM Trans. Graph. 31, 6 (2012), 142.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation