Vectorization of Line Drawings via Polyvector Fields

Abstract

Image tracing is a foundational component of the workflow in graphic design, engineering, and computer animation, linking hand-drawn concept images to collections of smooth curves needed for geometry processing and editing. Even for clean line drawings, modern algorithms often fail to faithfully vectorize junctions, or points at which curves meet; this produces vector drawings with incorrect connectivity. This subtle issue undermines the practical application of vectorization tools and accounts for hesitance among artists and engineers to use automatic vectorization software. To address this issue, we propose a novel image vectorization method based on state-of-the-art mathematical algorithms for frame field processing. Our algorithm is tailored specifically to disambiguate junctions without sacrificing quality.

References

1. David Adalsteinsson and James A. Sethian. 1995. A fast level set method for propagating interfaces. J. Comput. Phys. 118, 2 (1995), 269--277.
2. Alexandra Bartolo, Kenneth P. Camilleri, Simon G. Fabri, Jonathan C. Borg, and Philip J. Farrugia. 2007. Scribbles to vectors: Preparation of scribble drawings for CAD interpretation. Proceedings of the 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling (SBIM’07), 123--130.
3. Mikhail Bessmeltsev, Will Chang, Nicholas Vining, Alla Sheffer, and Karan Singh. 2015. Modeling character canvases from cartoon drawings. ACM Trans. Graph. 34, 5, Article 162 (Nov. 2015), 16 pages.
4. Pengbo Bo, Gongning Luo, and Kuanquan Wang. 2016. A graph-based method for fitting planar B-spline curves with intersections. J. Comput. Design Eng. 3, 1 (2016), 14--23.
5. Dengfeng Chai, Wolfgang Förstner, and Florent Lafarge. 2013. Recovering line-networks in images by junction-point processes. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 1894--1901.
6. Jiazhou Chen, GaÃńl Guennebaud, Pascal Barla, and Xavier Granier. 2013. Non-oriented MLS gradient fields. Comput. Graph. Forum 32, 8 (2013), 98--109. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.12164
7. JiaZhou Chen, Qi Lei, YongWei Miao, and QunSheng Peng. 2015. Vectorization of line drawing image based on junction analysis. Sci. China Inf. Sci. 58, 7 (2015), 1--14.
8. Olga Diamanti, Amir Vaxman, Daniele Panozzo, and Olga Sorkine-Hornung. 2015. Integrable PolyVector fields. ACM Trans. Graph. 34, 4, Article 38 (July 2015), 12 pages.
9. Luca Donati, Simone Cesano, and Andrea Prati. 2017. An accurate system for fashion hand-drawn sketches vectorization. In The IEEE International Conference on Computer Vision (ICCV’17). 2280--2286.
10. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. simplicity: A global approach to line drawing vectorization. ACM Trans. Graph. (SIGGRAPH Conf. Proc.) 35, 4, Article 120 (July 2016), 10 pages.
11. Fernando de Goes, David Cohen-Steiner, Pierre Alliez, and Mathieu Desbrun. 2011. An optimal transport approach to robust reconstruction and simplification of 2D shapes. Comput. Graph. Forum 30, 5 (July 2011), 1593--1602.
12. Chris Harris and Mike Stephens. 1988. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference. 147--151.
13. Aaron Hertzmann and Denis Zorin. 2000. Illustrating smooth surfaces. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’00). ACM Press/Addison-Wesley Publishing, New York, 517--526.
14. Xavier Hilaire and Karl Tombre. 2006. Robust and accurate vectorization of line drawings. IEEE Trans. Pattern Anal. Machine Intell. 28, 6 (June 2006), 890--904.
15. Emmanuel Iarussi, David Bommes, and Adrien Bousseau. 2015. BendFields: Regularized curvature fields from rough concept sketches. ACM Trans. Graph. 34, 3, Article 24 (May 2015), 16 pages.
16. Evangelos Kalogerakis, Derek Nowrouzezahrai, Simon Breslav, and Aaron Hertzmann. 2012. Learning hatching for pen-and-ink illustration of surfaces. ACM Trans. Graph. 31, 1 (2012), 17.
17. Henry Kang, Seungyong Lee, and Charles K. Chui. 2007. Coherent line drawing. In Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering (NPAR’07). ACM, New York, 43--50.
18. Gaetano Kanizsa. 1979. Organization in Vision: Essays on Gestalt Perception. Praeger, New York.
19. Michael Kass and Andrew Witkin. 1987. Analyzing oriented patterns. Comput. Vision Graph. Image Process. 37, 3 (March 1987), 362--385.
20. Gregory Lecot and Bruno Lévy. 2006. Ardeco: Automatic region detection and conversion. In Proceedings of the 17th Eurographics Conference on Rendering Techniques (EGSR’06). Eurographics Association, Aire-la-Ville, Switzerland, 349--360.
21. Jorge Nocedal and Stephen J. Wright. 2006. Numerical Optimization (2nd ed.). Springer, New York.
22. Gioacchino Noris, Alexander Hornung, Robert W. Sumner, Maryann Simmons, and Markus Gross. 2013. Topology-driven vectorization of clean line drawings. ACM Trans. Graph. 32, 1, Article 4 (Feb. 2013), 11 pages.
23. Alexandrina Orzan, Adrien Bousseau, Pascal Barla, Holger Winnemöller, Joëlle Thollot, and David Salesin. 2013. Diffusion curves: A vector representation for smooth-shaded images. Commun. ACM 56, 7 (July 2013), 101--108.
24. Kazim Pal, Nicole Avery, Pete Boston, Alberto Campagnolo, Caroline De Stefani, Helen Matheson-Pollock, Daniele Panozzo, Matthew Payne, Christian Schüller, Chris Sanderson, Chris Scott, Philippa Smith, Rachael Smither, Olga Sorkine-Hornung, Ann Stewart, Emma Stewart, Patricia Stewart, Melissa Terras, Bernadette Walsh, Laurence Ward, Liz Yamada, and Tim Weyrich. 2017. Digitally reconstructing the great parchment book: 3D recovery of fire-damaged historical documents. Literary Linguistic Comput. J. Digital Scholarship Humanities 32, 4 (2017), 887--917.
25. Jonathan Palacios and Eugene Zhang. 2007. Rotational symmetry field design on surfaces. ACM Trans. Graph. 26, 3, Article 55 (July 2007).
26. Daniele Panozzo, Enrico Puppo, Marco Tarini, and Olga Sorkine-Hornung. 2014. Frame fields: Anisotropic and non-orthogonal cross fields. ACM Trans. Graph. 33, 4, Article 134 (July 2014), 11 pages.
27. Yixuan Qiu, Naoaki Okazaki, and Jorge Nocedal. 2016. LBFGS++, A Header-only C++ Library for L-BFGS Algorithm. Retrieved from https://yixuan.cos.name/LBFGSpp/.
28. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2016. Learning to simplify: Fully convolutional networks for rough sketch cleanup. ACM Trans. Graph. 35, 4, Article 121 (July 2016), 11 pages.
29. Richard Szeliski. 2010. Computer Vision: Algorithms and Applications. Springer-Verlag New York, New York.
30. Engin Türetken, Fethallah Benmansour, Bjoern Andres, Hanspeter Pfister, and Pascal Fua. 2013. Reconstructing loopy curvilinear structures using integer programming. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 1822--1829.
31. Amir Vaxman, Marcel Campen, Olga Diamanti, Daniele Panozzo, David Bommes, Klaus Hildebrandt, and Mirela Ben-Chen. 2016. Directional field synthesis, design, and processing. Comput. Graph. Forum 35, 2 (2016), 545--572. Retrieved from http://graphics.tudelft.nl/Publications-new/2016/VCDPBHB16.
32. Brian Whited, Gioacchino Noris, Maryann Simmons, Robert W. Sumner, Markus Gross, and Jarek Rossignac. 2010. BetweenIT: An interactive tool for tight inbetweening. Comput. Graph. Forum 29, 2 (2010), 605--614.
33. Gui-Song Xia, Julie Delon, and Yann Gousseau. 2014. Accurate Junction Detection and Characterization in Natural Images. Vol. 106. Kluwer Academic Publishers, Hingham, MA. 31--56 pages.
34. Eugene Zhang, James Hays, and Greg Turk. 2007. Interactive tensor field design and visualization on surfaces. IEEE Trans. Visual. Comput Graph. 13, 1 (Jan. 2007), 94--107.
35. Song-Hai Zhang, Tao Chen, Yi-Fei Zhang, Shi-Min Hu, and Ralph R. Martin. 2009. Vectorizing cartoon animations. IEEE Trans. Visual. Comput. Graph. 15, 4 (July 2009), 618--629.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Reconstruction

  • Computer graphics

    • Shape modeling

      • Parametric curve and surface models