Shading Rig: Dynamic Art-directable Stylised Shading for 3D Characters

Abstract

Despite the popularity of three-dimensional (3D) animation techniques, the style of 2D cel animation is seeing increased use in games and interactive applications. However, conventional 3D toon shading frequently requires manual editing to clean up undesired shadows or add stylistic details based on art direction. This editing is impractical for the frame-by-frame editing in cartoon feature film post-production. For interactive stylised media and games, post-production is unavailable due to real-time constraints, so art-direction must be preserved automatically. For these reasons, artists often resort to mesh and texture edits to mitigate undesired shadows typical of toon shaders. Such edits allow real-time rendering but are limited in resolution, animation quality and lack detail control for stylised shadow design.

In our framework, artists build a “shading rig,” a collection of these edits, that allows artists to animate toon shading. Artists pre-animate the shading rig under changing lighting, to dynamically preserve artistic intent in a live application, without manual intervention. We show our method preserves continuous motion and shape interpolation, with fewer keyframes than previous work. Our shading shape interpolation is computationally cheaper than state-of-the-art image interpolation techniques. We achieve these improvements while preserving vector quality rendering, without resorting either to high texture resolution or mesh density.

References

1. Ken Anjyo and Katsuaki Hiramitsu. 2003. Stylized highlights for cartoon rendering and animation. IEEE Comput. Graph. Appl. 23, 4 (2003), 54–61.
2. Muhammad Arief, Kunio Kondo, Koji Mikami, Hideki Todo, and Yasushi Yamaguchi. 2015. Controllable region via texture projection for stylized shading. In Proceedings of the 14th ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCAI'15). ACM, New York, NY, 35–38. https://doi.org/10.1145/2817675.2817688
3. Pascal Barla, Joëlle Thollot, and Lee Markosian. 2006. X-toon: An extended toon shader. In Proceedings of the 4th International Symposium on Non-photorealistic Animation and Rendering. ACM, 127–132.
4. Pierre Bénard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden. 2013. Stylizing animation by example. ACM Trans. Graph. 32, 4 (2013), 119.
5. Pierre Bénard, Aaron Hertzmann, and Michael Kass. 2014. Computing smooth surface contours with accurate topology. ACM Trans. Graph. 33, 2, Article 19 (2014). https://doi.org/10.1145/2558307
6. Jeremy Birn. 2000. Digital Lighting & Rendering. New Riders Press.
7. Nicolas Bonneel, Michiel Van De Panne, Sylvain Paris, and Wolfgang Heidrich. 2011. Displacement interpolation using Lagrangian mass transport. ACM Trans. Graph. 30, 6, Article 158 (2011).
8. Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2018. The design and evolution of disney's hiperion renderer. ACM Trans. Graph. 37, 3 (2018), 33.
9. Mark Dokter, Jozef Hladky, Mathias Parger, Dieter Schmalstieg, Hans-Peter Seidel, and Markus Steinberger. 2019. Hierarchical rasterization of curved primitives for vector graphics rendering on the GPU. Comput. Graph. Forum 38, 2 (2019), 93–103. https://doi.org/10.1111/cgf.13622
10. Elmar Eisemann, Holger Winnemöller, John C. Hart, and David Salesin. 2008. Stylized vector art from 3D models with region support. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 1199–1207.
11. Jakub Fišer, Ondřej Jamriška, Michal Lukáč, Eli Shechtman, Paul Asente, Jingwan Lu, and Daniel Sỳkora. 2016. StyLit: Illumination-guided example-based stylization of 3D renderings. ACM Trans. Graph. 35, 4 (2016), 92.
12. Julien Guertault, Élie Setbon, and Pavel Martishevsky. 2018. Manga stylized rendering in VR. In Proceedings of the SIGGRAPH Asia 2018 Courses (SA'18). ACM, New York, NY, Article 13, 61 pages. https://doi.org/10.1145/3277644.3277768
13. Aaron Hertzmann, Charles E. Jacobs, Nuria Oliver, Brian Curless, and David H. Salesin. 2001. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 327–340.
14. Burne Hogarth. 1991. Dynamic Light and Shade.ERIC.
15. Matis Hudon, Mairéad Grogan, Rafael Pagés, Jan Ondřej, and Aljoša Smolić. 2019. 2DToonShade: A stroke based toon shading system. Comput. Graph.: X 1 (2019), 100003.
16. Ondřej Jamriška, Šárka Sochorová, Ondřej Texler, Michal Lukáč, Jakub Fišer, Jingwan Lu, Eli Shechtman, and Daniel Sỳkora. 2019. Stylizing video by example. ACM Trans. Graph. 38, 4 (2019), 107.
17. Robert D. Kalnins, Lee Markosian, Barbara J. Meier, Michael A. Kowalski, Joseph C. Lee, Philip L. Davidson, Matthew Webb, John F. Hughes, and Adam Finkelstein. 2002. WYSIWYG NPR: Drawing strokes directly on 3D models. ACM Trans. Graph. 21, 3 (Jul. 2002), 755–762.
18. Yosuke Katsura and Ken Anjyo. 2007. Anime perspective. In ACM SIGGRAPH 2007 Sketches. ACM, 75.
19. William B. Kerr and Fabio Pellacini. 2009. Toward evaluating lighting design interface paradigms for novice users. ACM Trans. Graph. 28, 3, Article 26 (Jul. 2009), 9 pages. https://doi.org/10.1145/1531326.1531332
20. Mark J. Kilgard and Jeff Bolz. 2012. GPU-accelerated path rendering. ACM Trans. Graph. 31, 6 (2012), 1–10.
21. Adam Lake, Carl Marshall, Mark Harris, and Marc Blackstein. 2000. Stylized rendering techniques for scalable real-time 3d animation. In Proceedings of the 1st International Symposium on Non-photorealistic Animation and Rendering. ACM, 13–20.
22. David Landau. 2014. Lighting for Cinematography: A Practical Guide to the Art and Craft of Lighting for the Moving Image. A&C Black.
23. Henrik Lieng, Flora P. Tasse, Jiří Kosinka, and Neil A. Dodgson. 2015. Shading curves: Vector-based drawing with explicit gradient control. Comput. Graph. Forum (2015).
24. Charles Loop and Jim Blinn. 2005. Resolution independent curve rendering using programmable graphics hardware. ACM Trans. Graph. 24, 3 (2005), 1000–1009.
25. Kris Malkiewicz and M. David Mullen. 2009. Cinematography. Simon & Schuster.
26. O. Mattausch, T. Igarashi, and M. Wimmer. 2013. Freeform shadow boundary editing. Comput. Graph. Forum 32, 2pt2 (2013), 175–184. https://doi.org/10.1111/cgf.12037
27. Scott McCloud. 1993. Understanding Comics: The Invisible Art.
28. Junya Christopher Motomura. 2015. GuiltyGearXrd's Art Style : The X Factor Between 2D and 3D. Retrieved from www.gdcvault.com/play/1022031/GuiltyGearXrd-s-Art-Style-The.
29. Georges Nader and Gael Guennebaud. 2018. Instant transport maps on 2D grids. ACM Trans. Graph. 37, 6 (2018), 13.
30. M. Okabe, Y. Matsushita, L. Shen, and T. Igarashi. 2007. Illumination brush: Interactive design of all-frequency lighting. In Proceedings of the 15th Pacific Conference on Computer Graphics and Applications (PG'07). 171–180. https://doi.org/10.1109/PG.2007.9
31. Romain Pacanowski, Xavier Granier, Christophe Schlick, and Pierre Poulin. 2008. Sketch and paint-based interface for highlight modeling. In Proceedings of the 5th Eurographics Conference on Sketch-Based Interfaces and Modeling (SBM'08). Eurographics Association, Goslar, DEU, 17–23.
32. Fabio Pellacini, Parag Tole, and Donald P. Greenberg. 2002. A user interface for interactive cinematic shadow design. ACM Trans. Graph. 21, 3 (2002), 563–566.
33. Gabriel Peyré, Marco Cuturi, et al. 2019. Computational optimal transport: With applications to data science. Found. Trends Mach. Learn. 11, 5-6 (2019), 355–607.
34. Nicolas Ray, Xavier Cavin, and Bruno Lévy. 2005. Vector Texture Maps on the GPU. Inst. ALICE (Algorithms, Comput., Geometry Image Dept. INRIA Nancy Grand-Est/Loria), Technical Report ALICE-TR-05-003 (2005).
35. Tobias Ritschel, Thorsten Thormählen, Carsten Dachsbacher, Jan Kautz, and Hans-Peter Seidel. 2010. Interactive on-surface signal deformation. ACM Trans. Graph. 29, 4, Article 36 (Jul. 2010), 8 pages. https://doi.org/10.1145/1778765.1778773
36. Thorsten-Walther Schmidt, Fabio Pellacini, Derek Nowrouzezahrai, Wojciech Jarosz, and Carsten Dachsbacher. 2016. State of the art in artistic editing of appearance, lighting and material. Comput. Graph. Forum 35, 1 (2016), 216–233.
37. Dario Seyb, Alec Jacobson, Derek Nowrouzezahrai, and Wojciech Jarosz. 2019. Non-linear sphere tracing for rendering deformed signed distance fields. ACM Trans. Graph. 38, 6, Article 229 (Nov. 2019), 12 pages. https://doi.org/10.1145/3355089.3356502
38. Zhixin Shu, Sunil Hadap, Eli Shechtman, Kalyan Sunkavalli, Sylvain Paris, and Dimitris Samaras. 2018. Portrait lighting transfer using a mass transport approach. ACM Trans. Graph. 37, 1 (2018), 2.
39. Steve Sint. 2009. Digital Portrait Photography: Art, Business and Style. Sterling Publishing Company, Inc.
40. Peter-Pike J. Sloan, William Martin, Amy Gooch, and Bruce Gooch. 2001. The lit sphere: A model for capturing NPR shading from art. In Graphics Interface, Vol. 2001. 143–150.
41. Justin Solomon, Fernando De Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, and Leonidas Guibas. 2015. Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Trans. Graph. 34, 4 (2015), 66.
42. Xin Sun, Guofu Xie, Yue Dong, Stephen Lin, Weiwei Xu, Wencheng Wang, Xin Tong, and Baining Guo. 2012. Diffusion curve textures for resolution independent texture mapping.ACM Trans. Graph. 31, 4, Article 74 (2012).
43. Daniel Sýkora, Ondřej Jamriška, Ondřej Texler, Jakub Fišer, Michal Lukáč, Jingwan Lu, and Eli Shechtman. 2019. StyleBlit: Fast example-based stylization with local guidance. Comput. Graph. Forum 38, 2 (2019), 83–91.
44. Mumehiro Tada, Yoshinori Dobashi, and Tsuyoshi Yamamoto. 2012. Feature-based interpolation for the interactive editing of shading effects. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. 47–50.
45. Hideki Todo, Ken Anjyo, William Baxter, and Takeo Igarashi. 2007. Locally controllable stylized shading. ACM Trans. Graph. 26, 3, Article 17 (Jul. 2007). https://doi.org/10.1145/1276377.1276399
46. Hideki Todo, Ken Anjyo, and Shun'Ichi Yokoyama. 2013. Lit-sphere extension for artistic rendering. Vis. Comput. 29, 6-8 (Jun. 2013), 473–480. https://doi.org/10.1007/s00371-013-0811-7
47. Chad Troftgruben. 2014. Learning Anime Studio. Packt Publishing.
48. Yu-Ting Tsai and Zen-Chung Shih. 2006. All-frequency precomputed radiance transfer using spherical radial basis functions and clustered tensor approximation. ACM Trans. Graph. 25, 3 (Jul. 2006), 967–976. https://doi.org/10.1145/1141911.1141981
49. Greg Turk and James F. O'Brien. 2005. Shape transformation using variational implicit functions. In ACM SIGGRAPH 2005 Courses (SIGGRAPH'05). ACM, New York, NY, USA, Article 13. https://doi.org/10.1145/1198555.1198639
50. David Vanderhaeghe, Romain Vergne, Pascal Barla, and William Baxter. 2011. Dynamic stylized shading primitives. In Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering (NPAR'11). 99–104. https://doi.org/10.1145/2024676.2024693
51. Lvdi Wang, Kun Zhou, Yizhou Yu, and Baining Guo. 2010. Vector solid textures. ACM Trans. Graph. 29, 4 (2010), 86.
52. Daniel N. Wood, Adam Finkelstein, John F. Hughes, Craig E. Thayer, and David H. Salesin. 1997. Multiperspective panoramas for cel animation. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley, 243–250.
53. Pete Wright. 2015. Cinematic Portraits: How to Create Classic Hollywood Photography. Amherst Media.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

    • Rendering

      • Non-photorealistic rendering