Amir Semmo
Max Reimann
ABSTRACT
We present ViVid, a mobile app for iOS that empowers users to express dynamics in stylized Live Photos. This app uses state-of-the-art computer-vision techniques based on convolutional neural networks to estimate motion in the video footage that is captured together with a photo. Based on this analysis and best practices of contemporary art, photos can be stylized as a pencil drawing or cartoon look that includes design elements to visually suggest motion, such as ghosts, motion lines and halos. Its interactive parameterizations enable users to filter and art-direct composition variables, such as color, size and opacity. ViVid is based on Apple's CoreML, Metal and PhotoKit APIs for optimized on-device processing. Thus, the motion estimation is scheduled to utilize the dedicated neural engine, while shading-based image stylization is able to process the video footage in real-time on the GPU. This way, the app provides a unique tool for creating lively photo stylizations with ease.
- Saeideh Bakhshi, David A Shamma, Lyndon Kennedy, and Eric Gilbert. 2015. Why We Filter Our Photos and How It Impacts Engagement. In Proc. ICWSM. AAAI Press, 12--21.Google Scholar
- John P. Collomosse, David Rowntree, and Peter M. Hall. 2003. Cartoon-Style Rendering of Motion from Video. In Proc. Vision, Video and Graphics (VVG). The Eurographics Association, 117--124.Google Scholar
- James E Cutting. 2002. Representing Motion in a Static Image: Constraints and Parallels in Art, Science, and Popular Culture. Perception 31, 10 (2002), 1165--1193.Google ScholarCross Ref
- Kapil Dev. 2013. Mobile Expressive Renderings: The State of the Art. IEEE Computer Graphics and Applications 33, 3 (May/June 2013), 22--31. Google ScholarDigital Library
- Cewu Lu, Li Xu, and Jiaya Jia. 2012. Combining Sketch and Tone for Pencil Drawing Production. In Proc. NPAR. The Eurographics Association, 65--73. Google ScholarDigital Library
- Scott McCloud. 2006. Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels. Harper New York.Google Scholar
- Scott McCloud. 2008. Understanding Comics: The Invisible Art. Paw Prints.Google Scholar
- Marc Nienhaus and Jürgen Döllner. 2005. Depicting Dynamics Using Principles of Visual Art and Narrations. IEEE Computer Graphics and Applications 25, 3 (2005), 40--51. Google ScholarDigital Library
- Marc Nienhaus, Holger Winnemöller, Jürgen Döllner, and Bruce Gooch. 2008. Forward Lean - Deriving Motion Illustrations From Video. In Proc. SIGGRAPH Asia Sketches.Google Scholar
- Richard Rudner. 1951. On Semiotic Aesthetics. The Journal of Aesthetics and Art Criticism 10, 1 (Sept. 1951), 67--77.Google ScholarCross Ref
- D. Sun, X. Yang, M. Liu, and J Kautz. 2018. PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume. In Proc. IEEE CVPR. IEEE, 8934--8943.Google ScholarCross Ref
- Peter Walker. 2015. Depicting Visual Motion in Still Images: Forward Leaning and a Left to Right Bias for Lateral Movement. Perception 44, 2 (2015), 111--128.Google ScholarCross Ref
- Holger Winnemöller, Jan Eric Kyprianidis, and Sven Olsen. 2012. XDoG: An eXtended Difference-of-Gaussians Compendium including Advanced Image Stylization. Computers & Graphics 36, 6 (Oct. 2012), 740--753. Google ScholarDigital Library
- Holger Winnemöller, Sven C. Olsen, and Bruce Gooch. 2006. Real-Time Video Abstraction. ACM Transactions on Graphics 25, 3 (July 2006), 1221--1226. Google ScholarDigital Library
Index Terms
ViVid: depicting dynamics in stylized live photos
Recommendations
A Layered Authoring Tool for Stylized 3D animations
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsGuided by the 12 principles of animation, stylization is a core 2D animation feature but has been utilized mainly by experienced animators. Although there are tools for stylizing 2D animations, creating stylized 3D animations remains a challenging ...
Enhancement of stage performances with projected CG and Noh motions
ACE '10: Proceedings of the 7th International Conference on Advances in Computer Entertainment TechnologyOur goal is to provide a support system for creating rich and flexible presentations in live performances using motion data. We created a live stage performance and a projection system which is for enhancing the performance with videos and CG ...
Skunk: DreamWorks fur motion system
DigiPro '19: Proceedings of the 2019 Digital Production SymposiumThis talk presents DreamWorks' fur motion system Skunk which is used to produce motion for fur on characters, garments, and props. Skunk's ease of use, speed, stability, interactive nature, flexible framework, layered simulation approach, on the fly fur ...
Comments