Dan Ring
ABSTRACT
Recent years has seen an explosion in Machine Learning (ML) research. The challenge is now to transfer these new algorithms into the hands of artists and TD's in visual effects and animation studios, so that they can start experimenting with ML within their existing pipelines. This paper presents some of the current challenges to experimentation and deployment of ML frameworks in the post-production industry. It introduces our open-source "ML-Server" client / server system as an answer to enabling rapid prototyping, experimentation and development of ML models in post-production software. Data, code and examples for the system can be found on the GitHub repository page:
https://github.com/TheFoundryVisionmongers/nuke-ML-server
Get full access to this Publication
Purchase, subscribe or recommend this publication to your librarian.
Already a Subscriber?Sign In
References
- Waleed Abdulla. 2017. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. https://github.com/matterport/Mask_RCNN. (2017).Google Scholar
- Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. 2012. Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives. CoRR abs/1206.5538 (2012).Google Scholar
- Blue Fairy. 2019. Blue Fairy Inc: Developing A.I. tools for the VFX industry. https://prisma-ai.com/. (2019). {Online; accessed 03-May-2019}.Google Scholar
- G. Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal of Software Tools (2000).Google Scholar
- Cronobo. 2019. Nexture Online: Coherent, photo-accurate textures for anyone. http://cronobo.com/. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Docker. 2019. Enterprise Container Platform for High-Velocity Innovation. http://www.docker.com/. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2016. Image Style Transfer Using Convolutional Neural Networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27--30, 2016. 2414--2423.Google Scholar
- Google. 2019. TensorBoard, TensorFlow's visualization toolkit. https://www.tensorflow.org/tensorboard. (2019). {Online; accessed 10-June-2019}.Google Scholar
- Byungsoo Kim, Vinicius C. Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. 2019. Deep Fluids: A Generative Network for Parameterized Fluid Simulations. Computer Graphics Forum (Proc. Eurographics) 38, 2 (2019).Google Scholar
- Kognat. 2019. RotoBot: Semantic Instance Segmentation Tool for VFX. https://kognat.com/. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Hugo Larochelle. 2019. Tweet:. https://twitter.com/hugo_larochelle/status/997620696967733249. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Wenbin Li, Fabio Viola, Jonathan Starck, Gabriel J. Brostow, and Neill D.F. Campbell. 2016. Roto++: Accelerating Professional Rotoscoping using Shape Manifolds. ACM Transactions on Graphics (In proceeding of ACM SIGGRAPH' 16) 35, 4 (2016). Google Scholar
- K.-K. Maninis, S. Caelles, J. Pont-Tuset, and L. Van Gool. 2018. Deep Extreme Cut: From Extreme Points to Object Segmentation. In Computer Vision and Pattern Recognition (CVPR).Google Scholar
- NVidia. 2019. NVidia Docker: Build and run Docker containers leveraging NVIDIA GPUs. https://github.com/NVIDIA/nvidia-docker. (2019). {Online; accessed 03-May-2019}.Google Scholar
- NVIDIA. 2019. TensorRT, Programmable Inference Accelerator. https://developer.nvidia.com/tensorrt. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Onnx. 2019. Open Neural Network Exchange Format. https://onnx.ai/. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. 2019. Semantic Image Synthesis with Spatially-Adaptive Normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- PlaidML. 2019. PlaidML: A platform for making deep learning work everywhere. https://github.com/plaidml/plaidml. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Micah J. Sheller, G. Anthony Reina, Brandon P. M. Edwards, Jason Martin, and Spyridon Bakas. 2018. Multi-institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation. In [email protected]Google Scholar
- AI Technology & Industry Review Synced. 2019. CVPR 2019 Accepts Record 1300 Papers. https://medium.com/syncedreview/cvpr-2019-accepts-record-1300-papers-91b9e3b315f5. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia. 2018. Scale-recurrent Network for Deep Image Deblurring. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, and Matthias Niessner. 2018. Face2Face: Real-time Face Capture and Reenactment of RGB Videos. Commun. ACM 62, 1 (Dec. 2018), 96--104. Google Scholar
- Wikipedia. 2019. Comparison of deep-learning software - Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Comparison%20of%20deep-learning%20software&oldid=893977550. (2019). {Online; accessed 03-May-2019}.Google Scholar
- Ning Xu, Brian Price, Scott Cohen, Jimei Yang, and Thomas S Huang. 2016. Deep interactive object selection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 373--381.Google Scholar
Index Terms
Jumping in at the deep end: how to experiment with machine learning in post-production software
Comments