Abstract
We present a novel up-resing technique for generating high-resolution liquids based on scene flow estimation using deep neural networks. Our approach infers and synthesizes small- and large-scale details solely from a low-resolution particle-based liquid simulation. The proposed network leverages neighborhood contributions to encode inherent liquid properties throughout convolutions. We also propose a particle-based approach to interpolate between liquids generated from varying simulation discretizations using a state-of-the-art bidirectional optical flow solver method for fluids in addition with a novel key-event topological alignment constraint. In conjunction with the neighborhood contributions, our loss formulation allows the inference model throughout epochs to reward important differences in regard to significant gaps in simulation discretizations. Even when applied in an untested simulation setup, our approach is able to generate plausible high-resolution details. Using this interpolation approach and the predicted displacements, our approach combines the input liquid properties with the predicted motion to infer semi-Lagrangian advection. We furthermore showcase how the proposed interpolation approach can facilitate generating large simulation datasets with a subset of initial condition parameters.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Neural UpFlow: A Scene Flow Learning Approach to Increase the Apparent Resolution of Particle-Based Liquids
- Mridul Aanjaneya, Ming Gao, Haixiang Liu, Christopher Batty, and Eftychios Sifakis. 2017. Power diagrams and sparse paged grids for high resolution adaptive liquids. ACM Trans. on Graphics (TOG) 36, 4, Article 140 (2017), 12 pages.Google Scholar
Digital Library
- Kfir Aberman, Peizh Uo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, and Baoquan Chen. 2020. Skeleton-aware networks for deep motion retargeting. ACM Trans. on Graphics (TOG) 39, 4, Article 62 (2020), 14 pages.Google Scholar
Digital Library
- Bart Adams, Mark Pauly, Richard Keiser, and Leonidas J Guibas. 2007. Adaptively sampled particle fluids. ACM Trans. on Graphics (TOG) 26, 3, Article 48 (2007), 8 pages.Google Scholar
Digital Library
- Ryoichi Ando, Nils Thurey, and Reiji Tsuruno. 2012. Preserving fluid sheets with adaptively sampled anisotropic particles. IEEE Trans. on Visualization and Computer Graphics 18, 8 (2012), 1202--1214.Google Scholar
Digital Library
- Ryoichi Ando, Nils Thürey, and Chris Wojtan. 2013. Highly adaptive liquid simulations on tetrahedral meshes. ACM Trans. on Graphics (TOG) 32, 4, Article 103 (2013), 10 pages.Google Scholar
Digital Library
- Markus Becker and Matthias Teschner. 2007. Weakly compressible SPH for free surface flows. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA). 209--217.Google Scholar
- Chakravarty R Alla Chaitanya, Anton S Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Trans. on Graphics (TOG) 36, 4, Article 98 (2017), 12 pages.Google Scholar
- Nuttapong Chentanez, Matthias Müller, and Tae-Yong Kim. 2015. Coupling 3D Eulerian, heightfield and particle methods for interactive simulation of large scale liquid phenomena. IEEE Trans. on Visualization and Computer Graphics 21, 10 (2015), 1116--1128.Google Scholar
Digital Library
- Mengyu Chu and Nils Thuerey. 2017. Data-driven synthesis of smoke flows with CNN-based feature descriptors. ACM Trans. on Graphics (TOG) 36, 4, Article 69 (2017), 14 pages.Google Scholar
Digital Library
- Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A Efros, and Adrien Bousseau. 2018. 3D sketching using multi-view deep volumetric prediction. Proc. ACM on Computer Graphics and Interactive Techniques 1, 1, Article 21 (2018), 22 pages.Google Scholar
Digital Library
- Florian Ferstl, Ryoichi Ando, Chris Wojtan, Rüdiger Westermann, and Nils Thuerey. 2016. Narrow band FLIP for liquid simulations. Computer Graphics Forum 35, 2 (2016), 225--232.Google Scholar
Cross Ref
- Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2015. Texture synthesis using convolutional neural networks. In Proc. International Conference on Neural Information Processing Systems. 262--270.Google Scholar
- Byungsoo Kim, Vinicius C Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. 2019. Deep fluids: A generative network for parameterized fluid simulations. Computer Graphics Forum 38, 2 (2019), 59--70.Google Scholar
Cross Ref
- Theodore Kim, Jerry Tessendorf, and Nils Thuerey. 2013. Closest point turbulence for liquid surfaces. ACM Trans. on Graphics (TOG) 32, 2, Article 15 (2013), 13 pages.Google Scholar
Digital Library
- Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations.Google Scholar
- L'ubor Ladický, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven fluid simulations using regression forests. ACM Trans. on Graphics (TOG) 34, 6, Article 199 (2015), 9 pages.Google Scholar
Digital Library
- Xingyu Liu, Charles R Qi, and Leonidas J Guibas. 2019. Flownet3d: Learning scene flow in 3D point clouds. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 529--537.Google Scholar
Cross Ref
- Olivier Mercier, Cynthia Beauchemin, Nils Thuerey, Theodore Kim, and Derek Nowrouzezahrai. 2015. Surface turbulence for particle-based liquid simulations. ACM Trans. on Graphics (TOG) 34, 6, Article 202 (2015), 10 pages.Google Scholar
Digital Library
- Lukas Prantl, Boris Bonev, and Nils Thuerey. 2018. Generating liquid simulations with deformation-aware neural networks. In International Conference on Learning Representations.Google Scholar
- Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition. 77--85.Google Scholar
- Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017b. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Proc. International Conference on Neural Information Processing Systems. 5105--5114.Google Scholar
- Karthik Raveendran, Chris Wojtan, and Greg Turk. 2011. Hybrid smoothed particle hydrodynamics. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA). 33--42.Google Scholar
Digital Library
- Bruno Roy, Eric Paquette, and Pierre Poulin. 2020. Particle upsampling as a flexible post-processing approach to increase details in animations of splashing liquids. Computers & Graphics 88 (2020), 57--69.Google Scholar
Cross Ref
- Bruno Roy and Pierre Poulin. 2018. A hybrid Eulerian-DFSPH scheme for efficient surface band liquid simulation. Computers & Graphics 77 (2018), 194--204.Google Scholar
Cross Ref
- Takahiro Sato, Christopher Wojtan, Nils Thuerey, Takeo Igarashi, and Ryoichi Ando. 2018. Extended narrow band FLIP for liquid simulations. Computer Graphics Forum 37, 2 (2018), 169--177.Google Scholar
Cross Ref
- Barbara Solenthaler and Markus Gross. 2011. Two-scale particle simulation. ACM Trans. on Graphics (TOG) 30, 4, Article 81 (2011), 8 pages.Google Scholar
Digital Library
- Nils Thuerey. 2017. Interpolations of smoke and liquid simulations. ACM Trans. on Graphics (TOG) 36, 1, Article 3 (2017), 16 pages.Google Scholar
Digital Library
- Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2017. Accelerating Eulerian fluid simulation with convolutional networks. In International Conference on Machine Learning. PMLR, 3424--3433.Google Scholar
- Hsiao-Yu Fish Tung, Hsiao-Wei Tung, Ersin Yumer, and Katerina Fragkiadaki. 2017. Self-supervised learning of motion capture. In Proc. International Conference on Neural Information Processing Systems. 5242--5252.Google Scholar
- Kiwon Um, Xiangyu Hu, and Nils Thuerey. 2018. Liquid splash modeling with neural networks. Computer Graphics Forum 37, 8 (2018), 171--182.Google Scholar
Cross Ref
- Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. 2019. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations.Google Scholar
- Gokul Varadhan, Shankar Krishnan, TVN Sriram, and Dinesh Manocha. 2004. Topology preserving surface extraction using adaptive subdivision. In Proc. Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP). 235--244.Google Scholar
Digital Library
- Tuanfeng Y Wang, Hao Su, Qixing Huang, Jingwei Huang, Leonidas J Guibas, and Niloy J Mitra. 2016. Unsupervised texture transfer from images to model collections. ACM Trans. on Graphics (TOG) 35, 6, Article 177 (2016), 13 pages.Google Scholar
Digital Library
- Zirui Wang, Shuda Li, Henry Howard-Jenkins, Victor Prisacariu, and Min Chen. 2020. FlowNet3D++: Geometric losses for deep scene flow estimation. In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision. 91--98.Google Scholar
Cross Ref
- Maximilian Werhahn, You Xie, Mengyu Chu, and Nils Thuerey. 2019. A multi-pass GAN for fluid flow super-resolution. Proc. ACM on Computer Graphics and Interactive Techniques 2, 2, Article 10 (2019), 21 pages.Google Scholar
Digital Library
- Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2019. Latent space physics: Towards learning the temporal evolution of fluid flow. Computer Graphics Forum 38, 2 (2019), 71--82.Google Scholar
Cross Ref
- Rene Winchenbach, Hendrik Hochstetter, and Andreas Kolb. 2017. Infinite continuous adaptivity for incompressible SPH. ACM Trans. on Graphics (TOG) 36, 4, Article 102 (2017), 10 pages.Google Scholar
Digital Library
- Rene Winchenbach and Andreas Kolb. 2021. Optimized refinement for spatially adaptive SPH. ACM Trans. on Graphics (TOG) 40, 1, Article 8 (2021), 15 pages.Google Scholar
Digital Library
- Chris Wojtan, Nils Thürey, Markus Gross, and Greg Turk. 2009. Deforming meshes that split and merge. ACM Trans. on Graphics (TOG) 28, 3, Article 76 (2009), 10 pages.Google Scholar
Digital Library
- You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. 2018. tempoGAN: A temporally coherent, volumetric GAN for super-resolution fluid flow. ACM Trans. on Graphics (TOG) 37, 4, Article 95 (2018), 15 pages.Google Scholar
Digital Library
- Cheng Yang, Xubo Yang, and Xiangyun Xiao. 2016. Data-driven projection method in fluid simulation. Computer Animation and Virtual Worlds 27, 3-4 (2016), 415--424.Google Scholar
Digital Library
- Yongning Zhu and Robert Bridson. 2005. Animating sand as a fluid. ACM Trans. on Graphics (TOG) 24, 3 (2005), 965--972.Google Scholar
Digital Library
Index Terms
Neural UpFlow: A Scene Flow Learning Approach to Increase the Apparent Resolution of Particle-Based Liquids
Recommendations
Lagrangian vortex sheets for animating fluids
Buoyant turbulent smoke plumes with a sharp smoke-air interface, such as volcanic plumes, are notoriously hard to simulate. The surface clearly shows small-scale turbulent structures which are costly to resolve. In addition, the turbulence onset is ...
Synthetic turbulence using artificial boundary layers
Turbulent vortices in fluid flows are crucial for a visually interesting appearance. Although there has been a significant amount of work on turbulence in graphics recently, these algorithms rely on the underlying simulation to resolve the flow around ...
Synthesizing Solid-Induced Turbulence for Particle-Based Fluids
CADGRAPHICS '13: Proceedings of the 2013 International Conference on Computer-Aided Design and Computer GraphicsSimulating the accompanying turbulent details of fluid-solid coupling is still challenging, as numerical dissipation always plagues current fluid solvers. In this paper, we propose a novel particle-based method to simulate turbulent details generated ...






Comments