skip to main content
research-article

Neural UpFlow: A Scene Flow Learning Approach to Increase the Apparent Resolution of Particle-Based Liquids

Published:27 September 2021Publication History
Skip Abstract Section

Abstract

We present a novel up-resing technique for generating high-resolution liquids based on scene flow estimation using deep neural networks. Our approach infers and synthesizes small- and large-scale details solely from a low-resolution particle-based liquid simulation. The proposed network leverages neighborhood contributions to encode inherent liquid properties throughout convolutions. We also propose a particle-based approach to interpolate between liquids generated from varying simulation discretizations using a state-of-the-art bidirectional optical flow solver method for fluids in addition with a novel key-event topological alignment constraint. In conjunction with the neighborhood contributions, our loss formulation allows the inference model throughout epochs to reward important differences in regard to significant gaps in simulation discretizations. Even when applied in an untested simulation setup, our approach is able to generate plausible high-resolution details. Using this interpolation approach and the predicted displacements, our approach combines the input liquid properties with the predicted motion to infer semi-Lagrangian advection. We furthermore showcase how the proposed interpolation approach can facilitate generating large simulation datasets with a subset of initial condition parameters.

Skip Supplemental Material Section

Supplemental Material

References

  1. Mridul Aanjaneya, Ming Gao, Haixiang Liu, Christopher Batty, and Eftychios Sifakis. 2017. Power diagrams and sparse paged grids for high resolution adaptive liquids. ACM Trans. on Graphics (TOG) 36, 4, Article 140 (2017), 12 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Kfir Aberman, Peizh Uo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, and Baoquan Chen. 2020. Skeleton-aware networks for deep motion retargeting. ACM Trans. on Graphics (TOG) 39, 4, Article 62 (2020), 14 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Bart Adams, Mark Pauly, Richard Keiser, and Leonidas J Guibas. 2007. Adaptively sampled particle fluids. ACM Trans. on Graphics (TOG) 26, 3, Article 48 (2007), 8 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ryoichi Ando, Nils Thurey, and Reiji Tsuruno. 2012. Preserving fluid sheets with adaptively sampled anisotropic particles. IEEE Trans. on Visualization and Computer Graphics 18, 8 (2012), 1202--1214.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ryoichi Ando, Nils Thürey, and Chris Wojtan. 2013. Highly adaptive liquid simulations on tetrahedral meshes. ACM Trans. on Graphics (TOG) 32, 4, Article 103 (2013), 10 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Markus Becker and Matthias Teschner. 2007. Weakly compressible SPH for free surface flows. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA). 209--217.Google ScholarGoogle Scholar
  7. Chakravarty R Alla Chaitanya, Anton S Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Trans. on Graphics (TOG) 36, 4, Article 98 (2017), 12 pages.Google ScholarGoogle Scholar
  8. Nuttapong Chentanez, Matthias Müller, and Tae-Yong Kim. 2015. Coupling 3D Eulerian, heightfield and particle methods for interactive simulation of large scale liquid phenomena. IEEE Trans. on Visualization and Computer Graphics 21, 10 (2015), 1116--1128.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Mengyu Chu and Nils Thuerey. 2017. Data-driven synthesis of smoke flows with CNN-based feature descriptors. ACM Trans. on Graphics (TOG) 36, 4, Article 69 (2017), 14 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A Efros, and Adrien Bousseau. 2018. 3D sketching using multi-view deep volumetric prediction. Proc. ACM on Computer Graphics and Interactive Techniques 1, 1, Article 21 (2018), 22 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Florian Ferstl, Ryoichi Ando, Chris Wojtan, Rüdiger Westermann, and Nils Thuerey. 2016. Narrow band FLIP for liquid simulations. Computer Graphics Forum 35, 2 (2016), 225--232.Google ScholarGoogle ScholarCross RefCross Ref
  12. Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2015. Texture synthesis using convolutional neural networks. In Proc. International Conference on Neural Information Processing Systems. 262--270.Google ScholarGoogle Scholar
  13. Byungsoo Kim, Vinicius C Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. 2019. Deep fluids: A generative network for parameterized fluid simulations. Computer Graphics Forum 38, 2 (2019), 59--70.Google ScholarGoogle ScholarCross RefCross Ref
  14. Theodore Kim, Jerry Tessendorf, and Nils Thuerey. 2013. Closest point turbulence for liquid surfaces. ACM Trans. on Graphics (TOG) 32, 2, Article 15 (2013), 13 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations.Google ScholarGoogle Scholar
  16. L'ubor Ladický, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven fluid simulations using regression forests. ACM Trans. on Graphics (TOG) 34, 6, Article 199 (2015), 9 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Xingyu Liu, Charles R Qi, and Leonidas J Guibas. 2019. Flownet3d: Learning scene flow in 3D point clouds. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 529--537.Google ScholarGoogle ScholarCross RefCross Ref
  18. Olivier Mercier, Cynthia Beauchemin, Nils Thuerey, Theodore Kim, and Derek Nowrouzezahrai. 2015. Surface turbulence for particle-based liquid simulations. ACM Trans. on Graphics (TOG) 34, 6, Article 202 (2015), 10 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lukas Prantl, Boris Bonev, and Nils Thuerey. 2018. Generating liquid simulations with deformation-aware neural networks. In International Conference on Learning Representations.Google ScholarGoogle Scholar
  20. Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition. 77--85.Google ScholarGoogle Scholar
  21. Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017b. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Proc. International Conference on Neural Information Processing Systems. 5105--5114.Google ScholarGoogle Scholar
  22. Karthik Raveendran, Chris Wojtan, and Greg Turk. 2011. Hybrid smoothed particle hydrodynamics. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA). 33--42.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bruno Roy, Eric Paquette, and Pierre Poulin. 2020. Particle upsampling as a flexible post-processing approach to increase details in animations of splashing liquids. Computers & Graphics 88 (2020), 57--69.Google ScholarGoogle ScholarCross RefCross Ref
  24. Bruno Roy and Pierre Poulin. 2018. A hybrid Eulerian-DFSPH scheme for efficient surface band liquid simulation. Computers & Graphics 77 (2018), 194--204.Google ScholarGoogle ScholarCross RefCross Ref
  25. Takahiro Sato, Christopher Wojtan, Nils Thuerey, Takeo Igarashi, and Ryoichi Ando. 2018. Extended narrow band FLIP for liquid simulations. Computer Graphics Forum 37, 2 (2018), 169--177.Google ScholarGoogle ScholarCross RefCross Ref
  26. Barbara Solenthaler and Markus Gross. 2011. Two-scale particle simulation. ACM Trans. on Graphics (TOG) 30, 4, Article 81 (2011), 8 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Nils Thuerey. 2017. Interpolations of smoke and liquid simulations. ACM Trans. on Graphics (TOG) 36, 1, Article 3 (2017), 16 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2017. Accelerating Eulerian fluid simulation with convolutional networks. In International Conference on Machine Learning. PMLR, 3424--3433.Google ScholarGoogle Scholar
  29. Hsiao-Yu Fish Tung, Hsiao-Wei Tung, Ersin Yumer, and Katerina Fragkiadaki. 2017. Self-supervised learning of motion capture. In Proc. International Conference on Neural Information Processing Systems. 5242--5252.Google ScholarGoogle Scholar
  30. Kiwon Um, Xiangyu Hu, and Nils Thuerey. 2018. Liquid splash modeling with neural networks. Computer Graphics Forum 37, 8 (2018), 171--182.Google ScholarGoogle ScholarCross RefCross Ref
  31. Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. 2019. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations.Google ScholarGoogle Scholar
  32. Gokul Varadhan, Shankar Krishnan, TVN Sriram, and Dinesh Manocha. 2004. Topology preserving surface extraction using adaptive subdivision. In Proc. Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP). 235--244.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Tuanfeng Y Wang, Hao Su, Qixing Huang, Jingwei Huang, Leonidas J Guibas, and Niloy J Mitra. 2016. Unsupervised texture transfer from images to model collections. ACM Trans. on Graphics (TOG) 35, 6, Article 177 (2016), 13 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Zirui Wang, Shuda Li, Henry Howard-Jenkins, Victor Prisacariu, and Min Chen. 2020. FlowNet3D++: Geometric losses for deep scene flow estimation. In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision. 91--98.Google ScholarGoogle ScholarCross RefCross Ref
  35. Maximilian Werhahn, You Xie, Mengyu Chu, and Nils Thuerey. 2019. A multi-pass GAN for fluid flow super-resolution. Proc. ACM on Computer Graphics and Interactive Techniques 2, 2, Article 10 (2019), 21 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2019. Latent space physics: Towards learning the temporal evolution of fluid flow. Computer Graphics Forum 38, 2 (2019), 71--82.Google ScholarGoogle ScholarCross RefCross Ref
  37. Rene Winchenbach, Hendrik Hochstetter, and Andreas Kolb. 2017. Infinite continuous adaptivity for incompressible SPH. ACM Trans. on Graphics (TOG) 36, 4, Article 102 (2017), 10 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Rene Winchenbach and Andreas Kolb. 2021. Optimized refinement for spatially adaptive SPH. ACM Trans. on Graphics (TOG) 40, 1, Article 8 (2021), 15 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Chris Wojtan, Nils Thürey, Markus Gross, and Greg Turk. 2009. Deforming meshes that split and merge. ACM Trans. on Graphics (TOG) 28, 3, Article 76 (2009), 10 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. 2018. tempoGAN: A temporally coherent, volumetric GAN for super-resolution fluid flow. ACM Trans. on Graphics (TOG) 37, 4, Article 95 (2018), 15 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Cheng Yang, Xubo Yang, and Xiangyun Xiao. 2016. Data-driven projection method in fluid simulation. Computer Animation and Virtual Worlds 27, 3-4 (2016), 415--424.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Yongning Zhu and Robert Bridson. 2005. Animating sand as a fluid. ACM Trans. on Graphics (TOG) 24, 3 (2005), 965--972.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Neural UpFlow: A Scene Flow Learning Approach to Increase the Apparent Resolution of Particle-Based Liquids

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!