Physically guided liquid surface modeling from videos

Abstract

We present an image-based reconstruction framework to model real water scenes captured by stereoscopic video. In contrast to many image-based modeling techniques that rely on user interaction to obtain high-quality 3D models, we instead apply automatically calculated physically-based constraints to refine the initial model. The combination of image-based reconstruction with physically-based simulation allows us to model complex and dynamic objects such as fluid. Using a depth map sequence as initial conditions, we use a physically based approach that automatically fills in missing regions, removes outliers, and refines the geometric shape so that the final 3D model is consistent to both the input video data and the laws of physics. Physically-guided modeling also makes interpolation or extrapolation in the space-time domain possible, and even allows the fusion of depth maps that were taken at different times or viewpoints. We demonstrated the effectiveness of our framework with a number of real scenes, all captured using only a single pair of cameras.

References

1. Atcheson, B., Ihrke, I., Heidrich, W., Tevs, A., Bradley, D., Magnor, M., and Seidel, H.-P. 2008. Time-resolved 3d capture of non-stationary gas flows. In Proc. of ACM SIGGRAPH Asia 2008, vol. 27.
2. Bhat, K. S., Seitz, S. M., Hodgins, J. K., and Khosla, P. K. 2004. Flow-based video synthesis and editing. In Proc. of ACM SIGGRAPH 2004, 360--363.
3. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. In Proc. of ACM SIGGRAPH 2008, vol. 27.
4. Carlson, M., Mucha, P. J., Van Horn III, R. B., and Turk, G. 2002. Melting and flowing. In Proc. of SCA '02, 167--174.
5. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. In Proc. of ACM SIGGRAPH '08, 1--10.
6. Egnal, G., and Wildes, R. P. 2002. Detecting binocular half-occlusions: Empirical comparisons of five approaches. IEEE Trans. Pattern Anal. Mach. Intell. 24, 8, 1127--1133.
7. Enright, D., Marschner, S., and Fedkiw, R. 2002. Animation and rendering of complex water surfaces. In Proc. of ACM SIGGRAPH '02, 736--744.
8. Fattal, R., and Lischinski, D. 2004. Target-driven smoke animation. In Proc. of ACM SIGGRAPH 2004.
9. Foster, N., and Fedkiw, R. 2001. Practical animation of liquids. In Proc. of SIGGRAPH '01, 23--30.
10. Foster, N., and Metaxas, D. 1996. Realistic animation of liquids. Graph. Models Image Process. 58, 5.
11. Grant, I. 1997. Particle image velocimetry: a review. In Proc. of the Institution of Mechanical Engineers, vol. 211, 55C76.
12. Hawkins, T., Einarsson, P., and Debevec, P. 2005. Acquisition of time-varying participating media. In Proc. of ACM SIGGRAPH 2005.
13. Hullin, M. B., Fuchs, M., Ihrke, I., Seidel, H.-P., and Lensch, H. P. A. 2008. Fluorescent immersion range scanning. In Proc. of ACM SIGGRAPH 2008.
14. Ihrke, I., Goldluecke, B., and Magnor, M. 2005. Reconstructing the geometry of flowing water. In ICCV '05, IEEE Computer Society, Washington, DC, USA, 1055--1060.
15. Kanade, T., Rander, P., Vedula, S., and Saito, H. 1999. Virtualized reality: Digitizing a 3d time-varying event as is and in real time. In Mixed Reality, Merging Real and Virtual Worlds. 41--57.
16. McNamara, A., Treuille, A., Popovic, Z., and Stam, J. 2004. Fluid control using the adjoint method. In Proc. of ACM SIGGRAPH 2004.
17. Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Eurographics Symposium on Geometry Processing.
18. Morris, N. J. W., and Kutulakos, K. N. 2005. Dynamic refraction stereo. In Proc. of International Conference on Computer Vision.
19. Morris, N. J. W., and Kutulakos, K. N. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. In Proc. of 11th Int. Conf. Computer Vision.
20. Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., and Kang, S. B. 2006. Image-based plant modeling. In Proc. of ACM SIGGRAPH 2006.
21. Schneider, R., and Kobbelt, L. 2001. Geometric fairing of irregular meshes for freeform surface design. Computer aided geometric design 18, 359--379.
22. Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., and Sheffer, A. 2008. Space-time surface reconstruction using incompressible flow. In Proc. of ACM SIGGRAPH Asia 2008, vol. 27, 1--10.
23. Shi, J., and Tomasi, C. 1994. Good features to track. In Proc. of CVPR 1994, 593--600.
24. Simon, S. V., Baker, S., Seitz, S., and Kanade, T. 2000. Shape and motion carving in 6d. In Computer Vision and Pattern Recognition.
25. Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3d architectural modeling from unordered photo collections. Proc. of SIGGRAPH Asia 2008 27.
26. Stam, J. 1999. Stable fluids. In Proc. of ACM SIGGRAPH '99, 121--128.
27. Staniforth, A., and Côté, J. 1991. Semi-lagrangian integration schemes for atmospheric models. Monthly Weather Review 119, 9, 2206.
28. Sun, J., Zheng, N.-N., and Shum, H.-Y. 2003. Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25, 7, 787.
29. Tan, P., Zeng, G., Wang, J., Kang, S. B., and Quan, L. 2007. Image-based tree modeling. In Proc. of ACM SIGGRAPH 2007.
30. Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Eurographics Symposium on Geometry Processing.
31. Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. In Proc. of ACM SIGGRAPH 2005.
32. White, R., Crane, K., and Forsyth, D. 2007. Capturing and animating occluded cloth. In Proc. of ACM SIGGRAPH 2007.
33. Xiao, J., Fang, T., Tan, P., Zhao, P., and Quan, L. 2008. Image-based facade modeling. Proc. of SIGGRAPH Asia 2008 27.
34. Yang, Q., Yang, R., Davis, J., and Nister, D. 2007. Spatial-depth super resolution for range images. In Proc. of CVPR 2007, vol. 0, 1--8.
35. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. In Proc. of ACM SIGGRAPH '04, 600--608.
36. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics, 23, 3, 600--608.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation

      • Physical simulation

    • Shape modeling

• Mathematics of computing

  • Continuous mathematics

    • Topology

      • Geometric topology