Dynamic hair manipulation in images and videos

Abstract

This paper presents a single-view hair modeling technique for generating visually and physically plausible 3D hair models with modest user interaction. By solving an unambiguous 3D vector field explicitly from the image and adopting an iterative hair generation algorithm, we can create hair models that not only visually match the original input very well but also possess physical plausibility (e.g., having strand roots fixed on the scalp and preserving the length and continuity of real strands in the image as much as possible). The latter property enables us to manipulate hair in many new ways that were previously very difficult with a single image, such as dynamic simulation or interactive hair shape editing. We further extend the modeling approach to handle simple video input, and generate dynamic 3D hair models. This allows users to manipulate hair in a video or transfer styles from images to videos.

References

1. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3, 24:1--11.
2. Beeler, T., Bickel, B., Noris, G., Beardsley, P., Marschner, S., Sumner, R. W., and Gross, M. 2012. Coupled 3D reconstruction of sparse facial hair and skin. ACM Trans. Graph. 31, 4, 117:1--10.
3. Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P. N., and Nayar, S. K. 2008. Face Swapping: Automatically replacing faces in photographs. ACM Trans. Graph. 27, 39:1--8.
4. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH '99, 187--194.
5. Bonneel, N., Paris, S., Panne, M. V. D., Durand, F., and Drettakis, G. 2009. Single photo estimation of hair appearance. Computer Graphics Forum 28, 1171--1180.
6. Chai, M., Wang, L., Yu, Y., Weng, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1--8.
7. Dale, K., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., and Pfister, H. 2011. Video face replacement. ACM Trans. Graph. 30, 6, 130:1--10.
8. Daviet, G., Bertails-Descoubes, F., and Boissieux, L. 2011. A hybrid iterative solver for robustly capturing Coulomb friction in hair dynamics. ACM Trans. Graph. 30, 6, 139:1--12.
9. Fu, H., Wei, Y., Tai, C.-L., and Quan, L. 2007. Sketching hairstyles. In Proc. EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling, 31--36.
10. Herrera, T. L., Zinke, A., and Weber, A. 2012. Lighting hair from the inside: A thermal approach to hair reconstruction. ACM Trans. Graph. 31, 6, 146:1--9.
11. Hoiem, D., Efros, A. A., and Hebert, M. 2005. Automatic photo pop-up. ACM Trans. Graph. 24, 3, 577--584.
12. Jain, A., Thormählen, T., Seidel, H.-P., and Theobalt, C. 2010. MovieReshape: Tracking and reshaping of humans in videos. ACM Trans. Graph. 29, 6, 148:1--10.
13. Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. 28, 5, 164:1--9.
14. Karp, R. M. 1972. Reducibility among combinatorial problems. Complexity of Computer Computations, 85--103.
15. Karsch, K., Hedau, V., Forsyth, D., and Hoiem, D. 2011. Rendering synthetic oobject into legacy photographs. ACM Trans. Graph. 30, 6, 157:1--12.
16. Lang, M., Wang, O., Aydin, T., Smolic, A., and Gross, M. 2012. Practical temporal consistency for image-based graphics applications. ACM Trans. Graph. 31, 4, 34:1--8.
17. Liu, J., Sun, J., and Shum, H.-Y. 2009. Paint selection. ACM Trans. Graph. 28, 3, 69:1--7.
18. Luo, L., Li, H., Paris, S., Weise, T., Pauly, M., and Rusinkiewicz, S. 2012. Multi-view hair capture using orientation fields. In Proc. CVPR 2012, 1490--1497.
19. Marschner, S., Jensen, H. W., Cammarano, M., Worley, S., and Hanrahan, P. 2003. Light scattering from human hair fibers. ACM Trans. Graph. 22, 3, 780--791.
20. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proc. SIGGRAPH '01, 433--442.
21. Paris, S., Briceño, H., and Sillion, F. 2004. Capture of hair geometry from multiple images. ACM Trans. Graph. 23, 3, 712--719.
22. Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1--9.
23. Perlin, K. 2002. Improving noise. In Proc. SIGGRAPH '02, 681--682.
24. Piuze, E., Kry, P. G., and Siddiqi, K. 2011. Generalized helicoids for modeling hair geometry. Computer Graphics Forum 30, 2, 247--256.
25. Selle, A., Lentine, M., and Fedkiw, R. 2008. A mass spring model for hair simulation. ACM Trans. Graph. 27, 3, 64:1--11.
26. Tan, P., Fang, T., Xiao, J., Zhao, P., and Quan, L. 2008. Single image tree modeling. ACM Trans. Graph. 27, 5, 108:1--7.
27. Vlasic, D., Brand, M., Pfister, H., and Popovic, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24, 3, 426--433.
28. Wang, L., Yu, Y., Zhou, K., and Guo, B. 2009. Example-based hair geometry synthesis. ACM Trans. Graph. 28, 3, 56:1--9.
29. Ward, K., Bertails, F., Kim, T.-Y., Marschner, S. R., Cani, M.-P., and Lin, M. C. 2007. A survey on hair modeling: styling, simulation, and rendering. IEEE Transactions on Visualization and Computer Graphics 13, 2, 213--234.
30. Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. ACM Trans. Graph. 24, 3, 816--820.
31. Yamaguchi, T., Wilburn, B., and Ofek, E. 2009. Video-based modeling of dynamic hair. In the 3rd Pacific Rim Symposium on Advances in Image and Video Technology, 585--596.
32. Yang, F., Wang, J., Shechtman, E., Bourdev, L., and Metaxas, D. 2011. Expression flow for 3D-aware face component transfer. ACM Trans. Graph. 30, 4, 60:1--10.
33. Yuksel, C., Schaefer, S., and Keyser, J. 2009. Hair meshes. ACM Trans. Graph. 28, 5.
34. Zhang, Q., Tong, J., Wang, H., Pan, Z., and Yang, R. 2012. Simulation guided hair dynamics modeling from video. Computer Graphics Forum 31, 7, 2003--2010.
35. Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: Cuboid proxies for smart image manipulation. ACM Trans. Graph. 31, 4, 99:1--11.
36. Zhou, S., Fu, H., Liu, L., Cohen-Or, D., and Han, X. 2010. Parametric reshaping of human bodies in images. ACM Trans. Graph. 29, 4, 126:1--10.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

    • Image manipulation

    • Rendering

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Graphics input devices