FlexiStickers: photogrammetric texture mapping using casual images

Abstract

Texturing 3D models using casual images has gained importance in the last decade, with the advent of huge databases of images. We present a novel approach for performing this task, which manages to account for the 3D geometry of the photographed object. Our method overcomes the limitation of both the constrained-parameterization approach, which does not account for the photography effects, and the photogrammetric approach, which cannot handle arbitrary images. The key idea of our algorithm is to formulate the mapping estimation as a Moving-Least-Squares problem for recovering local camera parameters at each vertex. The algorithm is realized in a FlexiStickers application, which enables fast interactive texture mapping using a small number of constraints.

References

1. Bernardini, F., Martin, I. M., and Rushmeier, H. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. on Visualization and Computer Graphics 7, 4, 318--332.
2. Bregler, C., Malik, J., and Pullen, K. 2004. Twist based acquisition and tracking of animal and human kinematics. Int. J. Comput. Vision 56, 3, 179--194.
3. Brox, T., Rosenhahn, B., and Weickert. 2007. Three-dimensional shape knowledge for joint image segmentation and pose tracking. Int. J. Comput. Vision 73, 3, 243--262.
4. Colombo, C., Bimbo, A. D., and Pernici, F. 2005. Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1, 99--114.
5. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs:. Tech. rep., Berkeley, CA, USA.
6. Desbrun, M., Meyer, M., and Alliez, P. 2002. Intrinsic parameterizations of surface meshes. Computer Graphics Forum 21, 3, 209--218.
7. Gingold, Y. I., Davidson, P. L., Han, J. Y., and Zorin, D. 2006. A direct texture placement and editing interface. In UIST, 23--32.
8. Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision. Cambridge University Press.
9. Hormann, K., Lévy, B., and Sheffer, A. 2007. Mesh parameterization: theory and practice. In SIGGRAPH courses.
10. Kraevoy, V., Sheffer, A., and Gotsman, C. 2003. Matchmaker: constructing constrained texture maps. In SIGGRAPH, 326--333.
11. Lee, T., Yen, S., and Yeh, I. 2008. Texture mapping with hard constraints using warping scheme. IEEE Trans. on Visualization and Computer Graphics 14, 2, 382--395.
12. Lensch, H. P. A., Heidrich, W., and Seidel, H. 2000. Automated texture registration and stitching for real world models. In Pacific Graphics, 317.
13. Lévy, B., Petitjean, S., Ray, N., and Maillot, J. 2002. Least squares conformal maps for automatic texture atlas generation. In SIGGRAPH, 362--371.
14. Lévy, B. 2001. Constrained texture mapping for polygonal meshes. In SIGGRAPH, 417--424.
15. Murray, R. M., Sastry, S. S., and Zexiang, L. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press, Inc., Boca Raton, FL, USA.
16. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Trans. on Graphics 25, 3, 533--540.
17. Schmidt, R., Grimm, C., and Wyvill, B. 2006. Interactive decal compositing with discrete exponential maps. ACM Trans. on Graphics 25, 3, 605--613.
18. Sinha, S., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3D architectural modeling from unordered photo collections. ACM Trans. on Graphics 25, 5, 159:1--10.
19. Tai, Y.-W., Brown, M., Tang, C.-K., and Shum, H.-Y. 2008. Texture amendment: reducing texture distortion in constrained parameterization. ACM Trans. on Graphics 27, 5, 1--6.
20. Weinhaus, F. M., and D., V. 1997. Texture mapping 3D models of real-world scenes. ACM Comput. Surv. 29, 4, 325--365.
21. Weinhaus, F., and Devich, R. 1999. Photogrammetric texture mapping onto planar polygons. Graphical Models and Image Processing 61, 2, 63--83.
22. Xiao, J., Fang, T., Tan, P., Zhao, P., Ofek, E., and Quan, L. 2008. Image-based facade modeling. ACM Trans. on Graphics 25, 5, 161:1--10.
23. Zhang, E., Mischaikow, K., and Turk, G. 2005. Feature-based surface parameterization and texture mapping. ACM Trans. on Graphics 24, 1, 1--27.
24. Zhou, K., Wang, X., Tong, Y., Desbrun, M., Guo, B., and Shum, H. 2005. TextureMontage: seamless texturing of arbitrary surfaces from multiple images. ACM Trans. on Graphics 24, 3, 1148--1155.

Index Terms

• Applied computing

  • Physical sciences and engineering

    • Mathematics and statistics

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation

    • Shape modeling

• Mathematics of computing

  • Mathematical analysis

    • Mathematical optimization

    • Numerical analysis

• Theory of computation

  • Design and analysis of algorithms

    • Mathematical optimization

  • Randomness, geometry and discrete structures

    • Computational geometry