Synthesis of tiled patterns using factor graphs

Abstract

Patterns with pleasing structure are common in art, video games, and virtual worlds. We describe a method for synthesizing new patterns of tiles on a regular grid that are similar in appearance to a set of example patterns. Exemplars are used both to specify valid tile arrangements and to emphasize multi-tile structures. We model a pattern as a probabilistic graphical model called a factor graph. Factors represent the hard logical constraints between tiles, the soft statistical relationships that determine style, and the local dependencies between tiles at neighboring sites. We describe a simple method for learning factor functions from a small exemplar. We then synthesize new patterns through a stochastic search method that is inspired by MC-SAT. Efficient synthesis is challenging because of the combination of hard and soft constraints. Our synthesis algorithm, called BlockSS, scales linearly with the number of tiles and the hardness of the problem. We use our technique to model building facades, cities, and decorative patterns.

References

1. Ashikhmin, M. 2001. Synthesizing natural textures. In Proceedings of the Symposium on Interactive 3D Graphics. ACM, 217--226.
2. Bokeloh, M., Wand, M., and Seidel, H.-P. 2010. A connection between partial symmetry and inverse procedural modeling. ACM Trans. Graph. 29, 4.
3. Campbell, C. 2006. Vitruvius Britannicus: The Classic of Eighteenth-Century British Architecture. Dover Publications.
4. Cho, T. S., Avidan, S., and Freeman, W. T. 2010. A probabilistic image jigsaw puzzle solver. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
5. Cohen, M., Shade, J., Hiller, S., and Deussen, O. 2003. Wang tiles for image and texture generation. ACM Trans. Graph. 22, 3, 287--294.
6. Edwards, R. and Sokal, A. 1988. Generalization of the Fortuin-Kasteleyn-Swendsen-Wang representation and Monte Carlo algorithm. Phys. Rev. D 38, 6, 2009--2012.
7. Efros, A. and Freeman, W. 2001. Image quilting for texture synthesis and transfer. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques ACM SIGGRAPH.
8. Efros, A. A. and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In ICCV (2). 1033--1038.
9. Gent, I., Jefferson, C., and Miguel, I. 2006. MINION: A fast, scalable, constraint solver. In Proceeding of the 17th European Conference on Artificial Intelligence. 98--102.
10. Goldberger, J., Gordon, S., and Greenspan, H. 2003. An efficient image similarity measure based on approximations of KL-divergence between two Gaussian mixtures. In Proceedings of the International Conference on Computer Vision (ICCV). 487--493.
11. Hastings, W. K. 1970. Monte carlo sampling methods using markov chains and their applications. Biometrika 57, 1, pp. 97--109.
12. Koller, D. and Friedman, N. 2009. Probabilistic Graphical Models. MIT Press.
13. Kschischang, F. R., Frey, B. J., and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47, 498-- 519.
14. Kwatra, V., Schdl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3, 277--286.
15. Lagae, A., Kaplan, C. S., Fu, C.-W., Ostromoukhov, V., and Deussen, O. 2008. Tile-Based methods for interactive applications. In ACM SIGGRAPH Classes. ACM, New York, 93:1--93:267.
16. Loeliger, H. A. 2004. An introduction to factor graphs. IEEE Signal Process. Mag. 21, 1, 28--41.
17. Merrell, P. 2007. Example-Based model synthesis. In (I3D '07) Proceedings of the Symposium on Interactive 3D Graphics and Games. ACM, New York, 105--112.
18. Mira, A. and Tierney, L. 1997. On the use of auxiliary variables in Markov chain Monte Carlo sampling. Scand. J. Statist.
19. Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Gool, L. V. 2006. Procedural modeling of buildings. ACM Trans. Graph. 25, 3, 614-- 623.
20. Neal, R. 2000. Slice sampling. Ann. Statist. 31, 705--767.
21. Parish, Y. I. H. and Müller, P. 2001. Procedural modeling of cities. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press, New York, 301--308.
22. Paskin, M. 2002. Maximum-Entropy probabilistic logics. Tech. rep., University of California at Berkeley.
23. Poon, H. and Domingos, P. 2006. Sound and efficient inference with probabilistic and deterministic dependencies. In Proceedings of the 21st National Conference on Artificial Intelligence. AAAI Press, 458-- 463.
24. Prusinkiewicz, P. and Lindenmayer, A. 1996. The Algorithmic Beauty of Plants. Springer.
25. Son, G. B. and Co. 1976. Victorian Frames, Borders and Cuts: From the 1882 Type Catalog of George Bruce's Son and Co. Dover Publications.
26. Wei, L.-Y., Lefebvre, S., Kwatra, V., and Turk, G. 2009. State of the art in example-based texture synthesis. In Eurographics State of the Art Report, EG-STAR. Eurographics Association.
27. Wei, L.-Y. and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. In Computer Graphics Proceedings, K. Akeley, Ed., ACM Press Addison Wesley Longman, 479--488.
28. Wei, W., Erenrich, J., and Selman, B. 2004. Towards efficient sampling: Exploiting random walk strategies. In Proceedings of the 19th National Conference on Artifical Intelligence. AAAI Press/The MIT Press, 670--676.
29. Wong, M. T., Zongker, D. E., and Salesin, D. H. 1998. Computer-generated floral ornament. Comput. Graph. 32. Annual Conference Series, 423--434.
30. Yokoo, M. 1997. Why adding more constraints makes a problem easier for hill-climbing algorithms: Analyzing landscapes of csps. Principl. and Pract. Constr. Program.-CP97, 356--370.
31. Zhu, S. C., Wu, Y., and Mumford, D. 1998. Filters, random fields and maximum entropy (frame) -- Towards a unified theory for texture modeling. Int. J. Comput. Vis. 27, 2, 1--20.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering