Abstract
In this article, we consider how to automatically create pleasing photo collages created by placing a set of images on a limited canvas area. The task is formulated as an optimization problem. Differently from existing state-of-the-art approaches, we here exploit subjective experiments to model and learn pleasantness from user preferences. To this end, we design an experimental framework for the identification of the criteria that need to be taken into account to generate a pleasing photo collage. Five different thematic photo datasets are used to create collages using state-of-the-art criteria. A first subjective experiment where several subjects evaluated the collages, emphasizes that different criteria are involved in the subjective definition of pleasantness. We then identify new global and local criteria and design algorithms to quantify them. The relative importance of these criteria are automatically learned by exploiting the user preferences, and new collages are generated. To validate our framework, we performed several psycho-visual experiments involving different users. The results shows that the proposed framework allows to learn a novel computational model which effectively encodes an inter-user definition of pleasantness. The learned definition of pleasantness generalizes well to new photo datasets of different themes and sizes not used in the learning. Moreover, compared with two state-of-the-art approaches, the collages created using our framework are preferred by the majority of the users.
- Borji Ali, Cheng Ming-Ming, Jiang Huaizu, and Li Jia. 2014. Salient Object Detection: A Survey. (2014) arXiv:1411.5878 {cs.CV}.Google Scholar
- Sebastiano Battiato, Gianluigi Ciocca, Francesca Gasparini, Giovanni Puglisi, and Raimondo Schettini. 2008. Smart photo sticking. In Adaptive Multimedia Retrieval: Retrieval, User, and Semantics, Lecture Notes in Computer Science, vol. 4918. Springer, 211--223. Google Scholar
Digital Library
- Simone Bianco and Raimondo Schettini. 2012. Sampling optimization for printer characterization by direct search. IEEE Trans. Image Process. 21, 12, 4868--4873. Google Scholar
Digital Library
- Simone Bianco and Francesco Tisato. 2012. Sensor placement optimization in buildings. In Image Processing: Machine Vision Applications V, vol. 8300, SPIE, 830003.Google Scholar
- J. Calic, D. P. Gibson, and N. W. Campbell. 2007. Efficient layout of comic-like video summaries. IEEE Trans. Circ. Syst. Video Tech. 17, 7, 931--936. Google Scholar
Digital Library
- Hui Chao, Daniel R. Tretter, Xuemei Zhang, and C. Brian Atkins. 2010. Blocked recursive image composition with exclusion zones. In Proceedings of the 10th ACM Symposium on Document Engineering (DocEng'10). ACM, 111--114. Google Scholar
Digital Library
- Jun-Cheng Chen, Wei-Ta Chu, Jin-Hau Kuo, Chung-Yi Weng, and Ja-Ling Wu. 2006. Tiling slideshow. In Proceedings of the 14th Annual ACM International Conference on Multimedia (MULTIMEDIA'06). ACM, 25--34. Google Scholar
Digital Library
- M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, and S. Hu. 2015. Global contrast based salient region detection. IEEE Trans. Patt. Anal. Mach. Intell. 37, 3, 569--582.Google Scholar
Cross Ref
- Gianluigi Ciocca and Raimondo Schettini. 2010. Multiple image thumbnailing. In Digital Photography VI, vol. 7537, SPIE, 75370S.Google Scholar
- Nicholas Diakopoulos and Irfan Essa. 2005. Mediating photo collage authoring. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST'05). ACM, New York, 183--186. Google Scholar
Digital Library
- K. Duncan and S. Sarkar. 2012. Saliency in images and video: A brief survey. Computer Vision, 6, 6, 514--523.Google Scholar
Cross Ref
- Russ C. Eberhart and James Kennedy. 1995. A new optimizer using particle swarm theory. In Proceedings of the 6th International Symposium on Micro Machine and Human Science. vol. 1, New York, 39--43.Google Scholar
- Hesam Ekhtiyar, Mahdi Sheida, and Mahmood Amintoosi. 2011. Picture collage with genetic algorithm and stereo vision. Int. J. Comput. Sci. Iss. 8, 4, 165--169.Google Scholar
- Jian Fan. 2012. Photo layout with a fast evaluation method and genetic algorithm. In Proceedings of the 2012 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, 308--313. Google Scholar
Digital Library
- Andreas Girgensohn and Patrick Chiu. 2003. Stained glass photo collages. In Proceedings of the IEEE International Conference on Image Processing. vol. 2. 871--874.Google Scholar
- Stas Goferman, Ayellet Tal, and Lihi Zelnik-Manor. 2010. Puzzle-like Collage. Comput. Graph. Forum 29, 2, 459--468.Google Scholar
Cross Ref
- David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization and Machine Learning (1st Ed.). Addison-Wesley Longman Publishing Co., Inc., Boston, MA. Google Scholar
Digital Library
- Robert Hooke and To A. Jeeves. 1961. “Direct search” solution of numerical and statistical problems. J. ACM 8, 2, 212--229. Google Scholar
Digital Library
- Hua Huang, Lei Zhang, and Hong-Chao Zhang. 2011. Arcimboldo-like collage using internet images. ACM Trans. Graph. 30, 6, 155:1--155:8. Google Scholar
Digital Library
- Phillip Isola, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2011. What makes an image memorable?. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 145--152. Google Scholar
Digital Library
- Maurice G. Kendall. 1938. A new measure of rank correlation. Biometrika 30, 1/2, 81--93.Google Scholar
- Aditya Khosla, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2012. Memorability of image regions. In NIPS, Vol. 2. 4.Google Scholar
- Akisato Kimura, Ryo Yonetani, and Takatsugu Hirayama. 2013. Computational models of human visual attention and their implementations: A survey. IEICE Trans. Inf. Syst. 96, 3, 562--578.Google Scholar
Cross Ref
- Tamara G. Kolda, Robert Michael Lewis, and Virginia Torczon. 2003. Optimization by direct search: New perspectives on some classical and modern methods. SIAM Review 45, 3, 385--482.Google Scholar
Cross Ref
- Man Hee Lee, Nitin Singhal, Sungdae Cho, and In Kyu Park. 2010. Mobile photo collage. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 24--30.Google Scholar
Cross Ref
- Tie Liu, Jingdong Wang, Jian Sun, Nanning Zheng, Xiaoou Tang, and Heung-Yeung Shum. 2009. Picture collage. IEEE Trans. Multimed. 11, 7, 1225--1239. Google Scholar
Digital Library
- Sheng-Jie Luo, Chun-Yu Tsai, Wei-Chao Chen, and Bing-Yu Chen. 2013. Dynamic media assemblage. IEEE Trans. Circ. Syst. Video Tech. 23, 12, 2044--2053. Google Scholar
Digital Library
- Yu-Fei Ma and Hong-Jiang Zhang. 2003. Contrast-based image attention analysis by using fuzzy growing. In Proceedings of the 11th ACM International Conference on Multimedia (MULTIMEDIA'03). ACM, 374--381. Google Scholar
Digital Library
- Tao Mei, Bo Yang, Shi-Qiang Yang, and Xian-Sheng Hua. 2009. Video collage: Presenting a video sequence using a single image. Visual Computer 25, 1, 39--51. Google Scholar
Digital Library
- A. Mittal, A. K. Moorthy, and A. C. Bovik. 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12, 4695--4708. Google Scholar
Digital Library
- John A. Nelder and Roger Mead. 1965. A simplex method for function minimization. Computer J. 7, 4, 308--313.Google Scholar
Cross Ref
- Li-Chen Ou and M. Ronnier Luo. 2006. A colour harmony model for two-colour combinations. Color Res. Appl. 31, 3, 191--204.Google Scholar
Cross Ref
- Carsten Rother, Lucas Bordeaux, Youssef Hamadi, and Andrew Blake. 2006. Autocollage. In ACM Trans. Graphics. vol. 25. ACM, New York, 847--852. Google Scholar
Digital Library
- Carsten Rother, Sanjiv Kumar, Vladimir Kolmogorov, and Andrew Blake. 2005. Digital tapestry {automatic image synthesis}. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'05). vol. 1. IEEE, 589--596. Google Scholar
Digital Library
- Philipp Sandhaus, Mohammad Rabbath, and Susanne Boll. 2011. Employing aesthetic principles for automatic photo book layout. In Advances in Multimedia Modeling, Lecture Notes in Computer Science, vol. 6523, Springer, Berlin Heidelberg, 84--95. Google Scholar
Digital Library
- Martin Solli and Reiner Lenz. 2009. Color harmony for image indexing. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), 1885--1892.Google Scholar
Cross Ref
- Jingdong Wang, Long Quan, Jian Sun, Xiaoou Tang, and Heung-Yeung Shum. 2006. Picture collage. In Proceedings of the 2006 IEEE Conference on Computer Vision and Pattern Recognition. vol. 1. IEEE, 347--354.Google Scholar
- Yichen Wei, Yasuyuki Matsushita, and Yingzhen Yang. 2009. Efficient optimization of photo collage. Tech. Rep. MSRTR-2009-59, Microsoft Research.Google Scholar
- Zhipeng Wu and K. Aizawa. 2013. PicWall: Photo collage on-the-fly. In Proceedings of the 2013 Asia -- Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE, 1--10.Google Scholar
- Zhipeng Wu and Kiyoharu Aizawa. 2014. Building friend wall for local photo repository by using social attribute annotation. J. Multimedia 9, 1, 4--13.Google Scholar
Cross Ref
- Yingzhen Yang, Yichen Wei, Chunxiao Liu, Qunsheng Peng, and Yasuyuki Matsushita. 2009. An improved belief propagation method for dynamic collage. Visual Computer 25, 5--7, 431--439. Google Scholar
Digital Library
- Zongqiao Yu, Lin Lu, Yanwen Guo, Rongfei Fan, Mingming Liu, and Wenping Wang. 2014. Content-aware photo collage using circle packing. IEEE Trans. Visual. Comput. Graph. 20, 2, 182--195. Google Scholar
Digital Library
Index Terms
User Preferences Modeling and Learning for Pleasing Photo Collage Generation
Recommendations
ILMICA - Interactive Learning Model of Image Collage Assessment: A Transfer Learning Approach for Aesthetic Principles
MultiMedia ModelingAbstractThe beauty of moments can be expressed in many ways. One of them is the image collage which captures events and expresses emotions. Nowadays there is a large number of digital images. Aesthetic analyses of image collages are rarely performed due ...
Pseudo-3D photo collage
SIGGRAPH '02: ACM SIGGRAPH 2002 conference abstracts and applicationsPseudo-3D photo collage is a new technique for creating extensive pseudo-3D scenes on the Web. This technique enables users to create, publish, navigate and share pseudo-3D scenes by easy operations. Our basic idea comes from an artistic representation "...
An Abstract Painting Generation Method Based on Deep Generative Model
AbstractComputer technology provides new conditions and possibilities for art creation and research, and also expands the forms of artistic expression. Computer-created art has thus become one of the important forms of art. In this paper, we proposed a ...






Comments