Abstract
In this article, we propose a novel personalized ranking system for amateur photographs. The proposed framework treats the photograph assessment as a ranking problem and we introduce the idea of personalized ranking, which ranks photographs considering both their aesthetic qualities and personal preferences. Photographs are described using three types of features: photo composition, color and intensity distribution, and personalized features. An aesthetic prediction model is learned from labeled photographs by using the proposed image features and RBF-ListNet learning algorithm. The experimental results show that the proposed framework outperforms in the ranking performance: a Kendall's tau value of 0.432 is significantly higher than those obtained by the features proposed in one of the state-of-the-art approaches (0.365) and by learning based on support vector regression (0.384). To realize personalization in ranking, three approaches are proposed: the feature-based approach allows users to select photographs with specific rules, the example-based approach takes the positive feedback from users to rerank the photograph, and the list-based approach takes both positive and negative feedback from users into consideration. User studies indicate that all three approaches are effective in both aesthetic and personalized ranking.
- Subhabrata Bhattacahrya, Rahul Sukthankar, and Mubarak Shah. 2010. A framework for photo-quality assessment and enhancement based on visual aesthetics. In Proceedings of the International Conference on Multimedia (MM'10). ACM Press, New York, 271--280. Google Scholar
Digital Library
- John Canny. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6, 679--698. Google Scholar
Digital Library
- Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: From pairwise approach to listwise approach. In Proceedings of the 24th International Conference on Machine Learning. ACM Press, New York, 129--136. Google Scholar
Digital Library
- Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 3, 27:1--27:27. http://www.csie.ntu.edu.tw/∼cjlin/libsvm. Google Scholar
Digital Library
- Rama Chellappa. 1989. Two-dimensional discrete Gaussian markov random field models for image processing. J. Institut. Electron. Telecomm. Engin. 35, 2, 114--120.Google Scholar
- Tianping Chen and Hong Chen. 1995. Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks. IEEE Trans. Neural Netw. 6, 4, 904--910. Google Scholar
Digital Library
- Bin Cheng, Bingbing Ni, Shuicheng Yan, and Qi Tian. 2010. Learning to photograph. In Proceedings of the International Conference on Multimedia (MM'10). ACM Press, New York, 291--300. Google Scholar
Digital Library
- Daniel Cohen-Or, Olga Sorkine, Ran Gal, Tommer Leyvand, and Ying-Qing Xu. 2006. Color harmonization. ACM Trans. Graph. 25, 3, 624--630. Google Scholar
Digital Library
- Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3, 273--297. Google Scholar
Digital Library
- Gabriella Csurka, Christopher R. Dance, Lixin Fan, Jutta Willamowski, and Cedric Bray. 2004. Visual categorization with bags of keypoints. In Proceedings of the Workshop on Statistical Learning in Computer Vision (ECCV'04). 1--22.Google Scholar
- Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang. 2006. Studying aesthetics in photographic images using a computational approach. In Proceedings of the 9th European Conference on Computer Vision (ECCV'06). 7--13. Google Scholar
Digital Library
- Richard O. Duda and Peter E. Hart. 1972. Use of the hough transformation to detect lines and curves in pictures. Comm. ACM 15, 1, 11--15. Google Scholar
Digital Library
- Pedro F. Felzenszwalb and Daniel P. Huttenlocher. 2004. Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 2, 167--181. Google Scholar
Digital Library
- Tom Grill and Mark Scanlon. 1990. Photographic Composition. Amphoto Books.Google Scholar
- Jonathan Harel, Christof Koch, and Pietro Perona. 2007. Graph-based visual saliency. Adv. Neural Inf. Process. Syst. 19, 545--552.Google Scholar
Digital Library
- Chung-Jung Hu. 2007. A real-time skin-color-enhanced face detection algorithm. Masters thesis, National Taiwan University, Taipei, Taiwan.Google Scholar
- Nicolaos B. Karayiannis and Mary M. Randolph-Gips. 2003. On the construction and training of reformulated radial basis function neural networks. IEEE Trans. Neural Netw. 14, 4, 835--846. Google Scholar
Digital Library
- Yan Ke, Xiaoou Tang, and Feng Jing. 2006. The design of high-level features for photo quality assessment. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 1, 419--426. Google Scholar
Digital Library
- Bert P. Krages. 2005. Photography: The Art of Composition. Allworth Press.Google Scholar
- William H. Kruskal. 1958. Ordinal measures of association. J. Amer. Statist. Assoc. 53, 284, 814--861.Google Scholar
Cross Ref
- Michael S. Lew, Nicu Sebe, Chabane Djeraba, and Ramesh Jain. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Comm. Appl. 2, 1, 1--19. Google Scholar
Digital Library
- Ligang Liu, Renjie Chen, Lior Wolf, and Daniel Cohen-Or. 2010. Optimizing photo composition. Comput. Graph. Forum 29, 2.Google Scholar
Cross Ref
- Yiwen Luo and Xiaoou Tang. 2008. Photo and video quality evaluation: Focusing on the subject. In Proceedings of the 10th European Conference on Computer Vision (ECCV'08). Springer, 386--399. Google Scholar
Digital Library
- Bangalore S. Manjunath and Wei-Ying Ma. 1996. Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 18, 8, 837--842. Google Scholar
Digital Library
- Luca Marchesotti, Florent Perronnin, Diane Larlus, and Gabriela Csurka. 2011. Assessing the aesthetic quality of photographs using generic image descriptors. In Proceedings of the International Conference on Computer Vision (ICCV'11). 1784--1791. Google Scholar
Digital Library
- Benjamin Martinez and Jacqueline Block. 1988. Visual Forces: An Introduction to Design. Prentice Hall.Google Scholar
- Masashi Nishiyama, Takahiro Okabe, Yoichi Sato, and Imari Sato. 2009. Sensation-based photo cropping. In Proceedings of the 17th ACM International Conference on Multimedia (MM'09). ACM Press, New York, 669--672. Google Scholar
Digital Library
- Aude Oliva and Antonio Torralba. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 3, 145--175. Google Scholar
Digital Library
- Florent Perronnin and Christopher Dance. 2007. Fisher kernels on visual vocabularies for image categorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07). 1--8.Google Scholar
Cross Ref
- Gabriele Peters. 2007. Aesthetic primitives of images for visualization. In Proceedings of the 11th International Conference on Information Visualization (IV'07). 316--325. Google Scholar
Digital Library
- Vera Rivotti, Joao Proenaa, Joaquim Jorge, and Mario Sousa. 2007. Composition principles for quality depiction and aesthetics. In Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging. 37--44. Google Scholar
Digital Library
- Yong Man Ro, Munchurl Kim, H. K. Kang, B. S. Manjunath, and Jinwoong Kim. 2001. MPEG-7 homogeneous texture descriptor. ETRI J. 23, 2, 41--51.Google Scholar
Cross Ref
- Yong Rui and Thomas Huang. 2000. Optimizing learning in image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Vol. 1. 236--243.Google Scholar
Cross Ref
- Yong Rui, Thomas S. Huang, Michael Ortega, and Sharad Mehrotra. 1998. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans. Circ. Syst. Video Technol. 8, 5, 644--655. Google Scholar
Digital Library
- Jose San Pedro and Stefan Siersdorfer. 2009. Ranking and classifying attractiveness of photos in folksonomies. In Proceedings of the 18th International Conference on World Wide Web (WWW'09). ACM Press, New York, 771--780. Google Scholar
Digital Library
- Anthony Santella, Maneesh Agrawala, Doug DeCarlo, David Salesin, and Michael Cohen. 2006. Gaze-based interaction for semi-automatic photo cropping. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'06). ACM Press, New York, 771--780. Google Scholar
Digital Library
- Gaurav Sharma, Wencheng Wu, and Edul N. Dalal. 2005. The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Res. Appl. 30, 1, 21--30.Google Scholar
Cross Ref
- Hamid R. Sheikh, Alan C. Bovik, and Gustavo de Veciana. 2005. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process. 14, 12, 2117--2128. Google Scholar
Digital Library
- Xiaoshuai Sun, Hongxun Yao, Rongrong Ji, and Shaohui Liu. 2009. Photo assessment based on computational visual attention model. In Proceedings of the 17th ACM International Conference on Multimedia (MM'09). ACM Press, New York, 541--544. Google Scholar
Digital Library
- Hanghang Tong, Mingjing Li, Hong-Jiang Zhang, Jingrui He, and Changshui Zhang. 2004. Classification of digital photos taken by photographers or home users. In Proceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing. Lecture Notes in Computer Science, vol. 3331, Springer, 198--205. Google Scholar
Digital Library
- Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4, 600--612. Google Scholar
Digital Library
- Zhou Wang, Hamid R. Sheikh, and Alan C. Bovik. 2002. No-reference perceptual quality assessment of jpeg compressed images. In Proceedings of the International Conference on Image Processing. Vol. 1. 477--480.Google Scholar
- Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008. Listwise approach to learning to rank: Theory and algorithm. In Proceedings of the 25th International Conference on Machine Learning (ICML'08). ACM Press, New York, 1192--1199. Google Scholar
Digital Library
- Yang Yang Xiang and Mohan S. Kankanhalli. 2010. Automated aesthetic enhancement of videos. In Proceedings of the International Conference on Multimedia (MM'10). ACM Press, New York, 281--290. Google Scholar
Digital Library
- Seungji Yang, Sang-Kyun Kim, and Yong Man Ro. 2007. Semantic home photo categorization. IEEE Trans. Circ. Syst. Video Technol. 17, 3, 324--335. Google Scholar
Digital Library
- Yi-Hsuan Yang and Homer H. Chen. 2011. Ranking-based emotion recognition for music organization and retrieval. IEEE Trans. Audio Speech Lang. Process. 19, 4, 762--774. Google Scholar
Digital Library
- Che-Hua Yeh, Yuan-Chen Ho, Brian A. Barsky, and Ming Ouhyoung. 2010. Personalized photograph ranking and selection system. In Proceedings of the International Conference on Multimedia (MM'10). ACM Press, New York, 211--220. Google Scholar
Digital Library
- Xiang Sean Zhou and Thomas S. Huang. 2003. Relevance feedback in image retrieval: A comprehensive review. Multimedia Syst. 8, 6, 536--544.Google Scholar
Cross Ref
Index Terms
Personalized Photograph Ranking and Selection System Considering Positive and Negative User Feedback
Recommendations
Personalized photograph ranking and selection system
MM '10: Proceedings of the 18th ACM international conference on MultimediaIn this paper, we propose a novel personalized ranking system for amateur photographs. Although some of the features used in our system are similar to previous work, new features, such as texture, RGB color, portrait (through face detection), and black-...
An Architecture of an Academic Search Engine with Personalized Search Result Ranking Mechanism
ICNCC '16: Proceedings of the Fifth International Conference on Network, Communication and ComputingA rapid increasing of information on the Internet and World Wide Web causes information overloaded problem. Thus, search engines become important tools to help WWW users to discover the information they need. With an exponentially increasing of ...
Leveraging Social Connections to Improve Personalized Ranking for Collaborative Filtering
CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge ManagementRecommending products to users means estimating their preferences for certain items over others. This can be cast either as a problem of estimating the rating that each user will give to each item, or as a problem of estimating users' relative ...






Comments