Abstract
Content-based web image retrieval based on the query-by-example (QBE) principle remains a challenging problem due to the semantic gap as well as the gap between a user's intent and the representativeness of a typical image query. In this article, we propose to address this problem by integrating query-related contextual information into an advanced query model to improve the performance of QBE-based web image retrieval. We consider both the local and global context of the query image. The local context can be inferred from the web pages and the click-through log associated with the query image, while the global context is derived from the entire corpus comprising all web images and the associated web pages. To effectively incorporate the local query context we propose a language modeling based approach to deal with the combined structured query representation from the contextual and visual information. The global query context is integrated by the multi-modal relevance model to “reconstruct” the query from the document models indexed in the corpus. In this way, the global query context is employed to address the noise or missing information in the query and its local context, so that a comprehensive and robust query model can be obtained. We evaluated the proposed approach on a representative product image dataset collected from the web and demonstrated that the inclusion of the local and global query contexts significantly improves the performance of QBE-based web image retrieval.
- Belkin, N. J. 2008. Some(what) grand challenges for information retrieval. In Proceedings of the 30th European Conference on IR Research on Advances in Information Retrieval (ECIR'08). 1. Google Scholar
Digital Library
- Cao, L., Luo, J., Kautz, H., and Huang, T. 2009. Image annotation within the context of personal photo collections using hierarchical event and scene models. IEEE Trans. Multimed. 11, 2, 208--219. Google Scholar
Digital Library
- Chang, S.-F., Hsu, W., Jiang, W., Kennedy, L., Xu, D., Yanagawa, A., and Zavesky, E. 2006. Columbia University TRECVID-2006 video search and high-level feature extraction. In Proceedings of the NIST TRECVID Workshop.Google Scholar
- Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. 40, 2, 1--60. Google Scholar
Digital Library
- Dou, Z., Song, R., Wen, J.-R., and Yuan, X. 2008. Evaluating the effectiveness of personalized web search. IEEE Trans. Knowl. Data Eng. 21, 1178--1190. Google Scholar
Digital Library
- Geng, B., Yang, L., and Xu, C. 2009. A study of language model for image retrieval. In Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW '09). IEEE Computer Society, Los Alamitos, CA, 158--163. Google Scholar
Digital Library
- Järvelin, K. and Kekäläinen, J. 2000. Ir evaluation methods for retrieving highly relevant documents. In Proceedings of the 23th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '00). 41--48. Google Scholar
Digital Library
- Jing, F., Li, M., Zhang, H.-J., and Zhang, B. 2005. A unified framework for image retrieval using keyword and visual features. IEEE Trans. Image Process. Google Scholar
Digital Library
- Lafferty, J. and Zhai, C. 2001. Document language models, query models, and risk minimization for information retrieval. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '01). ACM, New York, 111--119. Google Scholar
Digital Library
- Lavrenko, V., Choquette, M., and Croft, W. B. 2002. Cross-lingual relevance models. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '02). ACM, New York, 175--182. Google Scholar
Digital Library
- Lavrenko, V. and Croft, W. B. 2001. Relevance based language models. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '01). ACM, New York, 120--127. Google Scholar
Digital Library
- Lawrence, S. 2000. Context in web search. IEEE Data Eng. Bull. 23, 3, 25--32.Google Scholar
- Li, D., Yang, L., Hua, X.-S., and Zhang, H.-J. 2010. Large-scale robust visual codebook construction. In Proceedings of the 18th ACM International Conference on Multimedia (MULTIMEDIA '10). Google Scholar
Digital Library
- Luo, J., Hanjalic, A., Tian, Q., and Jaimes, A. 2009. Integration of context and content for multimedia management: An introduction to the special issue. IEEE Trans. Multimed. 11, 2(Jan.), 193--195. Google Scholar
Digital Library
- Manning, C. D., Raghavan, P., and Schütze, H. 2008. Introduction to Information Retrieval. Cambridge University Press. Google Scholar
Digital Library
- Mei, T., Hua, X.-S., and Li, S. 2008a. Contextual in-image advertising. In Proceeding of the 16th ACM International Conference on Multimedia (MULTIMEDIA '08). ACM, New York, 439--448. Google Scholar
Digital Library
- Mei, T., Zha, Z.-J., Liu, Y., Wang, M., Qi, G.-J., Tian, X., Wang, J., Yang, L., and Hua, X.-S. 2008b. MSRA at TRECVID 2008: High-level feature extraction and automatic search. In Proceedings of TRECVID.Google Scholar
- Page, L., Brin, S., Motwani, R., and Winograd, T. 1998. The pagerank citation ranking: Bringing order to the web. Tech. rep., Stanford Digital Library Technologies Project.Google Scholar
- Philbin, J., Chum, O., Isard, M., Sivic, J., and Zisserman, A. 2007. Object retrieval with large vocabularies and fast spatial matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '07). 1--8.Google Scholar
- Robertson, S. and Zaragoza, H. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retriev. 3, 333--389. Google Scholar
Digital Library
- Robertson, S., Zaragoza, H., and Taylor, M. 2004. Simple bm25 extension to multiple weighted fields. In Proceedings of the 13th ACM International Conference on Information and Knowledge Management (CIKM '04). ACM, New York, 42--49. Google Scholar
Digital Library
- Rui, Y., Huang, T., Ortega, M., and Mehrotra, S. 2002. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans. Circ. Syst. Video Tech. 8, 5, 644--655. Google Scholar
Digital Library
- Scholer, F. and Williams, H. E. 2002. Query association for effective retrieval. In Proceedings of the 11th International Conference on Information and Knowledge Management (CIKM '02). ACM, New York, 324--331. Google Scholar
Digital Library
- Scholer, F., Williams, H. E., and Turpin, A. 2004. Query association surrogates for web search: Research articles. J. Amer. Soc. Info. Sci. Technol. 55, 7, 637--650. Google Scholar
Digital Library
- Shen, H., Ooi, B., and Tan, K. 2000. Giving meanings to www images. In Proceedings of the 8th ACM International Conference on Multimedia (MULTIMEDIA '00). 39--47. Google Scholar
Digital Library
- Sinha, P. and Jain, R. 2008. Semantics in digital photos: A contextual analysis. In Proceedings of the IEEE International Conference on Semantic Computing (ICSC '08). IEEE Computer Society, Los Alamitos, CA, 58--65. Google Scholar
Digital Library
- Sivic, J. and Zisserman, A. 2003. Video google: A text retrieval approach to object matching in videos. In Proceedings of the 9th IEEE International Conference on Computer Vision (ICCV '03). IEEE Computer Society, Los Alamitos, CA, 1470. Google Scholar
Digital Library
- Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Patt. Anal. Machine Intel. 22, 12, 1349--1380. Google Scholar
Digital Library
- Snoek, C. G. M. and Worring, M. 2009. Concept-based video retrieval. Found. Trends Inf. Retr. 4, 2, 215--322. Google Scholar
Digital Library
- Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., and Hua, X.-S. 2008. Bayesian video search reranking. In Proceeding of the 16th ACM International Conference on Multimedia (MULTIMEDIA '08). Google Scholar
Digital Library
- Vapnik, V. N. 1998. Statistical Learning Theory. Wiley.Google Scholar
- Wang, X.-J., Ma, W.-Y., Xue, G.-R., and Li, X. 2004. Multi-model similarity propagation and its application for web image retrieval. In Proceedings of the 12th Annual ACM International Conference on Multimedia (MULTIMEDIA '04). ACM, New York, 944--951. Google Scholar
Digital Library
- Wang, X. J., Zhang, L., Li, X., and Ma, W. Y. 2008. Annotating images by mining image search results. IEEE Trans. Patt. Anal. Mach. Intell. 30, 11, 1919--1932. Google Scholar
Digital Library
- Westerveld, T., Vries, A. P. D., van Ballegooij, A., de Jong, F., and Hiemstra, D. 2003. A probabilistic multimedia retrieval model and its evaluation. EURASIP J. Appl. Signal Process. 2003, 186--198. Google Scholar
Digital Library
- Wilkinson, R. 1994. Effective retrieval of structured documents. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '94). Springer-Verlag, Berlin, 311--317. Google Scholar
Digital Library
- Xue, G.-R., Zeng, H.-J., Chen, Z., Yu, Y., Ma, W.-Y., Xi, W., and Fan, W. 2004. Optimizing web search using web click-through data. In Proceedings of the 13th ACM International Conference on Information and Knowledge Management (CIKM '04). ACM, New York, 118--126. Google Scholar
Digital Library
- Yan, R., Hauptmann, A. G., and Jin, R. 2003. Multimedia search with pseudo-relevance feedback. In Proceedings of the International Conference on Image and Video Retrieval (CIVR '03). Google Scholar
Digital Library
- Yang, L., Geng, B., Hanjalic, A., and Hua, X.-S. 2010. Contextual image retrieval model. In Proceedings of the ACM International Conference on Image and Video Retrieval (CIVR '10). ACM, New York, 406--413. Google Scholar
Digital Library
- Yang, L. and Hanjalic, A. 2010. Supervised reranking for web image search. In Proceeding of the 18th ACM International Conference on Multimedia (MULTIMEDIA '10). Google Scholar
Digital Library
- Yang, Y., Xu, D., Nie, F., Luo, J., and Zhuang, Y. 2009. Ranking with local regression and global alignment for cross media retrieval. In Proceedings of the 17th ACM International Conference on Multimedia (MULTIMEDIA '09). ACM, New York, 175--184. Google Scholar
Digital Library
- Yang, Y. H., Wu, P. T., Lee, C. W., Lin, K. H., Hsu, W. H., and Chen, H. H. 2008. Contextseer: Context search and recommendation at query time for shared consumer photos. In Proceeding of the 16th ACM International Conference on Multimedia (MULTIMEDIA '08). Google Scholar
Digital Library
- Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z., Chua, T.-S., and Hua, X.-S. 2010. Visual query suggestion: Towards capturing user intent in internet image search. ACM Trans. Multimed. Comput. Commun. Appl. 6. Google Scholar
Digital Library
- Zhai, C. 2009. Statistical Language Models for Information Retrieval. Morgan & Claypool. Google Scholar
Digital Library
- Zhai, C. and Lafferty, J. 2004. A study of smoothing methods for language models applied to information retrieval. ACM Trans. Inf. Syst. Sec. 22, 2, 179--214. Google Scholar
Digital Library
- Zhang, H.-J. 2009. Multimedia content analysis and search: New perspectives and approaches. www.acmmm09.org/ACM MM09 Keynote.pdf. Google Scholar
Digital Library
- Zhang, R., Zhang, Z. M., Li, M., Ma, W.-Y., and Zhang, H.-J. 2005. A probabilistic semantic model for image annotation and multi-modal image retrieva. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV '05). IEEE Computer Society, Los Alamitos, CA, 846--851. Google Scholar
Digital Library
- Zhao, R. and Grosky, W. I. 2002. Narrowing the semantic gap - improved text-based web document retrieval using visual features. IEEE Trans. Multimed. 4, 2, 189--200. Google Scholar
Digital Library
- Zhou, X. S. and Huang, T. S. 2003. Relevance feedback in image retrieval: A comprehensive review. Multimed. Syst. 8, 6, 536--544.Google Scholar
Cross Ref
Index Terms
A unified context model for web image retrieval
Recommendations
Web Image Retrieval for Abstract Queries Using Text and Image Information
AIRS '09: Proceedings of the 5th Asia Information Retrieval Symposium on Information Retrieval TechnologyIn this paper, we propose a method for image retrieval on the web. In this task, we focus on abstract words that do not directly link to images that we want. For example, a user might use a query "summer" to retrieve images of "fireworks" or "a white ...
Context-Aware Recommendation System Using Content Based Image Retrieval with Dynamic Context Considered
SITIS '13: Proceedings of the 2013 International Conference on Signal-Image Technology & Internet-Based SystemsThis paper conceptually proposes a context-aware recommendation system that gives/recommends optimal information for users based on 1) the content of an image using content-based image retrieval engines, 2) the contextual information of its similar ...
Web image retrieval systems with automatic web image annotating techniques
Due to the popularity of digital cameras and web authors'enriching the visual aesthetics, the number of web images is growing in an uncontrolled speed. The images in the World Wide Web are becoming a large image library for browsing. It is an important ...






Comments