skip to main content
research-article

Improving Web search using image snippets

Published:06 October 2008Publication History
Skip Abstract Section

Abstract

The Web has become the largest information repository in the world; thus, effectively and efficiently searching the Web becomes a key challenge. Interactive Web search divides the search process into several rounds, and for each round the search engine interacts with the user for more knowledge of the user's information requirement. Previous research mainly uses the text information on Web pages, while little attention is paid to other modalities. This article shows that Web search performance can be significantly improved if imagery is considered in interactive Web search. Compared with text, imagery has its own advantage: the time for “reading” an image is as little as that for reading one or two words, while the information brought by an image is as much as that conveyed by a whole passage of text. In order to exploit the advantages of imagery, a novel interactive Web search framework is proposed, where image snippets are first extracted from Web pages and then provided, along with the text snippets, to the user for result presentation and relevance feedback, as well as being presented alone to the user for image suggestion. User studies show that it is more convenient for the user to identify the Web pages he or she expects and to reformulate the initial query. Further experiments demonstrate the promise of introducing multimodal techniques into the proposed interactive Web search framework.

References

  1. Anick, P. 2003. Using terminological feedback for Web search refinement: A log-based study. In Proceedings of the 26th ACM International Conference on Research and Development in Information Retrieval, Toronto, Canada, 88--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Anick, P. and Tipirneni, S. 1999. The paraphrase search assistant: Terminological feedback for iteractive information seeking. In Proceedings of the 22nd ACM International Conference on Research and Development in Information Retrieval, Berleley, CA, 153--159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Baeza-Yates, R. and Ribeiro-Neto, B. 1999. Modern Information Retrieval. Addison-Wesley, Wokingham, UK. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Barnard, K. and Johnson, M. 2005. Word sense disambiguation with pictures. Aritif. Intell. 167, 12, 13--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cai, D., He, X.-F., Li, Z.-W., Ma, W.-Y., and Wen, J.-R. 2004. Hierarchical clustering of www image search results using visual, textual and link analysis. In Proceedings of the 12th ACM International Conference on Multimedia, New York, 952--959. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Cai, D., Yu, S.-P., Wen, J.-R., and Ma, W.-Y. 2003. Vips: A vision-based page segmentation algorithm. Tech. Rep. No. MSR-TR-2003-79, Microsoft.Google ScholarGoogle Scholar
  7. Chakrabarti, S. 2003. Mining the Web: Discovering Knowledge from Hypertext Data. Morgan Kaufmann, San Francisco, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Chapman, A. 1993. Making Sense: Teaching Critical Reading Across the Curriculum. The College Board, New York.Google ScholarGoogle Scholar
  9. Coltheart, V. 1999. Fleeting Memories: Cognition of Brief Visual Stimuli. MIT Press, Cambridge, MA.Google ScholarGoogle Scholar
  10. Dennis, S., Bruza, P., and McArthur, R. 2002. Web searching: A process-oriented experimental study of three interactive search paradigms. J. Amer. Soc. Inf. Sci. Technol. 53, 2, 120--133. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Fan, X., Xie, X., Li, Z., Li, M., and Ma, W.-Y. 2005. Photo-to-Search: Using mutlimodal queries to search the Web from mobile devices. In Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval, Singapore, 143--150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Huang, C.-K., Chien, L.-F., and Oyang, Y.-J. 2003. Relevant term suggestion in interactive Web search based on contextual information in query session logs. J. Amer. Soc. Inf. Sci. Technol. 54, 7, 638--649. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jaimes, A., Christel, M., Gilles, S., Sarukkai, R., and Ma, W.-Y. 2005. Multimedia information retrieval: What is it, and why isn't anyone using it? In Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval, Singapore, 3--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jing, Y. and Croft, W. 1994. An association thesaurus for information retrieval. In Proceedings of the International Conference on Intelligent Multimedia Information Retrieval Systems, New York, 146--160.Google ScholarGoogle Scholar
  15. Joachims, T. 2002. Optimizing search engines using clickthrough data. In Proceedings of the 8th ACM International Conference on Knowledge Discovery and Data Mining, Alberta, Canada, 133--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jones, S. and Staveley, M. 1999. Phrasier: A system for interactive document retrieval using keyphrases. In Proceedings of the 22nd ACM International Conference on Research and Development in Information Retrieval, Berleley, CA, 160--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kelly, D. and Belkin, N. 2004. Display time as implicit feedback: Understanding task effects. In Proceedings of the 27th ACM International Conference on Research and Development in Information Retrieval, Sheffield, UK, 377--384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kelly, D. and Teevan, J. 2003. Implicit feedback for inferring user preferrence. SIGIR Forum 37, 2, 18--28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Koenemann, J. and Belkin, N. 1996. A case for interaction: A study of interactive information retrieval behavior and effectiveness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, Canada, 205--212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Kraft, R. and Zien, J. 2004. Mining anchor text for query refinement. In Proceedings of the 13th International Conference on the World Wide Web, New York, 666--674. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Qiu, Y. and Frei, H.-P. 1993. Concept based query expansion. In Proceedings of the 16th ACM International Conference on Research and Development in Information Retrieval, Pittsburgh, PA, 160--169. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ruthven, I. 2003. Re-Examining the potential effectiveness of interactive query expansion. In Proceedings of the 26th ACM International Conference on Research and Development in Information Retrieval, Toronto, Canada, 213--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Shen, X., Tan, B., and Zhai, C. 2005. Context-Sensitive information retrieval using implicit feedback. In Proceedings of the 28th ACM International Conference on Research and Development in Information Retrieval, Salvador, Brazil, 43--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Silverstein, C., Henzinger, M., Marais, H., and Moricz, M. 1998. Analysis of a very large AltaVista query log. Tech. Rep. No.1998-014, Digital Systems Research Center.Google ScholarGoogle Scholar
  25. Song, R.-H., Liu, H.-F., Wen, J.-R., and Ma, W.-Y. 2004. Learning block importance models for Web pages. In Proceedings of the 13th International Conference on World Wide Web, New York, 203--211. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. White, R. W., Ruthven, I., and Jose, J. M. 2005. A study of factors affecting the utility of implicit relevance feedback. In Proceedings of the 28th ACM International Conference on Research and Development in Information Retrieval, Salvador, Brazil, 35--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Woodruff, A., Faulring, A., Rosenholtz, R., Morrison, J., and Pirolli, P. 2001. Using thumbnails to search the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Seatle, WA, 198--205. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Xu, J. and Croft, W. 2000. Improving the effectiveness of information retrieval with local context analysis. ACM Trans. Inf. Syst. 18, 1, 79--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Xue, X.-B., Zhou, Z.-H., and Zhang, Z. 2006. Improve Web search using image snippets. In Proceedings of the 21st National Conference on Artificial Intelligence, Boston, MA, 1431--1436. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Yang, J., Li, Q., and Zhuang, Y. 2002. Octopus: Aggressive search of multi-modality data using multifaceted knowledge base. In Proceedings of the 11th International World Wide Web Conference, Honolulu, Hawaii, 54--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Zhang, R., Zhang, Z., Li, M., Ma, W.-Y., and Zhang, H.-J. 2005. A probabilistic semantic model for image annotation and multi-modal image retrieval. In Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China, 846--851. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Zhou, Z.-H. and Dai, H.-B. 2007. Exploiting image contents in Web search. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hydrabad, India, 2928--2933. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Improving Web search using image snippets

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!