Abstract
Web search engines can perform poorly for long queries (i.e., those containing four or more terms), in part because of their high level of query specificity. The automatic assignment of labels to long queries can capture aspects of a user’s search intent that may not be apparent from the terms in the query. This affords search result matching or reranking based on queries and labels rather than the query text alone. Query labels can be derived from interaction logs generated from many users’ search result clicks or from query trails comprising the chain of URLs visited following query submission. However, since long queries are typically rare, they are difficult to label in this way because little or no historic log data exists for them. A subset of these queries may be amenable to labeling by detecting similarities between parts of a long and rare query and the queries which appear in logs. In this article, we present the comparison of four similarity algorithms for the automatic assignment of Open Directory Project category labels to long and rare queries, based solely on matching against similar satisfied query trails extracted from log data. Our findings show that although the similarity-matching algorithms we investigated have tradeoffs in terms of coverage and accuracy, one algorithm that bases similarity on a popular search result ranking function (effectively regarding potentially-similar queries as “documents”) outperforms the others. We find that it is possible to correctly predict the top label better than one in five times, even when no past query trail exactly matches the long and rare query. We show that these labels can be used to reorder top-ranked search results leading to a significant improvement in retrieval performance over baselines that do not utilize query labeling, but instead rank results using content-matching or click-through logs. The outcomes of our research have implications for search providers attempting to provide users with highly-relevant search results for long queries.
- }}Agichtein, E., Brill, E., and Dumais, S. 2006. Improving Web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, 19--26. Google Scholar
Digital Library
- }}Allan, J., Callan, J., Croft, W. B., Ballesteros, L., Broglio, J., Xu, J., and Shu, H. 1997. Inquery at TREC-5. In Proceedings of the 5th Text Retrieval Conference (TREC). NIST, 119--132.Google Scholar
- }}Allan, J. and Raghavan, H. 2002. Using part-of-speech patterns to reduce query ambiguity. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 307--314. Google Scholar
Digital Library
- }}Beitzel, S. M., Jensen, E. C., Frieder, O., Lewis, D. D., Chowdhury, A., and Kolcz, A. 2005. Improving automatic query classification via semi-supervised learning. In Proceedings of the International Conference on Data Mining. 42--49. Google Scholar
Digital Library
- }}Beitzel, S. M., Jensen, E. C., Lewis, D. D., Chowdhury, A., and Frieder, O. 2007. Automatic classification of Web queries using very large unlabeled query logs. ACM Trans. Inform. Syst. 25, 2. Google Scholar
Digital Library
- }}Bendersky, M. and Croft, W. B. 2008. Discovering key concepts in verbose queries. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, 491--498. Google Scholar
Digital Library
- }}Bennett, G., Scholer, F., and Uitdenbogerd, A. 2008. A comparative study of probabalistic and language models for information retrieval. In Proceedings of the 19th Annual Australasian Database Conference. 65--74. Google Scholar
Digital Library
- }}Bennett, P. N., Svore, K., and Dumais, S. 2010. Classification-enhanced ranking. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). Google Scholar
Digital Library
- }}Bilenko, M. and White, R. W. 2008. Mining the search trails of surfing crowds: Identifying relevant Web sites from user activity. In Proceedings of the 17th Annual Conference on the World Wide Web. 51--60. Google Scholar
Digital Library
- }}Bollegala, D., Matsuo, Y., and Ishizuka, M. 2007. Measuring semantic similarity between words using Web search engines. In Proceedings of the 16th International Conference on the World Wide Web. ACM, New York, 757--766. Google Scholar
Digital Library
- }}Broder, A. Z., Fontoura, M., Gabrilovich, E., Joshi, A., Josifovski, V., and Zhang, T. 2007. Robust classification of rare queries using Web knowledge. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 231--238. Google Scholar
Digital Library
- }}Callan, J. P., Croft, W. B., and Broglio, J. 1995. TREC and tipster experiments with inquery. Inform. Process. Manage. 31, 3, 327--343. Google Scholar
Digital Library
- }}Chirita, P. A., Nejdl, W., Paiu, R., and Kohlschütter, C. 2005. Using ODP metadata to personalize search. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’05). 178--185. Google Scholar
Digital Library
- }}Chowdhury, A. and Soboroff, I. 2002. Automatic evaluation of world wide Web search services. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 421--422. Google Scholar
Digital Library
- }}Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd Ed. Lawrence Earlbaum.Google Scholar
- }}Cucerzan, S. 2007. Large-scale named entity disambiguation based on Wikipedia data. In Proceedings of EMNLP-CoNLL. 708--716.Google Scholar
- }}Dwork, C., Kumar, R., Naor, M., and Sivakumar, D. 2001. Rank aggregation methods for the Web. In Proceedings of the 10th International Conference on World Wide Web (WWW’01). 613--622. Google Scholar
Digital Library
- }}Gravano, L., Hatzivassiloglou, V., and Lichtenstein, R. 2003. Categorizing Web queries according to geographical locality. In Proceedings of the 12th ACM CIKM Conference on Information and Knowledge Management. 325--333. Google Scholar
Digital Library
- }}Järvelin, K. and Kekäläinen, J. 2002. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inform. Syst. 20, 4, 422--446. Google Scholar
Digital Library
- }}Kardkovács, Z. T., Tikk, D., and Bánsághi, Z. 2005. The Ferrety algorithm for the KDD Cup 2005 problem. SIGKDD Explor. 7, 2, 111--116. Google Scholar
Digital Library
- }}Kumaran, G. and Allan, J. 2007. A case for shorter queries, and helping users create them. In Proceedings of the HLT-NAACL. 220--227.Google Scholar
- }}Kumaran, G. and Allan, J. 2008. Effective and efficient user interaction for long queries. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 11--18. Google Scholar
Digital Library
- }}Kumaran, G. and Carvalho, V. R. 2009. Reducing long queries using query quality predictors. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, In press. Google Scholar
Digital Library
- }}Lease, M., Allan, J., and Croft, W. B. 2009. Regression rank: Learning to meet the opportunity of descriptive queries. In Proceedings of the 31st European Conference on Information Retrieval. Springer-Verlag, 90--101. Google Scholar
Digital Library
- }}Li, X., Wang, Y.-Y., and Acero, A. 2008. Learning query intent from regularized click graphs. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 339--346. Google Scholar
Digital Library
- }}Li, Y., Zheng, Z., and Dai, H. K. 2005. KDD CUP-2005 report: Facing a great challenge. SIGKDD Explor. 7, 2, 91--99. Google Scholar
Digital Library
- }}Metzler, D., Dumais, S., and Meek, C. 2007. Similarity measures for short segments of text. In Proceedings of the 29th European Conference on Information Retrieval. 16--27. Google Scholar
Digital Library
- }}Najork, M. A., Zaragoza, H., and Taylor, M. J. 2007. HITS on the Web: How does it compare? In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 471--478. Google Scholar
Digital Library
- }}Phan, N., Bailey, P., and Wilkinson, R. 2007. Understanding the relationship of information need specificity to search query length. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 709--710. Google Scholar
Digital Library
- }}Ponte, J. M. and Croft, W. B. 1998. A language modeling approach to information retrieval. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 275--281. Google Scholar
Digital Library
- }}Qiu, F. and Cho, J. 2006. Automatic identification of user interest for personalized search. In Proceedings of the 15th International Conference on World Wide Web (WWW’06). 727--736. Google Scholar
Digital Library
- }}Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M., and Gatford, M. 1994. Okapi at TREC-3. In Proceedings of the 3rd Text REtrieval Conference (TREC’94).Google Scholar
- }}Robertson, S., Zaragoza, H., and Taylor, M. 2004. Simple BM25 extension to multiple weighted fields. In Proceedings of the 13th ACM CIKM Conference on Information and Knowledge Management. ACM, New York, 42--49. Google Scholar
Digital Library
- }}Shen, D., Pan, R., Sun, J.-T., Pan, J. J., Wu, K., Yin, J., and Yang, Q. 2005. Q2[email protected]: Our winning solution to query classification in KDDCUP 2005. SIGKDD Explor. 7, 2, 100--110. Google Scholar
Digital Library
- }}Shen, X., Dumais, S., and Horvitz, E. 2005. Analysis of topic dynamics in Web search. In Proceedings of the 14th International Conference on the World Wide Web. 1102--1103. Google Scholar
Digital Library
- }}Strohman, T., Metzler, D., Turtle, H., and Croft, W. B. 2005. Indri: A language-model based search engine for complex queries (extended version). IR 407, U. Massachusetts.Google Scholar
- }}Vogel, D. S., Bickel, S., Haider, P., Schimpfky, R., Siemen, P., Bridges, S., and Scheffer, T. 2005. Classifying search engine queries using the Web as background knowledge. SIGKDD Explor. 7, 2, 117--122. Google Scholar
Digital Library
- }}Voorhees, E. M. 1994. Query expansion using lexical-semantic relations. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 61--69. Google Scholar
Digital Library
- }}White, R. W., Bailey, P., and Chen, L. 2009. Predicting user interests from contextual information. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. In press. Google Scholar
Digital Library
- }}White, R. W. and Drucker, S. 2007. Investigating behavioral variability in Web search. In Proceedings of the 16th International Conference on the World Wide Web. ACM, New York, 21--30. Google Scholar
Digital Library
- }}Zhai, C. and Lafferty, J. 2001. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 334--342. Google Scholar
Digital Library
Index Terms
Mining Historic Query Trails to Label Long and Rare Search Engine Queries
Recommendations
A Query Substitution-Search Result Refinement Approach for Long Query Web Searches
WI-IAT '09: Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 01Long queries are widely used in current Web applications, such as literature searches, news searches, etc. However, since long queries are frequently expressed as natural language texts but not keywords, the current keywords-based search engines, like ...
Mining query subtopics from search log data
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrievalMost queries in web search are ambiguous and multifaceted. Identifying the major senses and facets of queries from search log data, referred to as query subtopic mining in this paper, is a very important issue in web search. Through search log analysis, ...
Analysis of long queries in a large scale search log
WSCD '09: Proceedings of the 2009 workshop on Web Search Click DataWe propose to use the search log to study long queries, in order to understand the types of information needs that are behind them, and to design techniques to improve search effectiveness when they are used. Long queries arise in many different ...






Comments