skip to main content
research-article

Learning to Recommend Related Entities With Serendipity for Web Search Users

Published:23 April 2018Publication History
Skip Abstract Section

Abstract

Entity recommendation, providing entity suggestions to assist users in discovering interesting information, has become an indispensable feature of today’s Web search engine. However, the majority of existing entity recommendation methods are not designed to boost the performance in terms of serendipity, which also plays an important role in the appreciation of users for a recommendation system. To keep users engaged, it is important to take into account serendipity when building an entity recommendation system. In this article, we propose a learning to recommend framework that consists of two components: related entity finding and candidate entity ranking. To boost serendipity performance, three different sets of features that correlate with the three aspects of serendipity are employed in the proposed framework. Extensive experiments are conducted on large-scale, real-world datasets collected from a widely used commercial Web search engine. The experiments show that our method significantly outperforms several strong baseline methods. An analysis on the impact of features reveals that the set of interestingness features is the most powerful feature set, and the set of unexpectedness features can significantly contribute to recommendation effectiveness. In addition, online controlled experiments conducted on a commercial Web search engine demonstrate that our method can significantly improve user engagement against multiple baseline methods. This further confirms the effectiveness of the proposed framework.

References

  1. Panagiotis Adamopoulos and Alexander Tuzhilin. 2014. On unexpectedness in recommender systems: Or how to better expect the unexpected. ACM Transactions on Intelligent Systems and Technology 5, 4, 54:1--54:32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Nitish Aggarwal, Peter Mika, Roi Blanco, and Paul Buitelaar. 2015. Insights into entity recommendation in Web search. In Proceedings of the International Semantic Web Conference (ISWC’15).Google ScholarGoogle Scholar
  3. P. André, M. C. Schraefel, J. Teevan, and S. T. Dumais. 2009. Discovery is never by chance: Designing for (un)serendipity. In Proceedings of the 7th ACM Conference on Creativity and Cognition (C&C’’’09). 305--314. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bin Bi, Hao Ma, Bo-June (Paul) Hsu, Wei Chu, Kuansan Wang, and Junghoo Cho. 2015. Learning to recommend related entities to search users. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining (WSDM’15). 139--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Roi Blanco, Berkant Barla Cambazoglu, Peter Mika, and Nicolas Torzec. 2013. Entity recommendations in Web search. In Proceedings of the 12th International Semantic Web Conference (ISWC’13). 33--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research 3, 4--5, 993--1022. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Ilaria Bordino, Yelena Mejova, and Mounia Lalmas. 2013. Penguins in sweaters, or serendipitous entity search on user-generated content. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management (CIKM’13). 109--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Marc Bron, Krisztian Balog, and Maarten de Rijke. 2009. Related entity finding based on co-occurrence. In Proceedings of the 2009 Text REtrieval Conference (TREC’09).Google ScholarGoogle Scholar
  9. Mark Claypool, Phong Le, Makoto Wased, and David Brown. 2001. Implicit interest indicators. In Proceedings of the 6th International Conference on Intelligent User Interfaces. 33--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, and Taylor Van Vleet. 2010. The YouTube video recommendation system. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). 293--296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Zhicheng Dou, Ruihua Song, and Ji-Rong Wen. 2007. A large-scale evaluation and analysis of personalized search strategies. In Proceedings of the 16th International Conference on World Wide Web (WWW’07). 581--590. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Zhicheng Dou, Ruihua Song, Xiaojie Yuan, and Ji-Rong Wen. 2008. Are click-through data adequate for learning Web search rankings? In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM’08). 73--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Ignacio Fernández-Tobías and Roi Blanco. 2016. Memory-based recommendations of entities for Web search users. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM’16). 35--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jerome H. Friedman. 2000. Greedy function approximation: A gradient boosting machine. Annals of Statistics 29, 1189--1232.Google ScholarGoogle ScholarCross RefCross Ref
  15. Jerome H. Friedman. 2002. Stochastic gradient boosting. Computational Statistics and Data Analysis 38, 4, 367--378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Michael Gamon, Arjun Mukherjee, and Patrick Pantel. 2014. Predicting interesting things in text. In Proceedings of the 25th International Conference on Computational Linguistics (COLING’14). 1477--1488.Google ScholarGoogle Scholar
  17. Jianfeng Gao, Xiaodong He, and Jian-Yun Nie. 2010. Clickthrough-based translation models for Web search: From word models to phrase models. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM’10). 1139--1148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jianfeng Gao, Patrick Pantel, Michael Gamon, Xiaodong He, and Li Deng. 2014. Modeling interestingness with deep neural networks. In Proceedings of the 2014 Conference on Empirical Methods on Natural Language Processing (EMNLP’14). 2--13.Google ScholarGoogle ScholarCross RefCross Ref
  19. Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. 2010. Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). 257--260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Thore Graepel, Joaquin Q. Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft’s Bing search engine. In Proceedings of the 27th International Conference on Machine Learning (ICML’10). 13--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Xianpei Han, Le Sun, and Jun Zhao. 2011. Collective entity linking in Web text: A graph-based method. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’11). 765--774. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Yunlong He, Jiliang Tang, Hua Ouyang, Changsung Kang, Dawei Yin, and Yi Chang. 2016. Learning to rewrite queries. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM’16). 1443--1452. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22, 1, 5--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jizhou Huang, Wei Zhang, Shiqi Zhao, Shiqiang Ding, and Haifeng Wang. 2017. Learning to explain entity relationships by pairwise ranking with convolutional neural networks. In Proceedings of the 2017 International Joint Conference on Artificial Intelligence (IJCAI’17). 4018--4025. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jizhou Huang, Shiqi Zhao, Shiqiang Ding, Haiyang Wu, Mingming Sun, and Haifeng Wang. 2016. Generating recommendation evidence using translation model. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence (IJCAI’16). 2810--2816. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Leo Iaquinta, Marco De Gemmis, Pasquale Lops, Giovanni Semeraro, Michele Filannino, and Piero Molino. 2008. Introducing serendipity in a content-based recommender system. In Proceedings of the 2008 8th International Conference on Hybrid Intelligent Systems (HIS’08). 168--173. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 4, 422--446. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ron Kohavi, Alex Deng, Brian Frasca, Roger Longbotham, Toby Walker, and Ya Xu. 2012. Trustworthy online controlled experiments: Five puzzling outcomes explained. In Proceedings of the 18th International Conference on Knowledge Discovery and Data Mining (SIGKDD’12). 786--794. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Tie-Yan Liu. 2009. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval 3, 3, 225--331. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Hao Ma, Haixuan Yang, Irwin King, and Michael R. Lyu. 2008. Learning latent semantic relations from clickthrough data for query suggestion. In Proceedings of the 17th ACM International Conference on Information and Knowledge Management (CIKM’08). 709--718. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Florian Mueller and Andrea Lockerd. 2001. Cheese: Tracking mouse movement activity on Websites, a tool for user modeling. In CHI’01 Extended Abstracts on Human Factors in Computing Systems. 279--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Umut Ozertem, Olivier Chapelle, Pinar Donmez, and Emre Velipasaoglu. 2012. Learning to suggest: A machine learning framework for ranking query suggestions. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’12). 25--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Ashok Kumar Ponnuswami, Kumaresh Pattabiraman, Qiang Wu, Ran Gilad-Bachrach, and Tapas Kanungo. 2011. On composition of a federated Web search result page: Using online users to provide pairwise preference for heterogeneous verticals. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining (WSDM’11). 715--724. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. 2011. Recommender Systems Handbook. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Upendra Shardanand and Pattie Maes. 1995. Social information filtering: Algorithms for automating “word of mouth.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 210--217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Beerud Sheth and Pattie Maes. 1993. Evolving agents for personalized information filtering. In Proceedings of the 9th Conference on Artificial Intelligence for Applications. 345--352.Google ScholarGoogle ScholarCross RefCross Ref
  37. Linqi Song, Cem Tekin, and Mihaela Van Der Schaar. 2014. Clustering based online learning in recommender systems: A bandit approach. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’14). 4528--4532.Google ScholarGoogle ScholarCross RefCross Ref
  38. Xuerui Wang, Wei Li, Ying Cui, Ruofei Zhang, and Jianchang Mao. 2011. Click-through rate estimation for rare events in online advertising. In Online Multimedia Advertising: Techniques and Technologies. IGI Global, 1--12.Google ScholarGoogle Scholar
  39. Zi Yang and Eric Nyberg. 2015. Leveraging procedural knowledge for task-oriented search. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). 513--522. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Xiao Yu, Hao Ma, Bo-June Paul Hsu, and Jiawei Han. 2014. On building entity recommender systems using user click log and freebase knowledge. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM’14). 263--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yi Zhang, Jamie Callan, and Thomas Minka. 2002. Novelty and redundancy detection in adaptive filtering. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’02). 81--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Yuan Cao Zhang, Diarmuid Séaghdha, Daniele Quercia, and Tamas Jambor. 2012. Auralist: Introducing serendipity into music recommendation. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining (WSDM’12). 13--22. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Learning to Recommend Related Entities With Serendipity for Web Search Users

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!