Abstract
Entity recommendation, providing entity suggestions to assist users in discovering interesting information, has become an indispensable feature of today’s Web search engine. However, the majority of existing entity recommendation methods are not designed to boost the performance in terms of serendipity, which also plays an important role in the appreciation of users for a recommendation system. To keep users engaged, it is important to take into account serendipity when building an entity recommendation system. In this article, we propose a learning to recommend framework that consists of two components: related entity finding and candidate entity ranking. To boost serendipity performance, three different sets of features that correlate with the three aspects of serendipity are employed in the proposed framework. Extensive experiments are conducted on large-scale, real-world datasets collected from a widely used commercial Web search engine. The experiments show that our method significantly outperforms several strong baseline methods. An analysis on the impact of features reveals that the set of interestingness features is the most powerful feature set, and the set of unexpectedness features can significantly contribute to recommendation effectiveness. In addition, online controlled experiments conducted on a commercial Web search engine demonstrate that our method can significantly improve user engagement against multiple baseline methods. This further confirms the effectiveness of the proposed framework.
- Panagiotis Adamopoulos and Alexander Tuzhilin. 2014. On unexpectedness in recommender systems: Or how to better expect the unexpected. ACM Transactions on Intelligent Systems and Technology 5, 4, 54:1--54:32. Google Scholar
Digital Library
- Nitish Aggarwal, Peter Mika, Roi Blanco, and Paul Buitelaar. 2015. Insights into entity recommendation in Web search. In Proceedings of the International Semantic Web Conference (ISWC’15).Google Scholar
- P. André, M. C. Schraefel, J. Teevan, and S. T. Dumais. 2009. Discovery is never by chance: Designing for (un)serendipity. In Proceedings of the 7th ACM Conference on Creativity and Cognition (C&C’’’09). 305--314. Google Scholar
Digital Library
- Bin Bi, Hao Ma, Bo-June (Paul) Hsu, Wei Chu, Kuansan Wang, and Junghoo Cho. 2015. Learning to recommend related entities to search users. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining (WSDM’15). 139--148. Google Scholar
Digital Library
- Roi Blanco, Berkant Barla Cambazoglu, Peter Mika, and Nicolas Torzec. 2013. Entity recommendations in Web search. In Proceedings of the 12th International Semantic Web Conference (ISWC’13). 33--48. Google Scholar
Digital Library
- David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research 3, 4--5, 993--1022. Google Scholar
Digital Library
- Ilaria Bordino, Yelena Mejova, and Mounia Lalmas. 2013. Penguins in sweaters, or serendipitous entity search on user-generated content. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management (CIKM’13). 109--118. Google Scholar
Digital Library
- Marc Bron, Krisztian Balog, and Maarten de Rijke. 2009. Related entity finding based on co-occurrence. In Proceedings of the 2009 Text REtrieval Conference (TREC’09).Google Scholar
- Mark Claypool, Phong Le, Makoto Wased, and David Brown. 2001. Implicit interest indicators. In Proceedings of the 6th International Conference on Intelligent User Interfaces. 33--40. Google Scholar
Digital Library
- James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, and Taylor Van Vleet. 2010. The YouTube video recommendation system. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). 293--296. Google Scholar
Digital Library
- Zhicheng Dou, Ruihua Song, and Ji-Rong Wen. 2007. A large-scale evaluation and analysis of personalized search strategies. In Proceedings of the 16th International Conference on World Wide Web (WWW’07). 581--590. Google Scholar
Digital Library
- Zhicheng Dou, Ruihua Song, Xiaojie Yuan, and Ji-Rong Wen. 2008. Are click-through data adequate for learning Web search rankings? In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM’08). 73--82. Google Scholar
Digital Library
- Ignacio Fernández-Tobías and Roi Blanco. 2016. Memory-based recommendations of entities for Web search users. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM’16). 35--44. Google Scholar
Digital Library
- Jerome H. Friedman. 2000. Greedy function approximation: A gradient boosting machine. Annals of Statistics 29, 1189--1232.Google Scholar
Cross Ref
- Jerome H. Friedman. 2002. Stochastic gradient boosting. Computational Statistics and Data Analysis 38, 4, 367--378. Google Scholar
Digital Library
- Michael Gamon, Arjun Mukherjee, and Patrick Pantel. 2014. Predicting interesting things in text. In Proceedings of the 25th International Conference on Computational Linguistics (COLING’14). 1477--1488.Google Scholar
- Jianfeng Gao, Xiaodong He, and Jian-Yun Nie. 2010. Clickthrough-based translation models for Web search: From word models to phrase models. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM’10). 1139--1148. Google Scholar
Digital Library
- Jianfeng Gao, Patrick Pantel, Michael Gamon, Xiaodong He, and Li Deng. 2014. Modeling interestingness with deep neural networks. In Proceedings of the 2014 Conference on Empirical Methods on Natural Language Processing (EMNLP’14). 2--13.Google Scholar
Cross Ref
- Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. 2010. Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys’10). 257--260. Google Scholar
Digital Library
- Thore Graepel, Joaquin Q. Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft’s Bing search engine. In Proceedings of the 27th International Conference on Machine Learning (ICML’10). 13--20. Google Scholar
Digital Library
- Xianpei Han, Le Sun, and Jun Zhao. 2011. Collective entity linking in Web text: A graph-based method. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’11). 765--774. Google Scholar
Digital Library
- Yunlong He, Jiliang Tang, Hua Ouyang, Changsung Kang, Dawei Yin, and Yi Chang. 2016. Learning to rewrite queries. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM’16). 1443--1452. Google Scholar
Digital Library
- Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22, 1, 5--53. Google Scholar
Digital Library
- Jizhou Huang, Wei Zhang, Shiqi Zhao, Shiqiang Ding, and Haifeng Wang. 2017. Learning to explain entity relationships by pairwise ranking with convolutional neural networks. In Proceedings of the 2017 International Joint Conference on Artificial Intelligence (IJCAI’17). 4018--4025. Google Scholar
Digital Library
- Jizhou Huang, Shiqi Zhao, Shiqiang Ding, Haiyang Wu, Mingming Sun, and Haifeng Wang. 2016. Generating recommendation evidence using translation model. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence (IJCAI’16). 2810--2816. Google Scholar
Digital Library
- Leo Iaquinta, Marco De Gemmis, Pasquale Lops, Giovanni Semeraro, Michele Filannino, and Piero Molino. 2008. Introducing serendipity in a content-based recommender system. In Proceedings of the 2008 8th International Conference on Hybrid Intelligent Systems (HIS’08). 168--173. Google Scholar
Digital Library
- Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 4, 422--446. Google Scholar
Digital Library
- Ron Kohavi, Alex Deng, Brian Frasca, Roger Longbotham, Toby Walker, and Ya Xu. 2012. Trustworthy online controlled experiments: Five puzzling outcomes explained. In Proceedings of the 18th International Conference on Knowledge Discovery and Data Mining (SIGKDD’12). 786--794. Google Scholar
Digital Library
- Tie-Yan Liu. 2009. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval 3, 3, 225--331. Google Scholar
Digital Library
- Hao Ma, Haixuan Yang, Irwin King, and Michael R. Lyu. 2008. Learning latent semantic relations from clickthrough data for query suggestion. In Proceedings of the 17th ACM International Conference on Information and Knowledge Management (CIKM’08). 709--718. Google Scholar
Digital Library
- Florian Mueller and Andrea Lockerd. 2001. Cheese: Tracking mouse movement activity on Websites, a tool for user modeling. In CHI’01 Extended Abstracts on Human Factors in Computing Systems. 279--280. Google Scholar
Digital Library
- Umut Ozertem, Olivier Chapelle, Pinar Donmez, and Emre Velipasaoglu. 2012. Learning to suggest: A machine learning framework for ranking query suggestions. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’12). 25--34. Google Scholar
Digital Library
- Ashok Kumar Ponnuswami, Kumaresh Pattabiraman, Qiang Wu, Ran Gilad-Bachrach, and Tapas Kanungo. 2011. On composition of a federated Web search result page: Using online users to provide pairwise preference for heterogeneous verticals. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining (WSDM’11). 715--724. Google Scholar
Digital Library
- Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. 2011. Recommender Systems Handbook. Springer. Google Scholar
Digital Library
- Upendra Shardanand and Pattie Maes. 1995. Social information filtering: Algorithms for automating “word of mouth.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 210--217. Google Scholar
Digital Library
- Beerud Sheth and Pattie Maes. 1993. Evolving agents for personalized information filtering. In Proceedings of the 9th Conference on Artificial Intelligence for Applications. 345--352.Google Scholar
Cross Ref
- Linqi Song, Cem Tekin, and Mihaela Van Der Schaar. 2014. Clustering based online learning in recommender systems: A bandit approach. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’14). 4528--4532.Google Scholar
Cross Ref
- Xuerui Wang, Wei Li, Ying Cui, Ruofei Zhang, and Jianchang Mao. 2011. Click-through rate estimation for rare events in online advertising. In Online Multimedia Advertising: Techniques and Technologies. IGI Global, 1--12.Google Scholar
- Zi Yang and Eric Nyberg. 2015. Leveraging procedural knowledge for task-oriented search. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’15). 513--522. Google Scholar
Digital Library
- Xiao Yu, Hao Ma, Bo-June Paul Hsu, and Jiawei Han. 2014. On building entity recommender systems using user click log and freebase knowledge. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM’14). 263--272. Google Scholar
Digital Library
- Yi Zhang, Jamie Callan, and Thomas Minka. 2002. Novelty and redundancy detection in adaptive filtering. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’02). 81--88. Google Scholar
Digital Library
- Yuan Cao Zhang, Diarmuid Séaghdha, Daniele Quercia, and Tamas Jambor. 2012. Auralist: Introducing serendipity into music recommendation. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining (WSDM’12). 13--22. Google Scholar
Digital Library
Index Terms
Learning to Recommend Related Entities With Serendipity for Web Search Users
Recommendations
Neural Serendipity Recommendation: Exploring the Balance between Accuracy and Novelty with Sparse Explicit Feedback
Recommender systems have been playing an important role in providing personalized information to users. However, there is always a trade-off between accuracy and novelty in recommender systems. Usually, many users are suffering from redundant or ...
Investigating serendipity in recommender systems based on real user feedback
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied ComputingOver the past several years, research in recommender systems has emphasized the importance of serendipity, but there is still no consensus on the definition of this concept and whether serendipitous items should be recommended is still not a well-...
Memory-based Recommendations of Entities for Web Search Users
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge ManagementModern search engines have evolved from mere document retrieval systems to platforms that assist the users in discovering new information. In this context, entity recommendation systems exploit query log data to proactively provide the users with ...






Comments