skip to main content
research-article
Public Access

Optimizing Whole-Page Presentation for Web Search

Published:17 July 2018Publication History
Skip Abstract Section

Abstract

Modern search engines aggregate results from different verticals: webpages, news, images, video, shopping, knowledge cards, local maps, and so on. Unlike “ten blue links,” these search results are heterogeneous in nature and not even arranged in a list on the page. This revolution directly challenges the conventional “ranked list” formulation in ad hoc search. Therefore, finding proper presentation for a gallery of heterogeneous results is critical for modern search engines.

We propose a novel framework that learns the optimal page presentation to render heterogeneous results onto search result page (SERP). Page presentation is broadly defined as the strategy to present a set of items on SERP, much more expressive than a ranked list. It can specify item positions, image sizes, text fonts, and any other styles as long as variations are within business and design constraints. The learned presentation is content aware, i.e., tailored to specific queries and returned results. Simulation experiments show that the framework automatically learns eye-catchy presentations for relevant results. Experiments on real data show that simple instantiations of the framework already outperform leading algorithm in federated search result presentation. It means the framework can learn its own result presentation strategy purely from data, without even knowing the “probability ranking principle.”

References

  1. Aman Agarwal, Soumya Basu, Tobias Schnabel, and Thorsten Joachims. 2017. Effective evaluation using logged bandit feedback from multiple loggers. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 687--696. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 19--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Jaime Arguello and Fernando Diaz. 2013. Vertical selection and aggregation. In Relevance Ranking for Vertical Search Engines, Bo Long and Yi Chang (Eds.). Elsevier.Google ScholarGoogle Scholar
  4. Jaime Arguello, Fernando Diaz, and Jamie Callan. 2011. Learning to aggregate vertical results into web search results. In Proceedings of the 20th ACM Conference on Information and Knowledge Management. 201--210. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Jaime Arguello, Fernando Diaz, Jamie Callan, and Ben Carterette. 2011. A methodology for evaluating aggregated search results. In Advances in Information Retrieval. Springer, 141--152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jaime Arguello, Fernando Diaz, Jamie Callan, and Jean-Francois Crespo. 2009. Sources of evidence for vertical selection. In Proceedings of the 32nd ACM SIGIR Conference on Research and Development in Information Retrieval. 315--322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Jaime Arguello, Fernando Diaz, and Jean-François Paiement. 2010. Vertical selection in the presence of unlabeled verticals. In Proceedings of the 33rd ACM SIGIR Conference on Research and Development in Information Retrieval. 691--698. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Peter Bailey, Nick Craswell, Ryen W. White, Liwei Chen, Ashwin Satyanarayana, and Seyed M. M. Tahaghoghi. 2010. Evaluating whole-page relevance. In Proceedings of the 33rd ACM SIGIR Conference on Research and Development in Information Retrieval. 767--768. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. G. H. Bakir, T. Hofmann, B. Schölkopf, A. J. Smola, B. Taskar, and S. V. N. Vishwanathan. 2007. Predicting Structured Data. The MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Nicholas J. Belkin. 2008. Some (what) grand challenges for information retrieval. In ACM SIGIR Forum, Vol. 42. ACM, 47--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Christopher J. C. Burges. 2010. From ranknet to lambdarank to lambdamart: An overview. Learning 11, 81 (2010), 23--581.Google ScholarGoogle Scholar
  12. Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st ACM SIGIR Conference on Research and Development in Information Retrieval. 335--336. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Danqi Chen, Weizhu Chen, Haixun Wang, Zheng Chen, and Qiang Yang. 2012. Beyond ten blue links: Enabling user click modeling in federated web search. In Proceedings of the 5th ACM Conference on Web Search and Data Mining. 463--472. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Tianqi Chen. eXtreme Gradient Boosting Library. Retrieved from https://github.com/dmlc/xgboost.Google ScholarGoogle Scholar
  15. Flavio Chierichetti, Ravi Kumar, and Prabhakar Raghavan. 2011. Optimizing two-dimensional search results presentation. In Proceedings of the 4th ACM Conference on Web Search and Data Mining. 257--266. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Aleksandr Chuklin and Maarten de Rijke. 2016. Incorporating clicks, attention and satisfaction into a search engine result page evaluation model. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 175--184. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. 2015. Click models for web search. Synth. Lect. Inf. Concept. Retriev. Serv. 7, 3 (2015), 1--115.Google ScholarGoogle Scholar
  18. Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An experimental comparison of click position-bias models. In Proceedings of the 2008 Conference on Web Search and Data Mining. 87--94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Ovidiu Dan and Brian D. Davison. 2016. Measuring and predicting search engine user satisfaction. ACM Comput. Surv. 49, 1 (2016), 18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Thomas Degris, Martha White, and Richard Sutton. 2012. Off-policy actor-critic. In Proceedings of the International Conference on Machine Learning. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Thomas Demeester, Dolf Trieschnigg, Dong Nguyen, Ke Zhou, and Djoerd Hiemstra. 2014. Overview of the TREC 2014 Federated Web Search Track. Technical Report. Ghent University.Google ScholarGoogle Scholar
  22. Nikhil R. Devanur, Zhiyi Huang, Nitish Korula, Vahab S. Mirrokni, and Qiqi Yan. 2013. Whole-page optimization and submodular welfare maximization with online bidders. In Proceedings of the 14th ACM Conference on Electronic Commerce. 305--322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Fernando Diaz, Ryen W. White, Georg Buscher, and Dan Liebling. 2013. Robust models of mouse movement on dynamic web search results pages. In Proceedings of the 22nd ACM Conference on Information and Knowledge Management. 1451--1460. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Pinar Donmez, Krysta M. Svore, and Christopher J. C. Burges. 2009. On the local optimality of LambdaRank. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 460--467. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve web search. ACM Trans. Inf. Syst. 23, 2 (2005), 147--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Jerome H. Friedman. 2002. Stochastic gradient boosting. Comput. Stat. Data Anal. 38, 4 (2002), 367--378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Hotchkiss Gord. 2007. Eye Tracking On Universal And Personalized Search. Report No 21 9 2007. Retrieved from http://searchengineland.com/eye-tracking-on-universal-and-personalized-search-12233. Technical report, Enquiro Research.Google ScholarGoogle Scholar
  28. Laura A. Granka, Thorsten Joachims, and Geri Gay. 2004. Eye-tracking analysis of user behavior in WWW search. In Proceedings of the 27th ACM SIGIR Conference on Research and Development in Information Retrieval. 478--479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ahmed Hassan, Rosie Jones, and Kristina Lisa Klinkner. 2010. Beyond DCG: User behavior as a predictor of a successful search. In Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. ACM, 221--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Marti Hearst. 2019. The evaluation of search user interfaces. In Search User Interfaces. Cambridge University Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Shihao Ji, Ke Zhou, Ciya Liao, Zhaohui Zheng, Gui-Rong Xue, Olivier Chapelle, Gordon Sun, and Hongyuan Zha. 2009. Global ranking by exploiting user clicks. In Proceedings of the 32nd ACM SIGIR Conference on Research and Development in Information Retrieval. 35--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Luo Jie, Sudarshan Lamkhede, Rochit Sapra, Evans Hsu, Helen Song, and Yi Chang. 2013. A unified search federation system based on online user feedback. In Proceedings of the 19th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 1195--1203. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 133--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th ACM SIGIR Conference on Research and Development in Information Retrieval. 154--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased learning-to-rank with biased feedback. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining. ACM, 781--789. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Diane Kelly and others. 2009. Methods for evaluating interactive information retrieval systems with users. Found. Trends Inf. Retriev. 3, 1--2 (2009), 1--224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Harold W. Kuhn. 1955. The hungarian method for the assignment problem. Nav. Res. Logist. Quart. 2, 1--2 (1955), 83--97.Google ScholarGoogle ScholarCross RefCross Ref
  38. Dmitry Lagun and Eugene Agichtein. 2014. Effects of task and domain on searcher attention. In Proceedings of the 37th ACM SIGIR Conference on Research and Development in Information Retrieval. 1087--1090. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. John Langford. Vowpal Wabbit. Retrieved from http://hunch.net/vw.Google ScholarGoogle Scholar
  40. Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th Conference on World Wide Web. 661--670. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. 2011. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining. ACM, 297--306. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Tie-Yan Liu. 2009. Learning to rank for information retrieval. Found. Trends Inf. Retriev. 3, 3 (2009), 225--331. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yiqun Liu, Ye Chen, Jinhui Tang, Jiashen Sun, Min Zhang, Shaoping Ma, and Xuan Zhu. 2015. Different users, different opinions: Predicting search satisfaction with mouse movement information. In Proceedings of the 38th ACM SIGIR Conference on Research and Development in Information Retrieval. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Zeyang Liu, Yiqun Liu, Ke Zhou, Min Zhang, and Shaoping Ma. 2015. Influence of vertical result in web search examination. In Proceedings of the 38th ACM SIGIR Conference on Research and Development in Information Retrieval. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Bo Long and Yi Chang. 2014. Relevance Ranking for Vertical Search Engines. Elsevier. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Jiyun Luo, Sicong Zhang, and Hui Yang. 2014. Win-win search: Dual-agent stochastic game in session search. In Proceedings of the 37th International ACM SIGIR Conference on Research 8 Development in Information Retrieval. ACM, 587--596. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Pavel Metrikov, Fernando Diaz, Sébastien Lahaie, and Justin Rao. 2014. Whole page optimization: How page elements interact with the position auction. In Proceedings of the 15th ACM Conference on Economics and Computation. 583--600. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Permutohedron. Retrieved from https://en.wikipedia.org/wiki/Permutohedron.Google ScholarGoogle Scholar
  49. Ashok Kumar Ponnuswami, Kumaresh Pattabiraman, Qiang Wu, Ran Gilad-Bachrach, and Tapas Kanungo. 2011. On composition of a federated web search result page: Using online users to provide pairwise preference for heterogeneous verticals. In Proceedings of the 4th ACM Conference on Web Search and Data Mining. 715--724. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Position bias. Google ads in second position get more attention. Retrieved from http://miratech.com/blog/eye-tracking-google.html.Google ScholarGoogle Scholar
  51. Tao Qin, Tie-Yan Liu, Xu-Dong Zhang, De-Sheng Wang, and Hang Li. 2009. Global ranking using continuous conditional random fields. In Advances in Neural Information Processing Systems. 1281--1288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Filip Radlinski and Thorsten Joachims. 2005. Query chains: Learning to rank from implicit feedback. In Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining. ACM, 239--248. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Filip Radlinski and Thorsten Joachims. 2006. Minimally invasive randomization for collecting unbiased preferences from clickthrough logs. In Proceedings of the National Conference on Artificial Intelligence, Vol. 21. MIT Press, Cambridge, NA, 1406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. S. E. Robertson. 1977. The probability ranking principle in IR. In J. Document. 294--304.Google ScholarGoogle Scholar
  55. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Inf. Process. Manage. 24, 5 (1988), 513--523. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Anne Schuth, Katja Hofmann, and Filip Radlinski. 2015. Predicting search satisfaction metrics with interleaved comparisons. In Proceedings of the 38th ACM SIGIR Conference on Research and Development in Information Retrieval. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Anne Schuth, Harrie Oosterhuis, Shimon Whiteson, and Maarten de Rijke. 2016. Multileave gradient descent for fast online learning to rank. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining. ACM, 457--466. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. 2014. Deterministic policy gradient algorithms. In Proceedings of the 31st International Conference on Machine Learning (ICML’14). 387--395. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, Ma, Chapters 5 and 9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Richard S. Sutton, David A. McAllester, Satinder P. Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems. 1057--1063. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Krysta M. Svore, Maksims N. Volkovs, and Christopher J. C. Burges. 2011. Learning to rank with multiple objective functions. In Proceedings of the 20th International Conference on World Wide Web. ACM, 367--376. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Adith Swaminathan and Thorsten Joachims. 2015. Counterfactual risk minimization: Learning from logged bandit feedback. In Proceedings of the International Conference on Machine Learning. 814--823. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Vertical bias. Eye Tracking Study: Google Results with Videos. Retrieved from https://www.looktracker.com/blog/eye-tracking-case-study/google-results-with-videos-eye-tracking-study.Google ScholarGoogle Scholar
  64. Yue Wang, Dawei Yin, Luo Jie, Pengyuan Wang, Makoto Yamada, Yi Chang, and Qiaozhu Mei. 2016. Beyond ranking: Optimizing whole-page presentation. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining. ACM, 103--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Emine Yilmaz, Milad Shokouhi, Nick Craswell, and Stephen Robertson. 2010. Expected browsing utility for web search evaluation. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management. ACM, 1561--1564. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Yisong Yue and Thorsten Joachims. 2009. Interactively optimizing information retrieval systems as a dueling bandits problem. In Proceedings of the 26th Annual International Conference on Machine Learning. ACM, 1201--1208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Yisong Yue, Rajan Patel, and Hein Roehrig. 2010. Beyond position bias: Examining result attractiveness as a source of presentation bias in clickthrough data. In Proceedings of the 19th Conference on World Wide Web. 1011--1018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Chengxiang Zhai and John Lafferty. 2001. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the 24th ACM SIGIR Conference on Research and Development in Information Retrieval. 334--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Yinan Zhang and Chengxiang Zhai. 2015. Information retrieval as card playing: A formal model for optimizing interactive retrieval interface. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 685--694. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Masrour Zoghi, Tomáš Tunys, Lihong Li, Damien Jose, Junyan Chen, Chun Ming Chin, and Maarten de Rijke. 2016. Click-based hot fixes for underperforming torso queries. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 195--204. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Optimizing Whole-Page Presentation for Web Search

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on the Web
            ACM Transactions on the Web  Volume 12, Issue 3
            August 2018
            207 pages
            ISSN:1559-1131
            EISSN:1559-114X
            DOI:10.1145/3240924
            Issue’s Table of Contents

            Copyright © 2018 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 17 July 2018
            • Accepted: 1 April 2018
            • Received: 1 October 2017
            Published in tweb Volume 12, Issue 3

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!