skip to main content
research-article

BanditProp: Bandit Selection of Review Properties for Effective Recommendation

Published:16 November 2022Publication History
Skip Abstract Section

Abstract

Many recent recommendation systems leverage the large quantity of reviews placed by users on items. However, it is both challenging and important to accurately measure the usefulness of such reviews for effective recommendation. In particular, users have been shown to exhibit distinct preferences over different types of reviews (e.g., preferring longer versus shorter or recent versus old reviews), indicating that users might differ in their viewpoints on what makes the reviews useful. Yet, there have been limited studies that account for the personalised usefulness of reviews when estimating the users’ preferences. In this article, we propose a novel neural model, called BanditProp, which addresses this gap in the literature. It first models reviews according to both their content and associated properties (e.g., length, sentiment and recency). Thereafter, it constructs a multi-task learning (MTL) framework to model the reviews’ content encoded with various properties.In such an MTL framework, each task corresponds to producing recommendations focusing on an individual property. Next, we address the selection of the features from reviews with different review properties as a bandit problem using multinomial rewards. We propose a neural contextual bandit algorithm (i.e., ConvBandit) and examine its effectiveness in comparison to eight existing bandit algorithms in addressing the bandit problem. Our extensive experiments on two well-known Amazon and Yelp datasets show that BanditProp can significantly outperform one classic and six existing state-of-the-art recommendation baselines. Moreover, BanditProp using ConvBandit consistently outperforms the use of other bandit algorithms over the two used datasets. In particular, we experimentally demonstrate the effectiveness of our proposed customised multinomial rewards in comparison to binary rewards, when addressing our bandit problem.

REFERENCES

  1. [1] Agarwal Rishabh, Liang Chen, Schuurmans Dale, and Norouzi Mohammad. 2019. Learning to generalize from sparse and underspecified rewards. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  2. [2] Amani Sanae and Thrampoulidis Christos. 2021. UCB-based algorithms for multinomial logistic regression bandits. Retrieved from https://arxiv.org/abs/2103.11489.Google ScholarGoogle Scholar
  3. [3] Antelmi Alessia. 2019. Towards an exhaustive framework for online social networks user behaviour modelling. In Proceedings of the UMAP.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Auer Peter, Cesa-Bianchi Nicolo, and Fischer Paul. 2002. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 2 (2002).Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Bauman Konstantin, Liu Bing, and Tuzhilin Alexander. 2017. Aspect based recommendations: Recommending items with the most valuable aspects based on user reviews. In Proceedings of the SIGKDD.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Bouneffouf Djallel, Rish Irina, Cecchi Guillermo A., and Féraud Raphaël. 2017. Context attentive bandits: Contextual bandit with restricted context. In Proceedings of the IJCAI.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Cañamares Rocío, Redondo Marcos, and Castells Pablo. 2019. Multi-armed recommender system bandit ensembles. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Caruana Rich. 1993. Multitask learning: A knowledge-based source of inductive bias. In Proceedings of the ICML.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Chen Chong, Zhang Min, Liu Yiqun, and Ma Shaoping. 2018. Neural attentional rating regression with review-level explanations. In Proceedings of the WWW.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Chen Zhongxia, Wang Xiting, Xie Xing, Wu Tong, Bu Guoqing, Wang Yining, and Chen Enhong. 2019. Co-attentive multi-task learning for explainable recommendation.. In Proceedings of the IJCAI.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Cheng Zhiyong, Ding Ying, He Xiangnan, Zhu Lei, Song Xuemeng, and Kankanhalli Mohan S. 2018. A^3NCF: An adaptive aspect attention model for rating prediction. In Proceedings of the IJCAI.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Cicirello Vincent A. and Smith Stephen F.. 2005. The max K-armed bandit: A new model of exploration applied to search heuristic selection. In Proceedings of the AAAI.Google ScholarGoogle Scholar
  13. [13] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the NAACL-HLT.Google ScholarGoogle Scholar
  14. [14] Mignon Alexandre dos Santos and Rocha Ricardo Luis de Azevedo da. 2017. An adaptive implementation of \( \epsilon \)-greedy in reinforcement learning. In Proceedings of the ANT.Google ScholarGoogle Scholar
  15. [15] Filieri Raffaele and McLeay Fraser. 2014. E-WOM and accommodation: An analysis of the factors that influence travelers’ adoption of information from online reviews. J. Travel Res. 53, 1 (2014).Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Guan Xinyu, Cheng Zhiyong, He Xiangnan, Zhang Yongfeng, Zhu Zhibo, Peng Qinke, and Chua Tat-Seng. 2019. Attentive aspect modeling for review-aware recommendation. ACM Trans. Info. Syst. 37, 3 (2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Guo Dalin and Yu Angela J.. 2018. Why so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit task. In Proceedings of the NeurIPS.Google ScholarGoogle Scholar
  18. [18] He Ruining and McAuley Julian. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the WWW.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] He Xiangnan, Chen Tao, Kan Min-Yen, and Chen Xiao. 2015. Trirank: Review-aware explainable recommendation by modeling aspects. In Proceedings of the CIKM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Huang Jizhou, Zhang Wei, Sun Yaming, Wang Haifeng, and Liu Ting. 2018. Improving entity recommendation with search log and multi-task learning. In Proceedings of the IJCAI.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Jin Long, Chen Yang, Wang Tianyi, Hui Pan, and Vasilakos Athanasios V.. 2013. Understanding user behavior in online social networks: A survey. IEEE Commun. Mag. 51, 9 (2013).Google ScholarGoogle Scholar
  22. [22] Kaufmann Emilie, Cappé Olivier, and Garivier Aurélien. 2012. On Bayesian upper confidence bounds for bandit problems. In Proceedings of the AISTATS.Google ScholarGoogle Scholar
  23. [23] Kim Donghyun, Park Chanyoung, Oh Jinoh, Lee Sungyoung, and Yu Hwanjo. 2016. Convolutional matrix factorization for document context-aware recommendation. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Kim Yoon. 2014. Convolutional neural networks for sentence classification. In Proceedings of the EMNLP.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Kingma Diederik P. and Ba Jimmy. 2015. Adam: A method for stochastic optimization. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  26. [26] Kuleshov Volodymyr and Precup Doina. 2000. Algorithms for multi-armed bandit problems. Mach. Learn. Res.1 (2000).Google ScholarGoogle Scholar
  27. [27] Kveton Branislav, Szepesvári Csaba, Vaswani Sharan, Wen Zheng, Lattimore Tor, and Ghavamzadeh Mohammad. 2019. Garbage in, reward out: Bootstrapping exploration in multi-armed bandits. In Proceedings of the ICML.Google ScholarGoogle Scholar
  28. [28] Li Chenliang, Niu Xichuan, Luo Xiangyang, Chen Zhenzhong, and Quan Cong. 2019. A review-driven neural model for sequential recommendation. In Proceedings of the IJCAI.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Li Lihong, Chu Wei, Langford John, and Schapire Robert E.. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the WWW.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Ling Guang, Lyu Michael R., and King Irwin. 2014. Ratings meet reviews, a combined approach to recommend. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Liu Feng, Tang Ruiming, Guo Huifeng, Li Xutao, Ye Yunming, and He Xiuqiang. 2020. Top-aware reinforcement learning based recommendation. Neurocomputing 417 (2020), 255269.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Lu Yichao, Dong Ruihai, and Smyth Barry. 2018. Why I like it: Multi-task learning for recommendation and explanation. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Macdonald Craig, Tonellotto Nicola, and MacAvaney Sean. 2021. IR from bag-of-words to BERT and beyond through practical experiments. In Proceedings of the CIKM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Manotumruksa Jarana and Yilmaz Emine. 2020. Sequential-based adversarial optimisation for personalised top-N item recommendation. In Proceedings of the SIGIR.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] McAuley Julian and Leskovec Jure. 2013. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Melamud Oren, Bornea Mihaela, and Barker Ken. 2019. Combining unsupervised pre-training and annotator rationales to improve low-shot text classification. In Proceedings of the EMNLP-IJCNLP.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Ning Xia and Karypis George. 2010. Multi-task learning for recommender system. In Proceedings of the ACML.Google ScholarGoogle Scholar
  38. [38] Paszke Adam, Gross Sam, Massa Francisco, Lerer Adam, Bradbury James, Chanan Gregory, Killeen Trevor, Lin Zeming, Gimelshein Natalia, Antiga Luca, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In Proceedings of the NeurIPS.Google ScholarGoogle Scholar
  39. [39] Rendle Steffen, Freudenthaler Christoph, Gantner Zeno, and Schmidt-Thieme Lars. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the UAI.Google ScholarGoogle Scholar
  40. [40] Riou Charles and Honda Junya. 2020. Bandit algorithms based on thompson sampling for bounded reward distributions. In Proceedings of the ALT.Google ScholarGoogle Scholar
  41. [41] Riquelme Carlos, Tucker George, and Snoek Jasper. 2018. Deep Bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  42. [42] Ruder Sebastian. 2017. An overview of multi-task learning in deep neural networks. Retrieved from https://arxiv.org/abs/1706.05098.Google ScholarGoogle Scholar
  43. [43] Russo Daniel, Roy Benjamin Van, Kazerouni Abbas, Osband Ian, and Wen Zheng. 2018. A tutorial on Thompson sampling. Found. Trends Mach. Learn. 11, 1 (2018).Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Sachdeva Noveen and McAuley Julian J.. 2020. How useful are reviews for recommendation? A critical review and potential improvements. In Proceedings of the SIGIR.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Sakhi Otmane, Bonner Stephen, Rohde David, and Vasile Flavian. 2020. BLOB: A probabilistic model for recommendation that combines organic and bandit signals. In Proceedings of the SIGKDD.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Sanz-Cruzado Javier, Castells Pablo, and López Esther. 2019. A simple multi-armed nearest-neighbor bandit for interactive recommendation. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Seo Sungyong, Huang Jing, Yang Hao, and Liu Yan. 2017. Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Shanks David R., Tunney Richard J., and McCarthy John D.. 2002. A re-examination of probability matching and rational choice. Behavioral Decision Making 15, 3 (2002).Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Shi Zhengxiang, Zhang Qiang, and Lipani Aldo. 2022. StepGame: A new benchmark for robust multi-hop spatial reasoning in texts. In Proceedings of the AAAI.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Sun Jianing, Zhang Yingxue, Guo Wei, Guo Huifeng, Tang Ruiming, He Xiuqiang, Ma Chen, and Coates Mark. 2020. Neighbor interaction aware graph convolution networks for recommendation. In Proceedings of the SIGIR.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Sussman Stephanie Watts and Siegal Wendy Schneier. 2003. Informational influence in organizations: An integrated approach to knowledge adoption. Info. Syst. Res. 14, 1 (2003).Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Tang Jiaxi and Wang Ke. 2018. Personalized top-N sequential recommendation via convolutional sequence embedding. In Proceedings of the WSDM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention is all you need. In Proceedings of the NeurIPS.Google ScholarGoogle Scholar
  54. [54] Wang Shuai, Zhuang Shengyao, and Zuccon Guido. 2021. Bert-based dense retrievers require interpolation with bm25 for effective passage retrieval. In Proceedings of the SIGIR.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Wang Xi, Ounis Iadh, and MacDonald Craig. 2020. Negative confidence-aware weakly supervised binary classification for effective review helpfulness classification. In Proceedings of the CIKM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. [56] Wang Xi, Ounis Iadh, and MacDonald Craig. 2021. Leveraging review properties for effective recommendation. In Proceedings of the WWW.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. [57] Xu Xiao, Dong Fang, Li Yanghua, He Shaojian, and Li Xin. 2020. Contextual-bandit based personalized recommendation with time-varying user interests. In Proceedings of the AAAI.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Yang Longqi, Cui Yin, Xuan Yuan, Wang Chenyang, Belongie Serge J., and Estrin Deborah. 2018. Unbiased offline recommender evaluation for missing-not-at-random implicit feedback. In Proceedings of the RecSys.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. [59] Yu Feng, Liu Qiang, Wu Shu, Wang Liang, and Tan Tieniu. 2016. A dynamic recurrent model for next basket recommendation. In Proceedings of the SIGIR.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Zahavy Tom and Mannor Shie. 2019. Deep neural linear bandits: Overcoming catastrophic forgetting through likelihood matching. Retrieved from https://arxiv.org/abs/1901.08612.Google ScholarGoogle Scholar
  61. [61] Zhang Yongfeng, Ai Qingyao, Chen Xu, and Croft W. Bruce. 2017. Joint representation learning for top-N recommendation with heterogeneous information sources. In Proceedings of the CIKM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. [62] Zhang Yin, Jin Rong, and Zhou Zhi-Hua. 2010. Understanding bag-of-words model: A statistical framework. Int. J. Mach. Learn. Cybernet. 1, 1 (2010), 4352.Google ScholarGoogle ScholarCross RefCross Ref
  63. [63] Zhao Xiangyu, Zhang Liang, Xia Long, Ding Zhuoye, Yin Dawei, and Tang Jiliang. 2018. Deep reinforcement learning for list-wise recommendations. Retrieved from https://arxiv.org/abs/1801.00209.Google ScholarGoogle Scholar
  64. [64] Zheng Lei, Noroozi Vahid, and Yu Philip S.. 2017. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the WSDM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. [65] Zhou Dongruo, Li Lihong, and Gu Quanquan. 2020. Neural contextual bandits with UCB-based exploration. In Proceedings of the ICML.Google ScholarGoogle Scholar

Index Terms

  1. BanditProp: Bandit Selection of Review Properties for Effective Recommendation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on the Web
        ACM Transactions on the Web  Volume 16, Issue 4
        November 2022
        165 pages
        ISSN:1559-1131
        EISSN:1559-114X
        DOI:10.1145/3571715
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 16 November 2022
        • Online AM: 21 July 2022
        • Accepted: 12 July 2022
        • Revised: 13 May 2022
        • Received: 2 November 2021
        Published in tweb Volume 16, Issue 4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!