Abstract
Many recent recommendation systems leverage the large quantity of reviews placed by users on items. However, it is both challenging and important to accurately measure the usefulness of such reviews for effective recommendation. In particular, users have been shown to exhibit distinct preferences over different types of reviews (e.g., preferring longer versus shorter or recent versus old reviews), indicating that users might differ in their viewpoints on what makes the reviews useful. Yet, there have been limited studies that account for the personalised usefulness of reviews when estimating the users’ preferences. In this article, we propose a novel neural model, called BanditProp, which addresses this gap in the literature. It first models reviews according to both their content and associated properties (e.g., length, sentiment and recency). Thereafter, it constructs a multi-task learning (MTL) framework to model the reviews’ content encoded with various properties.In such an MTL framework, each task corresponds to producing recommendations focusing on an individual property. Next, we address the selection of the features from reviews with different review properties as a bandit problem using multinomial rewards. We propose a neural contextual bandit algorithm (i.e., ConvBandit) and examine its effectiveness in comparison to eight existing bandit algorithms in addressing the bandit problem. Our extensive experiments on two well-known Amazon and Yelp datasets show that BanditProp can significantly outperform one classic and six existing state-of-the-art recommendation baselines. Moreover, BanditProp using ConvBandit consistently outperforms the use of other bandit algorithms over the two used datasets. In particular, we experimentally demonstrate the effectiveness of our proposed customised multinomial rewards in comparison to binary rewards, when addressing our bandit problem.
- [1] . 2019. Learning to generalize from sparse and underspecified rewards. In Proceedings of the ICLR.Google Scholar
- [2] . 2021. UCB-based algorithms for multinomial logistic regression bandits. Retrieved from https://arxiv.org/abs/2103.11489.Google Scholar
- [3] . 2019. Towards an exhaustive framework for online social networks user behaviour modelling. In Proceedings of the UMAP.Google Scholar
Digital Library
- [4] . 2002. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 2 (2002).Google Scholar
Digital Library
- [5] . 2017. Aspect based recommendations: Recommending items with the most valuable aspects based on user reviews. In Proceedings of the SIGKDD.Google Scholar
Digital Library
- [6] . 2017. Context attentive bandits: Contextual bandit with restricted context. In Proceedings of the IJCAI.Google Scholar
Cross Ref
- [7] . 2019. Multi-armed recommender system bandit ensembles. In Proceedings of the RecSys.Google Scholar
Digital Library
- [8] . 1993. Multitask learning: A knowledge-based source of inductive bias. In Proceedings of the ICML.Google Scholar
Cross Ref
- [9] . 2018. Neural attentional rating regression with review-level explanations. In Proceedings of the WWW.Google Scholar
Digital Library
- [10] . 2019. Co-attentive multi-task learning for explainable recommendation.. In Proceedings of the IJCAI.Google Scholar
Cross Ref
- [11] . 2018. A^3NCF: An adaptive aspect attention model for rating prediction. In Proceedings of the IJCAI.Google Scholar
Cross Ref
- [12] . 2005. The max K-armed bandit: A new model of exploration applied to search heuristic selection. In Proceedings of the AAAI.Google Scholar
- [13] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the NAACL-HLT.Google Scholar
- [14] . 2017. An adaptive implementation of \( \epsilon \)-greedy in reinforcement learning. In Proceedings of the ANT.Google Scholar
- [15] . 2014. E-WOM and accommodation: An analysis of the factors that influence travelers’ adoption of information from online reviews. J. Travel Res. 53, 1 (2014).Google Scholar
Cross Ref
- [16] . 2019. Attentive aspect modeling for review-aware recommendation. ACM Trans. Info. Syst. 37, 3 (2019).Google Scholar
Digital Library
- [17] . 2018. Why so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit task. In Proceedings of the NeurIPS.Google Scholar
- [18] . 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the WWW.Google Scholar
Digital Library
- [19] . 2015. Trirank: Review-aware explainable recommendation by modeling aspects. In Proceedings of the CIKM.Google Scholar
Digital Library
- [20] . 2018. Improving entity recommendation with search log and multi-task learning. In Proceedings of the IJCAI.Google Scholar
Cross Ref
- [21] . 2013. Understanding user behavior in online social networks: A survey. IEEE Commun. Mag. 51, 9 (2013).Google Scholar
- [22] . 2012. On Bayesian upper confidence bounds for bandit problems. In Proceedings of the AISTATS.Google Scholar
- [23] . 2016. Convolutional matrix factorization for document context-aware recommendation. In Proceedings of the RecSys.Google Scholar
Digital Library
- [24] . 2014. Convolutional neural networks for sentence classification. In Proceedings of the EMNLP.Google Scholar
Cross Ref
- [25] . 2015. Adam: A method for stochastic optimization. In Proceedings of the ICLR.Google Scholar
- [26] . 2000. Algorithms for multi-armed bandit problems. Mach. Learn. Res.1 (2000).Google Scholar
- [27] . 2019. Garbage in, reward out: Bootstrapping exploration in multi-armed bandits. In Proceedings of the ICML.Google Scholar
- [28] . 2019. A review-driven neural model for sequential recommendation. In Proceedings of the IJCAI.Google Scholar
Cross Ref
- [29] . 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the WWW.Google Scholar
Digital Library
- [30] . 2014. Ratings meet reviews, a combined approach to recommend. In Proceedings of the RecSys.Google Scholar
Digital Library
- [31] . 2020. Top-aware reinforcement learning based recommendation. Neurocomputing 417 (2020), 255–269.Google Scholar
Cross Ref
- [32] . 2018. Why I like it: Multi-task learning for recommendation and explanation. In Proceedings of the RecSys.Google Scholar
Digital Library
- [33] . 2021. IR from bag-of-words to BERT and beyond through practical experiments. In Proceedings of the CIKM.Google Scholar
Digital Library
- [34] . 2020. Sequential-based adversarial optimisation for personalised top-N item recommendation. In Proceedings of the SIGIR.Google Scholar
Digital Library
- [35] . 2013. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of the RecSys.Google Scholar
Digital Library
- [36] . 2019. Combining unsupervised pre-training and annotator rationales to improve low-shot text classification. In Proceedings of the EMNLP-IJCNLP.Google Scholar
Cross Ref
- [37] . 2010. Multi-task learning for recommender system. In Proceedings of the ACML.Google Scholar
- [38] . 2019. Pytorch: An imperative style, high-performance deep learning library. In Proceedings of the NeurIPS.Google Scholar
- [39] . 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the UAI.Google Scholar
- [40] . 2020. Bandit algorithms based on thompson sampling for bounded reward distributions. In Proceedings of the ALT.Google Scholar
- [41] . 2018. Deep Bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. In Proceedings of the ICLR.Google Scholar
- [42] . 2017. An overview of multi-task learning in deep neural networks. Retrieved from https://arxiv.org/abs/1706.05098.Google Scholar
- [43] . 2018. A tutorial on Thompson sampling. Found. Trends Mach. Learn. 11, 1 (2018).Google Scholar
Digital Library
- [44] . 2020. How useful are reviews for recommendation? A critical review and potential improvements. In Proceedings of the SIGIR.Google Scholar
Digital Library
- [45] . 2020. BLOB: A probabilistic model for recommendation that combines organic and bandit signals. In Proceedings of the SIGKDD.Google Scholar
Digital Library
- [46] . 2019. A simple multi-armed nearest-neighbor bandit for interactive recommendation. In Proceedings of the RecSys.Google Scholar
Digital Library
- [47] . 2017. Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In Proceedings of the RecSys.Google Scholar
Digital Library
- [48] . 2002. A re-examination of probability matching and rational choice. Behavioral Decision Making 15, 3 (2002).Google Scholar
Cross Ref
- [49] . 2022. StepGame: A new benchmark for robust multi-hop spatial reasoning in texts. In Proceedings of the AAAI.Google Scholar
Cross Ref
- [50] . 2020. Neighbor interaction aware graph convolution networks for recommendation. In Proceedings of the SIGIR.Google Scholar
Digital Library
- [51] . 2003. Informational influence in organizations: An integrated approach to knowledge adoption. Info. Syst. Res. 14, 1 (2003).Google Scholar
Digital Library
- [52] . 2018. Personalized top-N sequential recommendation via convolutional sequence embedding. In Proceedings of the WSDM.Google Scholar
Digital Library
- [53] . 2017. Attention is all you need. In Proceedings of the NeurIPS.Google Scholar
- [54] . 2021. Bert-based dense retrievers require interpolation with bm25 for effective passage retrieval. In Proceedings of the SIGIR.Google Scholar
Digital Library
- [55] . 2020. Negative confidence-aware weakly supervised binary classification for effective review helpfulness classification. In Proceedings of the CIKM.Google Scholar
Digital Library
- [56] . 2021. Leveraging review properties for effective recommendation. In Proceedings of the WWW.Google Scholar
Digital Library
- [57] . 2020. Contextual-bandit based personalized recommendation with time-varying user interests. In Proceedings of the AAAI.Google Scholar
Cross Ref
- [58] . 2018. Unbiased offline recommender evaluation for missing-not-at-random implicit feedback. In Proceedings of the RecSys.Google Scholar
Digital Library
- [59] . 2016. A dynamic recurrent model for next basket recommendation. In Proceedings of the SIGIR.Google Scholar
Digital Library
- [60] . 2019. Deep neural linear bandits: Overcoming catastrophic forgetting through likelihood matching. Retrieved from https://arxiv.org/abs/1901.08612.Google Scholar
- [61] . 2017. Joint representation learning for top-N recommendation with heterogeneous information sources. In Proceedings of the CIKM.Google Scholar
Digital Library
- [62] . 2010. Understanding bag-of-words model: A statistical framework. Int. J. Mach. Learn. Cybernet. 1, 1 (2010), 43–52.Google Scholar
Cross Ref
- [63] . 2018. Deep reinforcement learning for list-wise recommendations. Retrieved from https://arxiv.org/abs/1801.00209.Google Scholar
- [64] . 2017. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the WSDM.Google Scholar
Digital Library
- [65] . 2020. Neural contextual bandits with UCB-based exploration. In Proceedings of the ICML.Google Scholar
Index Terms
BanditProp: Bandit Selection of Review Properties for Effective Recommendation
Recommendations
Leveraging Review Properties for Effective Recommendation
WWW '21: Proceedings of the Web Conference 2021Many state-of-the-art recommendation systems leverage explicit item reviews posted by users by considering their usefulness in representing the users’ preferences and describing the items’ attributes. These posted reviews may have various associated ...
A General Rating Recommended Weight-Aware Model for Recommendation System
HCC 2016: Revised Selected Papers of the Second International Conference on Human Centered Computing - Volume 9567In recommendation system, the ratings represent the users' preference and play an important role in recommending items to users. However, the ratings of items may be influenced by many factors, such as time the latest ratings are more able to reflect ...
Using Implicit Feedback for Neighbors Selection: Alleviating the Sparsity Problem in Collaborative Recommendation Systems
WebMedia '17: Proceedings of the 23rd Brazillian Symposium on Multimedia and the WebThe most popular Recommender systems (RSs) employ Collaborative Filtering (CF) algorithms where users explicitly rate items. Based on these ratings, a user-item rating matrix is generated and used to select the items to be recommended for a target user. ...






Comments