skip to main content
research-article

Keywords-enhanced Deep Reinforcement Learning Model for Travel Recommendation

Published:20 December 2022Publication History
Skip Abstract Section

Abstract

Tourism is an important industry and a popular entertainment activity involving billions of visitors per annum. One challenging problem tourists face is identifying satisfactory products from vast tourism information. Most of travel recommendation methods regard the recommendation procedure as a static process and only focus on immediate rewards. Meanwhile, they often infer user intensions from click behaviors and ignore the informative keywords of the clicked products. To this end, in this article, we present a Keywords-enhanced Deep Reinforcement Learning model (KDRL) framework. Specifically, we formalize travel recommendation as a Markov Decision Process and implement it upon the Actor–Critic framework. It integrates keyword information into the reinforcement learning–(RL) based recommendation framework by devising novel state representation and reward function and learns the travel recommendation and keywords generation simultaneously. To the best of our knowledge, this is the first time that keywords are explicitly discussed and used in RL-based travel recommendations. Extensive experiments are performed on the real-world datasets and the results clearly show the superior performance of KDRL compared with the baseline methods.

REFERENCES

  1. [1] Chen Chi, Peng Xin, Xing Zhenchang, Sun Jun, Wang Xin, Zhao Yifan, and Zhao Wenyun. 2022. Holistic combination of structural and textual code information for context based api recommendation. IEEE Trans. Softw. Eng. 48, 8 (2022), 29873009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Chen Lei, Cao Jie, Wang Youquan, Liang Weichao, and Zhu Guixiang. 2022. Multi-view graph attention network for travel recommendation. Expert Syst. Appl. 191, 11 (2022), 116234.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Chen Lei, Wu Zhiang, Cao Jie, Zhu Guixiang, and Ge Yong. 2020. Travel recommendation via fusing multi-auxiliary information into matrix factorization. ACM Trans. Intell. Syst. Technol. 11, 2 (2020), 124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Chen Xu, Du Yali, Xia Long, and Wang Jun. 2021. Reinforcement recommendation with user multi-aspect preference. In Proceedings of the International Conference on World Wide Web. 425435.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Chen Xinshi, Li Shuang, Li Hui, Jiang Shaohua, Qi Yuan, and Song Le. 2019. Generative adversarial user model for reinforcement learning based recommendation system. In Proceedings of the International Conference on Machine Learning. PMLR, 10521061.Google ScholarGoogle Scholar
  6. [6] Chen Xu, Zhang Yongfeng, and Qin Zheng. 2019. Dynamic explainable recommendation based on neural attentive models. In Proceedings of the AAAI Conference on Artificial Intelligence. 5360.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Chen Yanhui, Huang Ling, Wang Changdong, and Lai Jianhuang. 2021. Hybrid-order gated graph neural network for session-based recommendation. IEEE Trans. Industr. Inf. 18, 3 (2021), 14581467.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Chen Yudong, Wang Xin, Fan Miao, Huang Jizhou, Yang Shengwen, and Zhu Wenwu. 2021. Curriculum meta-learning for next POI recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 26922702.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Chen Yi-Cheng, Thaipisutikul Tipajin, and Shih Timothy K.. 2022. A learning-based POI recommendation with spatiotemporal context awareness. IEEE Trans. Cybernet. 52, 4 (2022), 24532466.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Fu Liyong, Zhang Dong, and Ye Qiaolin. 2020. Recurrent thrifty attention network for remote sensing scene recognition. IEEE Trans. Geosci. Remote Sens. 59, 10 (2020), 82578268.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Fu Zhe, Yu Li, and Niu Xi. 2022. TRACE: Travel reinforcement recommendation based on location-aware context extraction. ACM Trans. Knowl. Discov. Data 16, 4 (2022), 122.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Ge Yong, Xiong Hui, Tuzhilin Alexander, and Liu Qi. 2014. Cost-aware collaborative filtering for travel tour recommendations. ACM Trans. Inf. Syst. 32, 1 (2014), 131.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] He Jiangning, Liu Hongyan, and Xiong Hui. 2016. SocoTraveler: Travel-package recommendations leveraging social influence of different relationship types. Inf. Manage. 53, 8 (2016), 934950.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Hidasi Balázs and Karatzoglou Alexandros. 2018. Recurrent neural networks with top-k gains for session-based recommendations. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 843852.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Huang Ling, Wang Changdong, and Hongyang Chao. 2019. Higher-order multi-layer community detection. In Proceedings of the AAAI Conference on Artificial Intelligence. 99459946.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Huang Zai, Tao Mingyuan, and Zhang Bufeng. 2021. Deep inclusion relation-aware network for user response prediction at fliggy. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 30593067.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Kolahkaj Maral, Harounabadi Ali, Nikravanshalmani Alireza, and Chinipardaz Rahim. 2020. A hybrid context-aware approach for e-tourism package recommendation based on asymmetric similarity measurement and sequential pattern mining. Electr. Commerce Res. Appl. 42, 5 (2020), 100978.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Lim Nicholas, Hooi Bryan, Ng See-Kiong, Wang Xueou, Goh Yong Liang, Weng Renrong, and Varadarajan Jagannadan. 2020. STP-UDGAT: Spatial-temporal-preference user dimensional graph attention network for next POI recommendation. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 845854.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Liu Chang, Li Xiaoguang, Cai Guohao, Dong Zhenhua, Zhu Hong, and Shang Lifeng. 2021. Noninvasive self-attention for side information fusion in sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence. 42494256.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Liu Peng, Zhang Lemei, and Gulla Jon Atle. 2020. Dynamic attention-based explainable recommendation with textual and visual fusion. Inf. Process. Manage. 57, 6 (2020), 102099.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Liu Yuanxing, Ren Zhaochun, Zhang Wei-Nan, Che Wanxiang, Liu Ting, and Yin Dawei. 2020. Keywords generation improves e-commerce session-based recommendation. In Proceedings of the International Conference on World Wide Web. 16041614.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Luo Yingtao, Liu Qiang, and Liu Zhaocheng. 2021. Stan: Spatio-temporal attention network for next location recommendation. In Proceedings of the 30th International Conference on World Wide Web. 21772185.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Lv Fuyu, Jin Taiwei, Yu Changlong, Sun Fei, Lin Quan, Yang Keping, and Ng Wilfred. 2019. SDM: Sequential deep matching model for online large-scale recommender system. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 26352643.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Rendle Steffen, Freudenthaler Christoph, Gantner Zeno, and Schmidt-Thieme Lars. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 452461.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Sachdeva Noveen and McAuley Julian. 2020. How useful are reviews for recommendation? A critical review and potential improvements. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 18451848.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Sarwar Badrul, Karypis George, Konstan Joseph, and Riedl John. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web. 285295.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Stamenkovic Dusan, Karatzoglou Alexandros, Arapakis Ioannis, Xin Xin, and Katevas Kleomenis. 2022. Choosing the best of both worlds: Diverse and novel recommendations through multi-objective reinforcement learning. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 957965.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Sun Peijie, Wu Le, Zhang Kun, Fu Yanjie, Hong Richang, and Wang Meng. 2020. Dual learning for explainable recommendation: Towards unifying user preference prediction and review generation. In Proceedings of the 29th International Conference on World Wide Web. 837847.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Sun Ying, Zhuang Fuzhen, Zhu Hengshu, He Qing, and Xiong Hui. 2021. Cost-effective and interpretable job skill recommendation with deep reinforcement learning. In Proceedings of the 30th International Conference on World Wide Web. 38273838.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Taylor Kendall, Lim Kwan Hui, and Chan Jeffrey. 2018. Travel itinerary recommendations with must-see points-of-interest. In Companion of the The Web Conference on The Web Conference. International World Wide Web Conferences Steering Committee, 11981205.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Łukasz, and Polosukhin Illia. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 59986008.Google ScholarGoogle Scholar
  32. [32] Wang Hao, Wang Naiyan, and Yeung Dit-Yan. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 12351244.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Wu Shu, Tang Yuyuan, Zhu Yanqiao, Wang Liang, Xie Xing, and Tan Tieniu. 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence. 346353.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Xi Wu-Dong, Huang Ling, Wang Chang-Dong, Zheng Yin-Yu, and Lai Jian-Huang. 2022. Deep rating and review neural network for item recommendation. IEEE Trans. Neural Netw. Learn. Syst. 33, 11 (2022), 67266736.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Xie Ruobing, Zhang Shaoliang, Wang Rui, Xia Feng, and Lin Leyu. 2021. Hierarchical reinforcement learning for integrated recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence. 45214528.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Xu Jia, Wang Ziyi, Chen Zulong, Lv Detao, Yu Yao, and Xu Chuanfei. 2021. Itinerary-aware personalized deep matching at fliggy. In Proceedings of the International Conference on World Wide Web. 32343245.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Ye Qiaolin, Huang Peng, Zhang Zhao, Zheng Yuhui, Fu Liyong, and Yang Wankou. 2021. Multiview learning with robust double-sided twin SVM. IEEE Trans. Cybernet. 52, 11 (2021), 11963–11976.Google ScholarGoogle Scholar
  38. [38] Yu Fuqiang, Cui Lizhen, Guo Wei, Lu Xudong, Li Qingzhong, and Lu Hua. 2020. A category-aware deep model for successive POI recommendation on sparse check-in data. In Proceedings of the 29th International Conference on World Wide Web. 12641274.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Zang Hongyu, Han Dongcheng, Li Xin, Wan Zhifeng, and Wang Mingzhong. 2021. CHA: Categorical hierarchy-based attention for next POI recommendation. ACM Trans. Inf. Syst. 40, 1 (2021), 122.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Zhang Weijia, Liu Hao, Wang Fan, Xu Tong, Xin Haoran, Dou Dejing, and Xiong Hui. 2021. Intelligent electric vehicle charging recommendation based on multi-agent reinforcement learning. In Proceedings of the 2021 International Conference on World Wide Web. 18561867.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Zhang Xuanyu, Yang Qing, and Xu Dongliang. 2021. Combining explicit entity graph with implicit text information for news recommendation. In Proceedings of the International Conference on World Wide Web. 412416.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Zhao Dongyang, Zhang Liang, Zhang Bo, Zheng Lizhou, Bao Yongjun, and Yan Weipeng. 2020. Mahrl: Multi-goals abstraction based deep hierarchical reinforcement learning for recommendations. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 871880.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Zhao Xiangyu, Xia Long, Zou Lixin, Liu Hui, Yin Dawei, and Tang Jiliang. 2020. Whole-chain recommendations. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 18831891.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Zheng Guanjie, Zhang Fuzheng, Zheng Zihan, Xiang Yang, Yuan Nicholas Jing, Xie Xing, and Li Zhenhui. 2018. DRN: A deep reinforcement learning framework for news recommendation. In Proceedings of the World Wide Web Conference. 167176.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Zhu Junnan, Zhou Yu, Zhang Jiajun, Li Haoran, Zong Chengqing, and Li Changliang. 2020. Multimodal summarization with guidance of multimodal reference. In Proceedings of the AAAI Conference on Artificial Intelligence. 97499756.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Keywords-enhanced Deep Reinforcement Learning Model for Travel Recommendation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on the Web
        ACM Transactions on the Web  Volume 17, Issue 1
        February 2023
        189 pages
        ISSN:1559-1131
        EISSN:1559-114X
        DOI:10.1145/3575872
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 20 December 2022
        • Online AM: 11 November 2022
        • Accepted: 6 November 2022
        • Revised: 8 October 2022
        • Received: 6 June 2022
        Published in tweb Volume 17, Issue 1

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!