Abstract
Recommendation system plays an important role in the rapid development of micro-video sharing platform. Micro-video has rich modal features, such as visual, audio, and text. It is of great significance to carry out personalized recommendation by integrating multi-modal features. However, most of the current multi-modal recommendation systems can only enrich the feature representation on the item side, while it leads to poor learning of user preferences. To solve this problem, we propose a novel module named Learning the User’s Deeper Preferences (LUDP), which constructs the item-item modal similarity graph and user preference graph in each modality to explore the learning of item and user representation. Specifically, we construct item-item similar modalities graph using multi-modal features, the item ID embedding is propagated and aggregated on the graph to learn the latent structural information of items; The user preference graph is constructed through the historical interaction between the user and item, on which the multi-modal features are aggregated as the user’s preference for the modal. Finally, combining the two parts as auxiliary information enhances the user and item representation learned from the collaborative signals to learn deeper user preferences. Through a large number of experiments on two public datasets (TikTok, Movielens), our model is proved to be superior to the most advanced multi-modal recommendation methods.
- [1] . 2018. Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 2 (2018), 423–443.Google Scholar
Digital Library
- [2] . 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).Google Scholar
- [3] . 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In International ACM SIGIR Conference. 335–344.Google Scholar
Digital Library
- [4] . 2016. Context-aware image tweet modelling and recommendation. In ACM on Multimedia Conference. 1018–1027.Google Scholar
Digital Library
- [5] . 2012. SVDFeature: A toolkit for feature-based collaborative filtering. Journal of Machine Learning Research 13, 1 (2012), 3619–3622.Google Scholar
Digital Library
- [6] . 2019. MMALFM: Explainable recommendation by leveraging reviews and images. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 1–28.Google Scholar
Digital Library
- [7] . 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).Google Scholar
- [8] . 2019. Neural networks for personalized item rankings. Neurocomputing 342 (2019), 60–65.Google Scholar
Digital Library
- [9] . [n. d.]. Modeling heterogeneous influences for point-of-interest recommendation in location-based social networks. Nanyang Technological University, Singapore ([n. d.]).Google Scholar
- [10] . 2021. Dual graph enhanced embedding neural network for CTRPrediction. arXiv preprint arXiv:2106.00314 (2021).Google Scholar
- [11] . 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Cross Ref
- [12] . 2021. Click-through rate prediction with multi-modal hypergraphs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 690–699.Google Scholar
Digital Library
- [13] . 2021. MPIA: Multiple preferences with item attributes for graph convolutional collaborative filtering. In International Conference on Web Engineering. Springer, 225–239.Google Scholar
Digital Library
- [14] . 2015. VBPR: Visual Bayesian personalized ranking from implicit feedback. (2015).Google Scholar
- [15] . 2016. Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. International World Wide Web Conferences Steering Committee.Google Scholar
- [16] . 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 639–648.Google Scholar
Digital Library
- [17] . 2016. CNN architectures for large-scale audio classification. IEEE (2016).Google Scholar
- [18] . 2008. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining. IEEE, 263–272.Google Scholar
Digital Library
- [19] . 2020. Meta-path augmented sequential recommendation with contextual co-attention network. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 16, 2 (2020), 1–24.Google Scholar
Digital Library
- [20] . 2017. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 99 (2017).Google Scholar
- [21] . 2014. Adam: A method for stochastic optimization. Computer Science (2014).Google Scholar
- [22] . 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- [23] . 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.Google Scholar
Digital Library
- [24] . 2021. Task-adaptive neural process for user cold-start recommendation. (2021).Google Scholar
- [25] . 2021. Interest-aware message-passing GCN for recommendation. In Proceedings of the Web Conference 2021. 1296–1305.Google Scholar
Digital Library
- [26] . 2019. Disentangled graph convolutional networks. In International Conference on Machine Learning. PMLR, 4212–4221.Google Scholar
- [27] . 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).Google Scholar
- [28] . 2019. Augmenting LOD-based recommender systems using graph centrality measures. Erasmus University Rotterdam, Rotterdam, The Netherlands (2019).Google Scholar
- [29] . 2009. A survey of collaborative filtering techniques. Advances in Artificial Intelligence 2009 (2009).Google Scholar
Digital Library
- [30] . 2020. Multi-modal knowledge graphs for recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1405–1414.Google Scholar
Digital Library
- [31] . 2020. MGAT: Multimodal graph attention network for recommendation. Information Processing & Management 57, 5 (2020), 102277.Google Scholar
Cross Ref
- [32] . 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).Google Scholar
- [33] . 2021. DualGNN: Dual graph neural network for multimedia recommendation. IEEE Transactions on Multimedia (2021).Google Scholar
- [34] . 2019. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 165–174.Google Scholar
Digital Library
- [35] . 2020. AM-GCN: Adaptive multi-channel graph convolutional networks. In KDD’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.Google Scholar
Digital Library
- [36] . 2021. Hierarchical user intent graph network for multimedia recommendation. IEEE Transactions on Multimedia (2021).Google Scholar
- [37] . 2021. Contrastive learning for cold-start recommendation. In Proceedings of the 29th ACM International Conference on Multimedia. 5382–5390.Google Scholar
Digital Library
- [38] . 2020. Graph-refined convolutional network for multimedia recommendation with implicit feedback. In Proceedings of the 28th ACM International Conference on Multimedia. 3541–3549.Google Scholar
Digital Library
- [39] . 2019. MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video. In Proceedings of the 27th ACM International Conference on Multimedia. 1437–1445.Google Scholar
Digital Library
- [40] . 2020. Recommendation by users’ multimodal preferences for smart city applications. IEEE Transactions on Industrial Informatics 17, 6 (2020), 4197–4205.Google Scholar
Cross Ref
- [41] . 2018. Graph convolutional neural networks for web-scale recommender systems. ACM (2018).Google Scholar
- [42] . 2021. Mining latent structures for multimedia recommendation. arXiv preprint arXiv:2104.09036 (2021).Google Scholar
- [43] . 2020. GuessUNeed: Recommending courses via neural attention network and course prerequisite relation embeddings. ACM Transactions on Multimedia Computing Communications and Applications 16, 4 (2020), 1–17.Google Scholar
Digital Library
Index Terms
Learning the User’s Deeper Preferences for Multi-modal Recommendation Systems
Recommendations
User preference through learning user profile for ubiquitous recommendation systems
KES'06: Proceedings of the 10th international conference on Knowledge-Based Intelligent Information and Engineering Systems - Volume Part IAs ubiquitous commerce is coming, the ubiquitous recommendation systems utilize collaborative filtering to help users with fast searches for the best suitable items by analyzing the similar preference. However, collaborative filtering may not provide ...
Preference-based user rating correction process for interactive recommendation systems
In most of the recommendation systems, user rating is an important user activity that reflects their opinions. Once the users return their ratings about items the systems have suggested, the user ratings can be used to adjust the recommendation ...
Multi-facet user preference learning for fine-grained item recommendation
AbstractExisting recommendation methods mainly learn user preference from historical user-item interaction data, while ignoring the extent of interactions, i.e., diverse user experience and user intention. To be more specific, for new users ...






Comments