skip to main content
research-article

On Scalability of Association-rule-based Recommendation: A Unified Distributed-computing Framework

Published:21 June 2020Publication History
Skip Abstract Section

Abstract

The association-rule-based approach is one of the most common technologies for building recommender systems and it has been extensively adopted for commercial use. A variety of techniques, mainly including eligible rule selection and multiple rules combination, have been developed to create effective recommendation. Unfortunately, little attention has been paid to the scalability concern of rule-based recommendation methods. However, the computational complexity of rule-based methods shall increase drastically with the growth of both online customers and rules, which are usually several millions in typical e-commerce platforms. Moreover, the dynamic change of users’ actions requires rule-based methods make recommendations in nearly real-time, which further highlights the scalability issue of rule-based recommender systems. In this article, we present a distributed framework that can scale different association-rule-based recommendation methods in a unified way. Specifically, based on the summarization of existing rule-based approaches, a generic tree-type structure is defined to store separate kinds of patterns, and an efficient algorithm is designed for mining eligible patterns along with computing recommendation scores. To handle the ever-increasing number of online customers, a distributed framework is proposed, where two load-balanced strategies for partitioning tree are put forward to fit sparse and dense data, respectively. Extensive experiments on five real-life data sets demonstrate that the efficiency of association-rule-based recommender systems can be significantly improved by the proposed framework.

References

  1. Gediminas Adomavicius and Alexander Tuzhilin. 2001. Expert-driven validation of rule-based user models in personalization applications. Data Min. Knowl. Discov. 5, 1 (2001), 33--58.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17, 6 (2005), 734--749.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Gediminas Adomavicius and Alexander Tuzhilin. 2007. Validation sequence optimization: A theoretical approach. INFORMS J. Comput. 19, 2 (2007), 185--200.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Charu C. Aggarwal. 2016. Recommender Systems. Springer.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Rakesh Agrawal, Tomasz Imielinski, and Arun Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM Press, New York, 207--216.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Gregory Buehrer, Srinivasan Parthasarathy, and Amol Ghoting. 2006. Out-of-core frequent pattern mining on a commodity PC. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 86--95.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. B. Barla Cambazoglu, Emre Varol, Enver Kayaaslan, Cevdet Aykanat, and Ricardo Baeza-Yates. 2010. Query forwarding in geographically distributed search engines. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 90--97.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2001. Introduction to Algorithms. MIT Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, and Dasarathi Sampath. 2010. The YouTube video recommendation system. In Proceedings of the ACM Conference on Recommender Systems (RecSys’10). 293--296.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Farzad Eskandanian, Bamshad Mobasher, and Robin Burke. 2017. A clustering approach for personalizing diversity in collaborative recommender systems. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP’17). 280--284.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Rana Forsati and Mohammad Reza Meybodi. 2010. Effective page recommendation algorithms based on distributed learning automata and weighted association rules. Expert Syst. Appl. 37, 2 (2010), 1316--1330.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Gustavo Machado Campagnani Gama, W. Meira, Márcio L. B. Carvalho, Dorgival O. Guedes, and Virgílio A. F. Almeida. 2001. Resource placement in distributed e-commerce servers. In Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM’01), Vol. 3. IEEE, 1677--1682.Google ScholarGoogle Scholar
  13. Abhijeet Ghoshal, Syam Menon, and Sumit Sarkar. 2015. Recommendations using information from multiple association rules: A probabilistic approach. Information Syst. Res. 26, 3 (2015), 532--551.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Abhijeet Ghoshal and Sumit Sarkar. 2014. Association rules for recommendations with multiple items. Informs J. Comput. 26, 3 (2014), 433--448.Google ScholarGoogle ScholarCross RefCross Ref
  15. R. L. Graham. 1969. Bounds on multiprocessing timing anomalies. SIAM J. Appl. Math. 17, 2 (1969), 416--429.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Gösta Grahne and Jianfei Zhu. 2004. Mining frequent itemsets from secondary memory. In Proceedings of the International Conference on Data Mining (ICDM’04). 91--98.Google ScholarGoogle ScholarCross RefCross Ref
  17. Grahne Grahne and Jianfei Zhu. 2005. Fast algorithms for frequent itemset mining using FP-trees. IEEE Trans. Knowl. Data Eng. 17, 10 (2005), 1347--1362.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Guibing Guo, Jie Zhang, and Neil Yorke-Smith. 2016. A novel evidence-based Bayesian similarity measure for recommender systems. ACM Trans. Web 10, 2 (2016), 8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jiawei Han, Jian Pei, and Yiwen Yin. 2000. Mining frequent patterns without candidate generation. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM Press, New York, 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Negar Hariri, Bamshad Mobasher, and Robin Burke. 2012. Context-aware music recommendation based on latent topic sequential patterns. In Proceedings of the 6th ACM Conference on Recommender Systems. ACM, 131--138.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Choonho Kim and Juntae Kim. 2003. A recommendation algorithm using multi-level association rules. In Proceedings of the IEEE / WIC International Conference on Web Intelligence (WI’03). 524--527.Google ScholarGoogle Scholar
  22. Carson K. Leung, Fan Jiang, and Adam GM Pazdor. 2017. Bitwise parallel association rule mining for web page recommendation. In Proceedings of the International Conference on Web Intelligence. ACM, 662--669.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Cane Wing-ki Leung, Stephen Chi-fai Chan, and Fu-lai Chung. 2006. A collaborative filtering framework based on fuzzy association rules and multiple-level similarity. Knowl. Info. Syst. 10, 3 (2006), 357--381.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Haoyuan Li, Yi Wang, Dong Zhang, Ming Zhang, and Edward Y. Chang. 2008. PFP: Parallel fp-growth for query recommendation. In Proceedings of the ACM Conference on Recommender Systems (RecSys’08). 107--114.Google ScholarGoogle Scholar
  25. Wenmin Li, Jiawei Han, and Jian Pei. 2001. CMAR: Accurate and efficient classification based on multiple class-association rules. In Proceedings of the IEEE International Conference on Data Mining. 19--21.Google ScholarGoogle Scholar
  26. Weiyang Lin, Sergio A. Alvarez, and Carolina Ruiz. 2002. Efficient adaptive-support association rule mining for recommender systems. Data Mining Knowl. Discov. 6, 1 (2002), 83--105.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 7, 1 (2003), 76--80.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Duen-Ren Liu, Chin-Hui Lai, and Wang-Jung Lee. 2009. A hybrid of sequential rules and collaborative filtering for product recommendation. Info. Sci. 179, 20 (2009), 3505--3519.Google ScholarGoogle Scholar
  29. Xin Luo, Mengchu Zhou, Yunni Xia, and Qingsheng Zhu. 2014. An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Trans. Industr. Inform. 10, 2 (2014), 1273--1284.Google ScholarGoogle ScholarCross RefCross Ref
  30. Julian J. McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys’13). 165--172.Google ScholarGoogle Scholar
  31. Harshitha Menon and Laxmikant V. Kalé. 2013. A distributed dynamic load balancer for iterative applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC’13). 15.Google ScholarGoogle Scholar
  32. Bamshad Mobasher, Honghua Dai, Tao Luo, and Miki Nakagawa. 2001. Effective personalization based on association rule discovery from web usage data. In Proceedings of the 3rd International Workshop on Web Information and Data Management (WIDM’01). 9--15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Bamshad Mobasher, Honghua Dai, Tao Luo, and Miki Nakagawa. 2002. Discovery and evaluation of aggregate usage profiles for web personalization. Data Mining Knowl. Discov. 6, 1 (2002), 61--82.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kaweh Djafari Naini, Ismail Sengor Altingovde, and Wolf Siberski. 2016. Scalable and efficient web search result diversification. ACM Trans. Web 10, 3 (2016), 15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Miki Nakagawa and Bamshad Mobasher. 2003. A hybrid web personalization model based on site connectivity. In Proceedings of the SIGKDD Workshop on Web Mining and Web Usage Analysis (WebKDD’03). 59--70.Google ScholarGoogle Scholar
  36. Sergio Oramas, Vito Claudio Ostuni, Tommaso Di Noia, Xavier Serra, and Eugenio Di Sciascio. 2016. Sound and music recommendation with knowledge graphs. ACM Trans. Intell. Syst. Technol. 8, 2 (2016), 21.Google ScholarGoogle Scholar
  37. Srinivasan Parthasarathy, Mohammed Javeed Zaki, Mitsunori Ogihara, and Wei Li. 2001. Parallel data mining for association rules on shared-memory systems. Knowl. Info. Syst. 3, 1 (2001), 1--29.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. David Patterson and John L. Hennessy. 2012. Computer Architecture: A Quantitative Approach. Elsevier.Google ScholarGoogle Scholar
  39. Zujie Ren, Jian Wan, Weisong Shi, Xianghua Xu, and Min Zhou. 2014. Workload analysis, implications, and optimization on a production hadoop cluster: A case study on Taobao. IEEE Trans. Services Comput. 7, 2 (2014), 307--321.Google ScholarGoogle ScholarCross RefCross Ref
  40. Cynthia Rudin, Benjamin Letham, Ansaf Salleb-Aouissi, Eugene Kogan, and David Madigan. 2011. Sequential event prediction with association rules. Omnipress 19 (2011), 615--634.Google ScholarGoogle Scholar
  41. J. J. Sandvig, Bamshad Mobasher, and Robin Burke. 2007. Robustness of collaborative recommendation based on association rule mining. In Proceedings of the ACM Conference on Recommender Systems. 105--112.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Evan R. Sparks, Ameet Talwalkar, Valton Smith, Jey Kottalam, Xinghao Pan, Jose Gonzalez, Michael J. Franklin, Michael I. Jordan, and Tim Kraska. 2013. MLI: An API for distributed machine learning. In Proceedings of the IEEE 13th International Conference on Data Mining (ICDM’13). IEEE, 1187--1192.Google ScholarGoogle ScholarCross RefCross Ref
  43. Myra Spiliopoulou and Lukas C. Faulstich. 1998. WUM: A web utilization miner. In Proceedings of the EDBT Workshop (WebDB’98). Springer Verlag, 109--115.Google ScholarGoogle Scholar
  44. Jiaxi Tang and Ke Wang. 2018. Personalized Top-N sequential recommendation via convolutional sequence embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM’18). 565--573.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Fatemeh Vahedian, Robin Burke, and Bamshad Mobasher. 2017. Multirelational recommendation in heterogeneous networks. ACM Trans. Web 11, 3 (2017), 15:1--15:34.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Feng Hsu Wang and Hsiu Mei Shao. 2004. Effective personalized recommendation based on time-framed navigation clustering and association mining. Expert Syst. Appl. 27, 3 (2004), 365--377.Google ScholarGoogle ScholarCross RefCross Ref
  47. Yaqiong Wang, Junjie Wu, Zhiang Wu, Hua Yuan, and Xu Zhang. 2014. Popular items or niche items: Flexible recommendation using cosine patterns. In Proceedings of the IEEE International Conference on Data Mining Workshops. 205--212.Google ScholarGoogle ScholarCross RefCross Ref
  48. Geoffrey I. Webb. 2000. Efficient search for association rules. In Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 99--107.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Kasun Wickramaratna, Miroslav Kubat, and Kamal Premaratne. 2009. Predicting missing items in shopping carts. IEEE Trans. Knowl. Data Eng. 21, 7 (2009), 985--998.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Le Wu, Qi Liu, Enhong Chen, Nicholas Jing Yuan, Guangming Guo, and Xing Xie. 2016. Relevance meets coverage: A unified framework to generate diversified recommendations. ACM Trans. Intell. Syst. Technol. 7, 3 (2016), 39:1--39:30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Hui Xiong, Pang-Ning Tan, and Vipin Kumar. 2006. Hyperclique pattern discovery. Data Mining Knowl. Discov. J. 13, 2 (2006), 219--242.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Osmar R. Zaïane. 2002. Building a recommender agent for e-learning systems. In Proceedings of the IEEE International Conference on Computers in Education. IEEE, 55--59.Google ScholarGoogle ScholarCross RefCross Ref
  53. Heng-Ru Zhang, Fan Min, and Bing Shi. 2017. Regression-based three-way recommendation. Info. Sci. 378 (2017), 444--461.Google ScholarGoogle Scholar
  54. Mei Zheng, Fan Min, Heng-Ru Zhang, and Wen-Bin Chen. 2016. Fast recommendations with the M-distance. IEEE Access 4 (2016), 1464--1468.Google ScholarGoogle ScholarCross RefCross Ref
  55. Guixiang Zhu, Jie Cao, Changsheng Li, and Zhiang Wu. 2017. A recommendation engine for travel products based on topic sequential patterns. Multimedia Tools Appl. 76, 16 (2017), 17595--17612.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Jiaqi Zhu, Kaijun Wang, Yunkun Wu, Zhongyi Hu, and Hongan Wang. 2016. Mining user-aware rare sequential topic patterns in document streams. IEEE Trans. Knowl. Data Eng. 28, 7 (2016), 1790--1804.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. On Scalability of Association-rule-based Recommendation: A Unified Distributed-computing Framework

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!