skip to main content
research-article

White Box: On the Prediction of Collaborative Filtering Recommendation Systems’ Performance

Published:23 February 2023Publication History
Skip Abstract Section

Abstract

Collaborative Filtering (CF) recommendation algorithms are a popular solution to the information overload problem, aiding users in the item selection process. Relevant research has long focused on refining and improving these models to produce better (more effective) recommendations, and has converged on a methodology to predict their effectiveness on target datasets by evaluating them on random samples of the latter. However, predicting the efficiency of the solutions—especially with regard to their time- and resource-hungry training phase, whose requirements dwarf those of the prediction/recommendation phase—has received little to no attention in the literature. This article addresses this gap for a number of representative and highly popular CF models, including algorithms based on matrix factorization, k-nearest neighbors, co-clustering, and slope one schemes. To this end, we first study the computational complexity of the training phase of said CF models and derive time and space complexity equations. Then, using characteristics of the input and the aforementioned equations, we contribute a methodology for predicting the processing time and memory usage of their training phase. Our contributions further include an adaptive sampling strategy, to address the tradeoff between resource usage costs and prediction accuracy, and a framework that quantifies both the efficiency and effectiveness of CF. Finally, a systematic experimental evaluation demonstrates that our method outperforms state-of-the-art regression schemes by a considerable margin, with an overhead that is a small fraction of the overall requirements of CF training.

REFERENCES

  1. [1] Adomavicius Gediminas and Zhang Jingjing. 2012. Impact of data characteristics on recommender systems performance. ACM Transactions on Management Information Systems (TMIS) 3, 1 (2012), 117.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Charu C. Aggarwal 2016. Recommender Systems. Vol. 1. Springer, Berlin.Google ScholarGoogle Scholar
  3. [3] Arora Sanjeev and Barak Boaz. 2009. Computational Complexity: A Modern Approach. Cambridge University Press, Cambridge, UK.Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Austin Homer W.. 1983. Sample size: How much is enough? Quality and Quantity 17, 3 (1983), 239245.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Awad Mariette and Khanna Rahul. 2015. Support vector regression. In Efficient Learning Machines. Springer, Berlin, 6780.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Bartz-Beielstein Thomas and Markon Sandor. 2004. Tuning search algorithms for real-world applications: A regression tree based approach. In Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753), Vol. 1. IEEE, 11111118.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Beladev Moran, Rokach Lior, and Shapira Bracha. 2016. Recommender systems for product bundling. Knowledge-based Systems 111 (2016), 193206.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Bellogín Alejandro and Castells Pablo. 2010. A performance prediction approach to enhance collaborative filtering performance. In European Conference on Information Retrieval. Springer, Berlin, 382393.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Bellogín Alejandro, Castells Pablo, and Cantador Iván. 2017. Statistical biases in information retrieval metrics for recommender systems. Information Retrieval Journal 20, 6 (2017), 606634.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Bjorck Ake. 1996. Numerical Methods for Least Squares Problems. Vol. 51. SIAM, Philadelphia, PA.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Brewer Eric A.. 1995. High-level optimization via automated statistical modeling. ACM SIGPLAN Notices 30, 8 (1995), 8091.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Brooks Steve, Gelman Andrew, Jones Galin, and Meng Xiao-Li. 2011. Handbook of Markov Chain Monte Carlo. CRC Press, Florida.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Dinh-Mao Bui, YongIk Yoon, Eui-Nam Huh, SungIk Jun, and Sungyoung Lee. 2017. Energy efficiency for cloud computing system based on predictive optimization. J. Parallel and Distrib. Comput. 102 (2017), 103–114.Google ScholarGoogle Scholar
  14. [14] Bulteau Laurent, Froese Vincent, Hartung Sepp, and Niedermeier Rolf. 2016. Co-clustering under the maximum norm. Algorithms 9, 1 (2016), 17.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Cacheda Fidel, Carneiro Víctor, Fernández Diego, and Formoso Vreixo. 2011. Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Transactions on the Web (TWEB) 5, 1 (2011), 133.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Cacheda Fidel, Carneiro Víctor, Fernández Diego, and Formoso Vreixo. 2011. Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Trans. Web 5, 1, Article 2 (Feb.2011), 33 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Cañamares Rocío and Castells Pablo. 2018. Should I follow the crowd? A probabilistic analysis of the effectiveness of popularity in recommender systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 415424.Google ScholarGoogle Scholar
  18. [18] Cañamares Rocío and Castells Pablo. 2020. On target item sampling in offline recommender system evaluation. In 14th ACM Conference on Recommender Systems. ACM, 259268.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Cañamares Rocío, Castells Pablo, and Moffat Alistair. 2020. Offline evaluation options for recommender systems. Information Retrieval Journal 23, 4 (2020), 387410.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Carlin Bradley P. and Louis Thomas A.. 2008. Bayesian Methods for Data Analysis. CRC Press, Florida.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Surajit Chaudhuri, Rajeev Motwani, and Vivek Narasayya. 1998. Random sampling for histogram construction: How much is enough? ACM SIGMOD Record 27, 2 (1998), 436–447.Google ScholarGoogle Scholar
  22. [22] Cline Alan Kaylor and Dhillon Inderjit S.. 2007. Computation of the singular value decomposition. In Handbook of Linear Algebra, Hogben Leslie (Ed.). Chapman & Hall/CRC, Boca Raton, FL, Chapter 45, 45–1–45–13.Google ScholarGoogle Scholar
  23. [23] Cormen Thomas H., Leiserson Charles E., Rivest Ronald L., and Stein Clifford. 2009. Introduction to Algorithms (3rd ed.). MIT Press, Cambridge, MA.Google ScholarGoogle Scholar
  24. [24] Deldjoo Yashar, Noia Tommaso Di, Sciascio Eugenio Di, and Merra Felice Antonio. 2020. How dataset characteristics affect the robustness of collaborative recommendation models. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 951960.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Desrosiers Christian and Karypis George. 2011. A comprehensive survey of neighborhood-based recommendation methods. In Recommender Systems Handbook. Springer, Boston, MA, 107144.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Dice Dave and Kogan Alex. 2021. Optimizing inference performance of transformers on CPUs. In Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys’21). ACM, 18.Google ScholarGoogle Scholar
  27. [27] Draper Norman R. and Smith Harry. 1998. Applied Regression Analysis. Vol. 326. John Wiley & Sons, Hoboken, NJ.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Eugene Fink. 1998. How to solve it automatically: Selection among problem solving methods. In Proceedings of the Fourth International Conference on Artificial Intelligence Planning Systems. Association for the Advancement of Artificial Intelligence (AAAI) Press, 128–136. Pittsburgh, USA.Google ScholarGoogle Scholar
  29. [29] Freund Yoav and Schapire Robert E.. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 1 (1997), 119139.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Friedman Jerome, Hastie Trevor, and Tibshirani Robert. 2001. The Elements of Statistical Learning. Vol. 1 (10). Springer Series in Statistics, New York.Google ScholarGoogle Scholar
  31. [31] Gelman Andrew, Carlin John B., Stern Hal S., Dunson David B., Vehtari Aki, and Rubin Donald B.. 2013. In Bayesian Data Analysis. CRC Press, Florida.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Thomas George and Srujana Merugu. 2005. A scalable collaborative filtering framework based on co-clustering. In Fifth IEEE International Conference on Data Mining (ICDM’05), Houston, USA. IEEE, 4–pp.Google ScholarGoogle Scholar
  33. [33] Gibbons Phillip B., Matias Yossi, and Poosala Viswanath. 2002. Fast incremental maintenance of approximate histograms. ACM Transactions on Database Systems (TODS) 27, 3 (2002), 261298.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Gini Corrado. 1921. Measurement of inequality of incomes. Economic Journal 31, 121 (1921), 124126.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Gunawardana Asela and Shani Guy. 2015. Evaluating recommender systems. In Recommender Systems Handbook. Springer, Boston, MA, 265308.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Haas Peter J. and Swami Arun N.. 1992. Sequential sampling procedures for query size estimation. In Proceedings of the 1992 ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 341350.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Harper F. Maxwell and Konstan Joseph A.. 2015. The Movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TIIS) 5, 4 (2015), 119.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Herlocker Jonathan L., Konstan Joseph A., Borchers Al, and Riedl John. 1999. An algorithmic framework for performing collaborative filtering. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 230237.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Herlocker Jonathan L., Konstan Joseph A., Terveen Loren G., and Riedl John T.. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS) 22, 1 (2004), 553.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Hou Wen-Chi, Ozsoyoglu Gultekin, and Dogdu Erdogan. 1991. Error-constrained COUNT query evaluation in relational databases. ACM SIGMOD Record 20, 2 (1991), 278287.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Howe Adele E., Dahlman Eric, Hansen Christopher, Scheetz Michael, and Mayrhauser Anneliese Von. 1999. Exploiting competitive planner performance. In European Conference on Planning. Springer, Berlin, 6272.Google ScholarGoogle Scholar
  42. [42] Hu Yifan, Koren Yehuda, and Volinsky Chris. 2008. Collaborative filtering for implicit feedback datasets. In 2008 8th IEEE International Conference on Data Mining. IEEE, 263272.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Huang Jing, Li Renfa, An Jiyao, Ntalasha Derrick, Yang Fan, and Li Keqin. 2017. Energy-efficient resource utilization for heterogeneous embedded computing systems. IEEE Trans. Comput. 66, 9 (2017), 15181531.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Huang Zan, Zeng Daniel, and Chen Hsinchun. 2007. A comparison of collaborative-filtering recommendation algorithms for e-commerce. IEEE Intelligent Systems 22, 5 (2007), 6878.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Hug Nicolas. 2020. Surprise: A python library for recommender systems. Journal of Open Source Software 5, 52 (2020), 2174. Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Koren Yehuda. 2010. Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Transactions on Knowledge Discovery from Data (TKDD) 4, 1 (2010), 124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Lam Shyong K., LaPitz Adam, Karypis George, Riedl John, et al. 2006. Towards a scalable kNN CF algorithm: Exploring effective applications of clustering. In International Workshop on Knowledge Discovery on the Web. Springer, Berlin, 147166.Google ScholarGoogle Scholar
  48. [48] Lan Hai, Bao Zhifeng, and Peng Yuwei. 2021. A survey on advancing the DBMS query optimizer: Cardinality estimation, cost model, and plan enumeration. Data Science and Engineering 6, 1 (2021), 116.Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] LeDell Erin and Poirier Sebastien. 2020. H2O AutoML: Scalable automatic machine learning. In 7th ICML Workshop on Automated Machine Learning (AutoML’20). ICML, 116. https://www.automl.org/wp-content/uploads/2020/07/AutoML_2020_paper_61.pdfGoogle ScholarGoogle Scholar
  50. [50] Leis Viktor and Kuschewski Maximilian. 2021. Towards cost-optimal query processing in the cloud. Proceedings of the VLDB Endowment 14, 9 (2021), 16061612.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Lemire Daniel and Maclachlan Anna. 2005. Slope one predictors for online rating-based collaborative filtering. In Proceedings of the 2005 SIAM International Conference on Data Mining. SIAM, 471475.Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Leyton-Brown Kevin, Nudelman Eugene, and Shoham Yoav. 2002. Learning the empirical hardness of optimization problems: The case of combinatorial auctions. In International Conference on Principles and Practice of Constraint Programming. Springer, 556572.Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Li Jiexing, König Arnd Christian, Narasayya Vivek, and Chaudhuri Surajit. 2012. Robust estimation of resource consumption for SQL queries using statistical techniques. Proceedings of the VLDB Endowment 5, 11 (2012), 112.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. [54] Liang Ting-Peng, Lai Hung-Jen, and Ku Yi-Cheng. 2006. Personalized content recommendation and user satisfaction: Theoretical synthesis and empirical findings. Journal of Management Information Systems 23, 3 (2006), 4570.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Lipton Richard J. and Naughton Jeffrey F.. 1995. Query size estimation by adaptive sampling. J. Comput. System Sci. 51, 1 (1995), 1825.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. [56] Ludewig Malte, Mauro Noemi, Latifi Sara, and Jannach Dietmar. 2019. Performance comparison of neural and non-neural approaches to session-based recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. ACM, 462466.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. [57] Luo Xin, Zhou Mengchu, Xia Yunni, and Zhu Qingsheng. 2014. An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Transactions on Industrial Informatics 10, 2 (2014), 12731284.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] L’heureux Alexandra, Grolinger Katarina, Elyamany Hany F., and Capretz Miriam A. M.. 2017. Machine learning with big data: Challenges and approaches. IEEE Access 5 (2017), 77767797.Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Manku Gurmeet Singh, Rajagopalan Sridhar, and Lindsay Bruce G.. 1999. Random sampling techniques for space efficient online computation of order statistics of large datasets. ACM SIGMOD Record 28, 2 (1999), 251262.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Meeker William Q., Hahn Gerald J., and Escobar Luis A.. 2017. Statistical Intervals: A Guide for Practitioners and Researchers. Vol. 541. John Wiley & Sons, Hoboken, NJ.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. [61] Moulin Pierre and Veeravalli Venugopal V.. 2018. Maximum likelihood estimation. In Statistical Inference for Engineers and Data Scientists. Cambridge University Press, Cambridge, UK, 319357. Google ScholarGoogle ScholarCross RefCross Ref
  62. [62] Murphy Kevin P.. 2012. Machine Learning: A Probabilistic Perspective. MIT Press, London, UK.Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. [63] Paun Iulia. 2020. Efficiency-effectiveness trade-offs in recommendation systems. In Proceedings of the 14th ACM Conference on Recommender Systems (RecSys’20). ACM, 770775.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. [64] Paun Iulia, Moshfeghi Yashar, and Ntarmos Nikos. 2021. Are we there yet? Estimating training time for recommendation systems. In Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys’21). ACM, 19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. [65] Reiss Charles, Tumanov Alexey, Ganger Gregory R., Katz Randy H., and Kozuch Michael A.. 2012. Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In Proceedings of the 3rd ACM Symposium on Cloud Computing. ACM, 113.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. [66] Ricci Francesco, Rokach Lior, Shapira Bracha, and Kantor Paul B.. 2010. Recommender Systems Handbook (1st ed.). Springer, Boston, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. [67] Ridge Enda and Kudenko Daniel. 2007. Tuning the performance of the MMAS heuristic. In International Workshop on Engineering Stochastic Local Search Algorithms. Springer, Berlin, 4660.Google ScholarGoogle Scholar
  68. [68] Roberts Mark, Howe Adele, and Flom Landon. 2007. Learned models of performance for many planners. In ICAPS 2007 Workshop AI Planning and Learning. ICAPS, 3640.Google ScholarGoogle Scholar
  69. [69] Rook Laurens, Sabic Adem, and Zanker Markus. 2020. Engagement in proactive recommendations. Journal of Intelligent Information Systems 54, 1 (2020), 79100.Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. [70] Scikit-learn.org. 2020. Stochastic gradient descent 0.23.0 documentation. https://scikit-learn.org/stable/modules/sgd.html #complexity.Google ScholarGoogle Scholar
  71. [71] Shapiro Alexander. 2003. Monte Carlo sampling methods. Handbooks in Operations Research and Management Science 10 (2003), 353425.Google ScholarGoogle ScholarCross RefCross Ref
  72. [72] Shardanand Upendra and Maes Pattie. 1995. Social information filtering: Algorithms for automating “word of mouth.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 210217.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. [73] Shrestha Durga L. and Solomatine Dimitri P.. 2006. Machine learning approaches for estimation of prediction interval for the model output. Neural Networks 19, 2 (2006), 225235.Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. [74] Smith Brent and Linden Greg. 2017. Two decades of recommender systems at Amazon.com. IEEE Internet Computing 21, 3 (2017), 1218.Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. [75] Smith Ralph C.. 2013. Uncertainty Quantification: Theory, Implementation, and Applications. Vol. 12. SIAM, Philadelphia, PA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. [76] Strubell Emma, Ganesh Ananya, and McCallum Andrew. 2019. Energy and policy considerations for deep learning in NLP. CoRR abs/1906.02243 (2019), 16. arxiv:1906.02243 http://arxiv.org/abs/1906.02243Google ScholarGoogle Scholar
  77. [77] Sun Xiaoyang, Hu Chunming, Yang Renyu, Garraghan Peter, Wo Tianyu, Xu Jie, Zhu Jianyong, and Li Chao. 2018. Rose: Cluster resource scheduling via speculative over-subscription. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS’18). IEEE, 949960.Google ScholarGoogle ScholarCross RefCross Ref
  78. [78] Sun Zilei, Luo Nianlong, and Kuang Wei. 2011. One real-time personalized recommendation systems based on slope one algorithm. In 2011 8th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD’11), Vol. 3. IEEE, 18261830.Google ScholarGoogle ScholarCross RefCross Ref
  79. [79] Tan Zhiqiang. 2006. Monte Carlo integration with acceptance-rejection. Journal of Computational and Graphical Statistics 15, 3 (2006), 735752.Google ScholarGoogle ScholarCross RefCross Ref
  80. [80] Tosh Christopher and Dasgupta Sanjoy. 2019. The relative complexity of maximum likelihood estimation, map estimation, and sampling. In Conference on Learning Theory. PMLR, 29933035.Google ScholarGoogle Scholar
  81. [81] Vairale Vaishali S. and Shukla Samiksha. 2021. Recommendation of food items for thyroid patients using content-based KNN method. In Data Science and Security. Springer, Berlin, 7177.Google ScholarGoogle ScholarCross RefCross Ref
  82. [82] Aken Dana Van, Pavlo Andrew, Gordon Geoffrey J., and Zhang Bohan. 2017. Automatic database management system tuning through large-scale machine learning. In Proceedings of the 2017 ACM International Conference on Management of Data. ACM, 10091024.Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. [83] Wang Zhongya, Liu Ying, and Ma Pengshan. 2014. A CUDA-enabled parallel implementation of collaborative filtering. Procedia Computer Science 30 (2014), 6674.Google ScholarGoogle ScholarCross RefCross Ref
  84. [84] Wu Ling-Ling, Joung Yuh-Jzer, and Lee Jonglin. 2013. Recommendation systems and consumer satisfaction online: Moderating effects of consumer product awareness. In 2013 46th Hawaii International Conference on System Sciences. IEEE, 27532762.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. [85] Yang Longqi, Bagdasaryan Eugene, Gruenstein Joshua, Hsieh Cheng-Kang, and Estrin Deborah. 2018. Openrec: A modular framework for extensible and adaptable recommendation algorithms. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 664672.Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. [86] Yeomans Michael, Shah Anuj, Mullainathan Sendhil, and Kleinberg Jon. 2019. Making sense of recommendations. Journal of Behavioral Decision Making 32, 4 (2019), 403414.Google ScholarGoogle ScholarCross RefCross Ref
  87. [87] Yeung Gingfung, Borowiec Damian, Friday Adrian, Harper Richard, and Garraghan Peter. 2020. Towards GPU utilization prediction for cloud deep learning. In 12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud’20). USENIX, Online, 19.Google ScholarGoogle Scholar
  88. [88] Zajac Zygmunt. 2017. Goodbooks-10k: a new dataset for book recommendations. http://fastml.com/goodbooks-10k.Google ScholarGoogle Scholar
  89. [89] Zhang Heng-Ru, Min Fan, Zhang Zhi-Heng, and Wang Song. 2019. Efficient collaborative filtering recommendations with multi-channel feature vectors. International Journal of Machine Learning and Cybernetics 10, 5 (2019), 11651172.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. White Box: On the Prediction of Collaborative Filtering Recommendation Systems’ Performance

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Internet Technology
        ACM Transactions on Internet Technology  Volume 23, Issue 1
        February 2023
        564 pages
        ISSN:1533-5399
        EISSN:1557-6051
        DOI:10.1145/3584863
        • Editor:
        • Ling Liu
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 23 February 2023
        • Online AM: 12 August 2022
        • Accepted: 21 July 2022
        • Revised: 6 June 2022
        • Received: 13 September 2021
        Published in toit Volume 23, Issue 1

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
      • Article Metrics

        • Downloads (Last 12 months)155
        • Downloads (Last 6 weeks)16

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!