skip to main content
research-article

Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems

Published:17 February 2011Publication History
Skip Abstract Section

Abstract

The technique of collaborative filtering is especially successful in generating personalized recommendations. More than a decade of research has resulted in numerous algorithms, although no comparison of the different strategies has been made. In fact, a universally accepted way of evaluating a collaborative filtering algorithm does not exist yet. In this work, we compare different techniques found in the literature, and we study the characteristics of each one, highlighting their principal strengths and weaknesses. Several experiments have been performed, using the most popular metrics and algorithms. Moreover, two new metrics designed to measure the precision on good items have been proposed.

The results have revealed the weaknesses of many algorithms in extracting information from user profiles especially under sparsity conditions. We have also confirmed the good results of SVD-based techniques already reported by other authors. As an alternative, we present a new approach based on the interpretation of the tendencies or differences between users and items. Despite its extraordinary simplicity, in our experiments, it obtained noticeably better results than more complex algorithms. In fact, in the cases analyzed, its results are at least equivalent to those of the best approaches studied. Under sparsity conditions, there is more than a 20% improvement in accuracy over the traditional user-based algorithms, while maintaining over 90% coverage. Moreover, it is much more efficient computationally than any other algorithm, making it especially adequate for large amounts of data.

References

  1. Aggarwal, C. C., Wolf, J. L., Wu, K.-L., and Yu, P. S. 1999. Horting hatches an egg: a new graph-theoretic approach to collaborative filtering. In Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'99). ACM, New York, NY, 201--212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Balabanović, M. and Shoham, Y. 1997. Fab: content-based, collaborative recommendation. Comm. ACM 40, 3, 66--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Basilico, J. and Hofmann, T. 2004. Unifying collaborative and content-based filtering. In Proceedings of the 21st International Conference on Machine Learning (ICML'04). ACM, New York, NY, 9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bennett, J. and Lanning, S. 2007. The netflix prixe. In Proceedings of KDD Cup and Workshop (KDDCup'07). ACM, 4.Google ScholarGoogle Scholar
  5. Billsus, D. and Pazzani, M. J. 1998. Learning collaborative information filters. In Proceedings of the 15th International Conference on Machine Learning. Morgan Kaufmann, San Francisco, CA, 46--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Breese, J. S., Heckerman, D., and Kadie, C. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence. 43--52. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Calderón-Benavides, M. L., González-Caro, C. N., de J. Pérez-Alcázar, J., García-Díaz, J. C., and Delgado, J. 2004. A comparison of several predictive algorithms for collaborative filtering on multi-valued ratings. In Proceedings of the ACM Symposium on Applied Computing (SAC'04). ACM, New York, NY, 1033--1039. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Canny, J. 2002. Collaborative filtering with privacy via factor analysis. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'02). ACM, New York, NY, 238--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chirita, P.-A., Nejdl, W., and Zamfir, C. 2005. Preventing shilling attacks in online recommender systems. In Proceedings of the 7th Annual ACM International Workshop on Web Information and Data Management (WIDM'05). ACM, New York, NY, 67--74. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Ellenberg, J. 2008. This psychologist might outsmart the math brains competing for the Netflix prize. Wired Maga. 16, 3.Google ScholarGoogle Scholar
  11. Fisher, D., Hildrum, K., Hong, J., Newman, M., Thomas, M., and Vuduc, R. 2000. Swami (poster session): A framework for collaborative filtering algorithm development and evaluation. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'00). ACM, New York, NY, 366--368. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Foltz, P. W. and Dumais, S. T. 1992. Personalized information delivery: an analysis of information filtering methods. Comm. ACM 35, 12, 51--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Funk, S. 2006. Netflix update: Try this at home. http://sifter.org/simon/journal/20061211.html.Google ScholarGoogle Scholar
  14. George, T. and Merugu, S. 2005. A scalable collaborative filtering framework based on co-clustering. In Proceedings of the 5th IEEE International Conference on Data Mining (ICDM'05). IEEE Computer Society, Los Alamitos, CA, 625--628. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Goldberg, D., Nichols, D., Oki, B. M., and Terry, D. 1992. Using collaborative filtering to weave an information tapestry. Comm. ACM 35, 12, 61--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Goldberg, K., Roeder, T., Gupta, D., and Perkins, C. 2001. Eigentaste: A constant time collaborative filtering algorithm. Inform. Retr. 4, 2, 133--151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Herlocker, J., Konstan, J. A., and Riedl, J. 2002. An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inform. Retr. 5, 4, 287--310. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Herlocker, J. L., Konstan, J. A., Borchers, A., and Riedl, J. 1999. An algorithmic framework for performing collaborative filtering. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'99). ACM, New York, NY, 230--237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. 2004. Evaluating collaborative filtering recommender systems. ACM Trans. Inform. Syst. 22, 1, 5--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Hofmann, T. 2004. Latent semantic models for collaborative filtering. ACM Trans. Inform. Syst. 22, 1, 89--115. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Huang, Z., Chen, H., and Zeng, D. 2004. Applying associative retrieval techniques to alleviate the sparsity problem in collaborative filtering. ACM Trans. Inform. Syst. 22, 1, 116--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Huang, Z., Zeng, D., and Chen, H. 2007. A comparison of collaborative-filtering recommendation algorithms for e-commerce. IEEE Intell. Syst. 22, 5, 68--78. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Jin, R., Si, L., and Zhai, C. 2003. Preference-based graphic models for collaborative filtering. In Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence. 329--336. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Karypis, G. 2001. Evaluation of item-based top-n recommendation algorithms. In Proceedings of the 10th International Conference on Information and Knowledge Management (CIKM'01). ACM, New York, NY, 247--254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Kohrs, A. and Mérialdo, B. 1999. Clustering for collaborative filtering applications. In Proceedings of the International Conference on Computational Intelligence for Modeling, Control and Automation (CIMCA'99).Google ScholarGoogle Scholar
  26. Koren, Y. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'08). ACM, New York, NY, 426--434. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Lam, S. K. and Riedl, J. 2004. Shilling recommender systems for fun and profit. In Proceedings of the 13th International Conference on World Wide Web (WWW'04). ACM, New York, NY, 393--402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Lemire, D. and Maclachlan, A. 2005. Slope one predictors for online rating-based collaborative filtering. In Proceedings of SIAM Data Mining (SDM'05).Google ScholarGoogle Scholar
  29. Marlin, B. 2004. Collaborative filtering: A machine learning perspective. M.S. thesis, University of Toronto.Google ScholarGoogle Scholar
  30. Marlin, B. M. and Zemel, R. S. 2009. Collaborative prediction and ranking with non-random missing data. In Proceedings of the 3rd ACM Conference on Recommender Systems (RecSys'09). ACM, New York, NY, 5--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. McNee, S. M., Riedl, J., and Konstan, J. A. 2006. Being accurate is not enough: how accuracy metrics have hurt recommender systems. In Extended Abstracts on Human Factors in Computing Systems (CHI'06). ACM, New York, NY, 1097--1101. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Melville, P., Mooney, R., and Nagarajan, R. 2001. Content-boosted collaborative filtering. In Proceedings of the ACM SIGIR Workshop on Recommender Systems.Google ScholarGoogle Scholar
  33. Mobasher, B., Burke, R., Bhaumik, R., and Williams, C. 2007. Toward trustworthy recommender systems: An analysis of attack models and algorithm robustness. ACM Trans. Internet Technol. 7, 4, 23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Paterek, A. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of the KDD Cup Workshop at the 13th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD'07). 39--42.Google ScholarGoogle Scholar
  35. Pennock, D., Horvitz, E., Lawrence, S., and Giles, C. L. 2000. Collaborative filtering by personality diagnosis: A hybrid memory- and model-based approach. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (UAI'2000). 473--480. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Rennie, J. D. M. and Srebro, N. 2005. Fast maximum margin matrix factorization for collaborative prediction. In Proceedings of the 22nd International Conference on Machine Learning (ICML'05). ACM, New York, NY, 713--719. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., and Riedl, J. 1994. Grouplens: an open architecture for collaborative filtering of netnews. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'94). ACM, New York, NY, 175--186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Sandvig, J. J., Mobasher, B., and Burke, R. 2007. Robustness of collaborative recommendation based on association rule mining. In Proceedings of the ACM Conference on Recommender Systems (RecSys'07). ACM, New York, NY, 105--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web (WWW'01). ACM, New York, NY, 285--295. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. 2000. Application of dimensionality reduction in recommender systems--a case study. In Proceedings of the ACM WebKDD Workshop.Google ScholarGoogle Scholar
  41. Schein, A. I., Popescul, A., Ungar, L. H., and Pennock, D. M. 2002. Methods and metrics for cold-start recommendations. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'02). ACM, New York, NY, 253--260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Shafiei, M. and Milios, E. 2006. Model-based Overlapping Co-Clustering. In Proceedings of the 4th Workshop on Text Mining at the 6th SIAM International Conference on Data Mining.Google ScholarGoogle Scholar
  43. Shardanand, U. 1994. Social information filtering for music recommendation. M.S. thesis, Massachussets Institute of Technology.Google ScholarGoogle Scholar
  44. Shardanand, U. and Maes, P. 1995. Social information filtering: algorithms for automating “word of mouth”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'95). ACM Press/Addison-Wesley Publishing Co., New York, NY, 210--217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Si, L. and Jin, R. 2003. A flexible mixture model for collaborative filtering. In Proceedings of the 20th International Conference on Machine Learning.Google ScholarGoogle Scholar
  46. Ungar, L. and Foster, D. 1998. Clustering methods for collaborative filtering. In Proceedings of the Workshop on Recommendation Systems. AAAI Press, Menlo Park, CA.Google ScholarGoogle Scholar
  47. Vucetic, S. and Obradovic, Z. 2000. A regression-based approach for scaling-up personalized recommender systems in e-commerce. In Proceedings of the ACM WebKDD Workshop.Google ScholarGoogle Scholar
  48. Wang, J., de Vries, A. P., and Reinders, M. J. T. 2006. Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, NY, 501--508. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Xue, G.-R., Lin, C., Yang, Q., Xi, W., Zeng, H.-J., Yu, Y., and Chen, Z. 2005. Scalable collaborative filtering using cluster-based smoothing. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'05). ACM, New York, NY, 114--121. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on the Web
      ACM Transactions on the Web  Volume 5, Issue 1
      February 2011
      150 pages
      ISSN:1559-1131
      EISSN:1559-114X
      DOI:10.1145/1921591
      Issue’s Table of Contents

      Copyright © 2011 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 17 February 2011
      • Accepted: 1 July 2010
      • Revised: 1 May 2009
      • Received: 1 September 2008
      Published in tweb Volume 5, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!