skip to main content
research-article
Open Access

A survey of music similarity and recommendation from music context data

Published:27 December 2013Publication History
Skip Abstract Section

Abstract

In this survey article, we give an overview of methods for music similarity estimation and music recommendation based on music context data. Unlike approaches that rely on music content and have been researched for almost two decades, music-context-based (or contextual) approaches to music retrieval are a quite recent field of research within music information retrieval (MIR). Contextual data refers to all music-relevant information that is not included in the audio signal itself. In this article, we focus on contextual aspects of music primarily accessible through web technology. We discuss different sources of context-based data for individual music pieces and for music artists. We summarize various approaches for constructing similarity measures based on the collaborative or cultural knowledge incorporated into these data sources. In particular, we identify and review three main types of context-based similarity approaches: text-retrieval-based approaches (relying on web-texts, tags, or lyrics), co-occurrence-based approaches (relying on playlists, page counts, microblogs, or peer-to-peer-networks), and approaches based on user ratings or listening habits. This article elaborates the characteristics of the presented context-based measures and discusses their strengths as well as their weaknesses.

References

  1. Aizenberg, N., Koren, Y., and Somekh, O. 2012. Build your own music recommender by modeling internet radio streams. In Proceedings of the 21st International Conference on World Wide Web. ACM, New York, 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Aucouturier, J.-J. and Pachet, F. 2002. Scaling up music playlist generation. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'02). 105--108.Google ScholarGoogle Scholar
  3. Aucouturier, J.-J., Pachet, F., Roy, P., and Beurivé, A. 2007. Signal + Context = Better Classification. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  4. Aucouturier, J.-J., Pachet, F., and Sandler, M. 2005. “The Way It Sounds”: Timbre models for analysis and retrieval of music signals. IEEE Trans. Multimed. 7, 6, 1028--1035. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Baccigalupo, C., Plaza, E., and Donaldson, J. 2008. Uncovering affinity of artists to multiple genres from social behaviour data. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google ScholarGoogle Scholar
  6. Barrington, L., Turnbull, D., Yazdani, M., and Lanckriet, G. 2009. Combining audio content and social context for semantic music discovery. In Proceedings of the 32nd ACM SIGIR. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Baumann, S. and Hummel, O. 2003. Using cultural metadata for artist recommendation. In Proceedings of the 3rd International Conference on Web Delivering of Music (WEDELMUSIC'03).Google ScholarGoogle Scholar
  8. Bell, R. M. and Koren, Y. 2007. Lessons from the Netflix prize challenge. SIGKDD Explorat. 9, 2, 75--79. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Berenzweig, A., Logan, B., Ellis, D. P., and Whitman, B. 2003. A large-scale evaluation of acoustic and subjective music similarity measures. In Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR'03).Google ScholarGoogle Scholar
  10. Brill, E. 1992. A simple rule-based part of speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing. 152--155. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Brochu, E., de Freitas, N., and Bao, K. 2003. The sound of an album cover: Probabilistic multimedia and IR. In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics.Google ScholarGoogle Scholar
  12. Cano, P. and Koppenberger, M. 2004. The emergence of complex network patterns in music artist networks. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 466--469.Google ScholarGoogle Scholar
  13. Casey, M. A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., and Slaney, M. 2008. Content-based music information retrieval: Current directions and future challenges. Proc. IEEE 96, 668--696.Google ScholarGoogle ScholarCross RefCross Ref
  14. Celma, O. 2008. Music recommendation and discovery in the long tail. Ph.D. thesis, Universitat Pompeu Fabra, Barcelona, Spain.Google ScholarGoogle Scholar
  15. Celma, O., Cano, P., and Herrera, P. 2006. SearchSounds: An audio crawler focused on weblogs. In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06).Google ScholarGoogle Scholar
  16. Celma, O. and Lamere, P. 2007. ISMIR 2007 tutorial: Music recommendation. http://www.dtic.upf.edu/˜ocelma/Music RecommendationTutorial-ISMIR2007/ (last accessed: April 2013).Google ScholarGoogle Scholar
  17. Charniak, E. 1997. Statistical techniques for natural language parsing. AI Magazine 18, 33--44.Google ScholarGoogle Scholar
  18. Chen, S., Moore, J., Turnbull, D., and Joachims, T. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 714--722. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Cohen, W. W. and Fan, W. 2000. Web-collaborative filtering: Recommending music by crawling the web. Computer Netw. 33, 1--6, 685--698. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. 1990. Indexing by latent semantic analysis. J. ASIS 41, 391--407.Google ScholarGoogle ScholarCross RefCross Ref
  21. Dror, G., Koenigstein, N., and Koren, Y. 2011a. Yahoo! music recommendations: Modeling music ratings with temporal dynamics and item taxonomy. In Proceedings of the 5th ACM Conference on Recommender Systems. ACM. 165--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Dror, G., Koenigstein, N., Koren, Y., and Weimer, M. 2011b. The Yahoo! music dataset and KDD-Cup'11. J. Mach. Learn. Res. 8, 3--18.Google ScholarGoogle Scholar
  23. Eck, D., Lamere, P., Bertin-Mahieux, T., and Green, S. 2008. Automatic generation of social tags for music recommendation. In Advances in Neural Information Processing Systems 20 (NIPS'07). MIT Press.Google ScholarGoogle Scholar
  24. Ellis, D. P., Whitman, B., Berenzweig, A., and Lawrence, S. 2002. The quest for ground truth in musical artist similarity. In Proceedings of 3rd International Conference on Music Information Retrieval (ISMIR'02).Google ScholarGoogle Scholar
  25. Fields, B., Casey, M., Jacobson, K., and Sandler, M. 2008. Do you sound like your friends? Exploring artist similarity via artist social network relationships and audio signal processing. In Proceedings of the International Computer Music Conference (ICMC'08).Google ScholarGoogle Scholar
  26. Geleijnse, G., Schedl, M., and Knees, P. 2007. The quest for ground truth in musical artist tagging in the social web era. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  27. Hofmann, T. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence (UAI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Hu, X., Downie, J. S., and Ehmann, A. F. 2009. Lyric text mining in music mood classification. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).Google ScholarGoogle Scholar
  29. Hu, X., Downie, J. S., West, K., and Ehmann, A. 2005. Mining music reviews: Promising preliminary results. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR'05).Google ScholarGoogle Scholar
  30. Jacobson, K., Fields, B., and Sandler, M. 2008. Using audio analysis and network structure to identify communities in on-line social networks of artists. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google ScholarGoogle Scholar
  31. Kim, J. H., Tomasik, B., and Turnbull, D. 2009. Using artist similarity to propagate semantic information. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).Google ScholarGoogle Scholar
  32. Kleedorfer, F., Knees, P., and Pohle, T. 2008. Oh Oh Oh Whoah! Towards automatic topic detection in song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08). 287--292.Google ScholarGoogle Scholar
  33. Knees, P., Pampalk, E., and Widmer, G. 2004. Artist classification with web-based data. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 517--524.Google ScholarGoogle Scholar
  34. Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2006a. Combining audio-based similarity with web-based data to accelerate automatic music playlist generation. In Proceedings of the 8th ACM SIGMM International Workshop on Multimedia Information Retrieval (MIR'06) (Santa Barbara, CA). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2007. A music search engine built upon audio-based and web-based similarity measures. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Knees, P., Schedl, M., and Pohle, T. 2008. A deeper look into web-based classification of music artists. In Proceedings of 2nd Workshop on Learning the Semantics of Audio Signals (LSAS'08).Google ScholarGoogle Scholar
  37. Knees, P., Schedl, M., Pohle, T., and Widmer, G. 2006b. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In Proceedings of the 14th ACM International Conference on Multimedia (MM'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Laurier, C., Grivolla, J., and Herrera, P. 2008. Multimodal music mood classification using audio and lyrics. In Proceedings of the International Conference on Machine Learning and Applications. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Law, E., von Ahn, L., Dannenberg, R., and Crawford, M. 2007. Tagatune: A game for music and sound annotation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  40. Levy, M. and Sandler, M. 2007. A semantic space for music derived from social tags. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  41. L&iamcr;beks, J. and Turnbull, D. 2011. You can judge an artist by an album cover: Using images for music annotation. IEEE MultiMedia, 18, 4, 30--37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 4, 1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Logan, B., Ellis, D. P., and Berenzweig, A. 2003. Toward evaluation techniques for music similarity. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03): Workshop on the Evaluation of Music Information Retrieval Systems.Google ScholarGoogle Scholar
  44. Logan, B., Kositsky, A., and Moreno, P. 2004. Semantic analysis of song lyrics. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'04).Google ScholarGoogle Scholar
  45. MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, L. M. L. Cam and J. Neyman, Eds., Statistics Series, vol. I, University of California Press, Berkeley and Los Angeles, CA, 281--297.Google ScholarGoogle Scholar
  46. Mahedero, J. P. G., Martínez, A., Cano, P., Koppenberger, M., and Gouyon, F. 2005. Natural language processing of lyrics. In Proceedings of the 13th ACM International Conference on Multimedia (MM'05). 475--478. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Mandel, M. I. and Ellis, D. P. 2007. A web-based game for collecting music metadata. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  48. Mayer, R., Neumayer, R., and Rauber, A. 2008. Rhyme and style features for musical genre classification by song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google ScholarGoogle Scholar
  49. McFee, B. and Lanckriet, G. 2009. Heterogeneous embedding for subjective artist similarity. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).Google ScholarGoogle Scholar
  50. Mesnage, C. S., Rafiq, A., Dixon, S., and Brixtel, R. P. 2011. Music discovery with social networks. In Proceedings of the 2nd Workshop on Music Recommendation and Discovery (WOMRAD'11). 7--12.Google ScholarGoogle Scholar
  51. Nanopoulos, A., Rafailidis, D., Symeonidis, P., and Manolopoulos, Y. 2010. Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans. Audio, Speech, Lang. Process. 18, 2, 407--412. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Pachet, F., Westerman, G., and Laigre, D. 2001. Musical data mining for electronic music distribution. In Proceedings of the 1st International Conference on Web Delivering of Music (WEDELMUSIC'01). Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Pampalk, E., Flexer, A., and Widmer, G. 2005. Hierarchical organization and description of music collections at the artist level. In Proceedings of the 9th European Conference on Research and Advanced Technology for Digital Libraries (ECDL'05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Pampalk, E. and Goto, M. 2007. MusicSun: A new approach to artist recommendation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  55. Pohle, T., Knees, P., Schedl, M., Pampalk, E., and Widmer, G. 2007. “Reinventing the Wheel”: A novel approach to music player interfaces. IEEE Trans. Multimed. 9, 567--575. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Pohle, T., Knees, P., Schedl, M., and Widmer, G. 2007. Building an interactive next-generation artist recommender based on automatically derived high-level concepts. In Proceedings of the 5th International Workshop on Content-Based Multimedia Indexing (CBMI'07).Google ScholarGoogle Scholar
  57. Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of 10th International Conference on World Wide Web (WWW'01). 285--295. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Schedl, M. 2008. Automatically extracting, analyzing, and visualizing information on music artists from the world wide web. Ph.D. thesis, Johannes Kepler University, Linz, Austria.Google ScholarGoogle Scholar
  59. Schedl, M. and Hauger, D. 2012. Mining microblogs to infer music artist similarity and cultural listening patterns. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Schedl, M., Hauger, D., and Schnitzer, D. 2012. A model for serendipitous music retrieval. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'12): 2nd International Workshop on Context-awareness in Retrieval and Recommendation (CaRR'12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Schedl, M. and Knees, P. 2009. Context-based music similarity estimation. In Proceedings of the 3rd International Workshop on Learning the Semantics of Audio Signals (LSAS'09).Google ScholarGoogle Scholar
  62. Schedl, M. and Knees, P. 2011. Personalization in multimodal music retrieval. In Proceedings of the 9th Workshop on Adaptive Multimedia Retrieval (AMR'11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Schedl, M., Knees, P., Pohle, T., and Widmer, G. 2006. Towards automatic retrieval of album covers. In Proceedings of the 28th European Conference on Information Retrieval (ECIR'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Schedl, M., Knees, P., and Widmer, G. 2005. A web-based approach to assessing artist similarity using co-occurrences. In Proceedings of the 4th International Workshop on Content-Based Multimedia Indexing (CBMI'05).Google ScholarGoogle Scholar
  65. Schedl, M., Pohle, T., Knees, P., and Widmer, G. 2011. Exploring the music similarity space on the web. ACM Trans. Info. Syst. 29, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Serra, X. 2012. Data gathering for a culture specific approach in MIR. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Shavitt, Y. and Weinsberg, U. 2009. Songs clustering using peer-to-peer co-occurrences. In Proceedings of the IEEE International Symposium on Multimedia (ISM'09): International Workshop on Advances in Music Information Research (AdMIRe'09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Shen, J., Meng, W., Yan, S., Pang, H., and Hua, X. 2010. Effective music tagging through advanced statistical modeling. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 635--642. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Slaney, M. 2011. Web-scale multimedia analysis: Does content matter? IEEE MultiMedia 18, 2, 12--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Slaney, M. and White, W. 2007. Similarity based on rating data. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  71. Sordo, M., Laurier, C., and Celma, O. 2007. Annotating music collections: How content-based similarity helps to propagate labels. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07). 531--534.Google ScholarGoogle Scholar
  72. Turnbull, D., Barrington, L., and Lanckriet, G. 2008. Five approaches to collecting tags for music. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google ScholarGoogle Scholar
  73. Turnbull, D., Liu, R., Barrington, L., and Lanckriet, G. 2007. A game-based approach for collecting semantic annotations of music. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google ScholarGoogle Scholar
  74. Wang, X., Rosenblum, D., and Wang, Y. 2012. Context-aware mobile music recommendation for daily activities. In Proceedings of the 20th ACM International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Whitman, B. and Lawrence, S. 2002. Inferring descriptions and similarity for music from community metadata. In Proceedings of the 2002 International Computer Music Conference (ICMC'02). 591--598.Google ScholarGoogle Scholar
  76. Yang, D., Chen, T., Zhang, W., Lu, Q., and Yu, Y. 2012. Local implicit feedback mining for music recommendation. In Proceedings of the 6th ACM Conference on Recommender Systems. 91--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Zadel, M. and Fujinaga, I. 2004. Web services for music information retrieval. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04).Google ScholarGoogle Scholar
  78. Zangerle, E., Gassler, W., and Specht, G. 2012. Exploiting Twitter's collective knowledge for music recommendations. In Proceedings of the 21st International World Wide Web Conference (WWW'12): Making Sense of Microposts (#MSM'12). 14--17.Google ScholarGoogle Scholar
  79. Zhang, B., Xiang, Q., Lu, H., Shen, J., and Wang, Y. 2009. Comprehensive query-dependent fusion using regression-on-folksonomies: A case study of multimodal music search. In Proceedings of the 17th ACM International Conference on Multimedia. 213--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Zhao, Z., Wang, X., Xiang, Q., Sarroff, A., Li, Z., and Wang, Y. 2010. Large-scale music tag recommendation with explicit multiple attributes. In Proceedings of the 18th ACM International Conference on Multimedia. 401--410. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Zobel, J. and Moffat, A. 1998. Exploring the Similarity Space. ACM SIGIR Forum 32, 1, 18--34. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A survey of music similarity and recommendation from music context data

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 10, Issue 1
            December 2013
            166 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/2559928
            Issue’s Table of Contents

            Copyright © 2013 Owner/Author

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 27 December 2013
            • Accepted: 1 April 2013
            • Revised: 1 January 2013
            • Received: 1 September 2012
            Published in tomm Volume 10, Issue 1

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!