Abstract
In this survey article, we give an overview of methods for music similarity estimation and music recommendation based on music context data. Unlike approaches that rely on music content and have been researched for almost two decades, music-context-based (or contextual) approaches to music retrieval are a quite recent field of research within music information retrieval (MIR). Contextual data refers to all music-relevant information that is not included in the audio signal itself. In this article, we focus on contextual aspects of music primarily accessible through web technology. We discuss different sources of context-based data for individual music pieces and for music artists. We summarize various approaches for constructing similarity measures based on the collaborative or cultural knowledge incorporated into these data sources. In particular, we identify and review three main types of context-based similarity approaches: text-retrieval-based approaches (relying on web-texts, tags, or lyrics), co-occurrence-based approaches (relying on playlists, page counts, microblogs, or peer-to-peer-networks), and approaches based on user ratings or listening habits. This article elaborates the characteristics of the presented context-based measures and discusses their strengths as well as their weaknesses.
- Aizenberg, N., Koren, Y., and Somekh, O. 2012. Build your own music recommender by modeling internet radio streams. In Proceedings of the 21st International Conference on World Wide Web. ACM, New York, 1--10. Google Scholar
Digital Library
- Aucouturier, J.-J. and Pachet, F. 2002. Scaling up music playlist generation. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'02). 105--108.Google Scholar
- Aucouturier, J.-J., Pachet, F., Roy, P., and Beurivé, A. 2007. Signal + Context = Better Classification. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Aucouturier, J.-J., Pachet, F., and Sandler, M. 2005. “The Way It Sounds”: Timbre models for analysis and retrieval of music signals. IEEE Trans. Multimed. 7, 6, 1028--1035. Google Scholar
Digital Library
- Baccigalupo, C., Plaza, E., and Donaldson, J. 2008. Uncovering affinity of artists to multiple genres from social behaviour data. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google Scholar
- Barrington, L., Turnbull, D., Yazdani, M., and Lanckriet, G. 2009. Combining audio content and social context for semantic music discovery. In Proceedings of the 32nd ACM SIGIR. Google Scholar
Digital Library
- Baumann, S. and Hummel, O. 2003. Using cultural metadata for artist recommendation. In Proceedings of the 3rd International Conference on Web Delivering of Music (WEDELMUSIC'03).Google Scholar
- Bell, R. M. and Koren, Y. 2007. Lessons from the Netflix prize challenge. SIGKDD Explorat. 9, 2, 75--79. Google Scholar
Digital Library
- Berenzweig, A., Logan, B., Ellis, D. P., and Whitman, B. 2003. A large-scale evaluation of acoustic and subjective music similarity measures. In Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR'03).Google Scholar
- Brill, E. 1992. A simple rule-based part of speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing. 152--155. Google Scholar
Digital Library
- Brochu, E., de Freitas, N., and Bao, K. 2003. The sound of an album cover: Probabilistic multimedia and IR. In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics.Google Scholar
- Cano, P. and Koppenberger, M. 2004. The emergence of complex network patterns in music artist networks. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 466--469.Google Scholar
- Casey, M. A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., and Slaney, M. 2008. Content-based music information retrieval: Current directions and future challenges. Proc. IEEE 96, 668--696.Google Scholar
Cross Ref
- Celma, O. 2008. Music recommendation and discovery in the long tail. Ph.D. thesis, Universitat Pompeu Fabra, Barcelona, Spain.Google Scholar
- Celma, O., Cano, P., and Herrera, P. 2006. SearchSounds: An audio crawler focused on weblogs. In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06).Google Scholar
- Celma, O. and Lamere, P. 2007. ISMIR 2007 tutorial: Music recommendation. http://www.dtic.upf.edu/˜ocelma/Music RecommendationTutorial-ISMIR2007/ (last accessed: April 2013).Google Scholar
- Charniak, E. 1997. Statistical techniques for natural language parsing. AI Magazine 18, 33--44.Google Scholar
- Chen, S., Moore, J., Turnbull, D., and Joachims, T. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 714--722. Google Scholar
Digital Library
- Cohen, W. W. and Fan, W. 2000. Web-collaborative filtering: Recommending music by crawling the web. Computer Netw. 33, 1--6, 685--698. Google Scholar
Digital Library
- Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. 1990. Indexing by latent semantic analysis. J. ASIS 41, 391--407.Google Scholar
Cross Ref
- Dror, G., Koenigstein, N., and Koren, Y. 2011a. Yahoo! music recommendations: Modeling music ratings with temporal dynamics and item taxonomy. In Proceedings of the 5th ACM Conference on Recommender Systems. ACM. 165--172. Google Scholar
Digital Library
- Dror, G., Koenigstein, N., Koren, Y., and Weimer, M. 2011b. The Yahoo! music dataset and KDD-Cup'11. J. Mach. Learn. Res. 8, 3--18.Google Scholar
- Eck, D., Lamere, P., Bertin-Mahieux, T., and Green, S. 2008. Automatic generation of social tags for music recommendation. In Advances in Neural Information Processing Systems 20 (NIPS'07). MIT Press.Google Scholar
- Ellis, D. P., Whitman, B., Berenzweig, A., and Lawrence, S. 2002. The quest for ground truth in musical artist similarity. In Proceedings of 3rd International Conference on Music Information Retrieval (ISMIR'02).Google Scholar
- Fields, B., Casey, M., Jacobson, K., and Sandler, M. 2008. Do you sound like your friends? Exploring artist similarity via artist social network relationships and audio signal processing. In Proceedings of the International Computer Music Conference (ICMC'08).Google Scholar
- Geleijnse, G., Schedl, M., and Knees, P. 2007. The quest for ground truth in musical artist tagging in the social web era. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Hofmann, T. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence (UAI). Google Scholar
Digital Library
- Hu, X., Downie, J. S., and Ehmann, A. F. 2009. Lyric text mining in music mood classification. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).Google Scholar
- Hu, X., Downie, J. S., West, K., and Ehmann, A. 2005. Mining music reviews: Promising preliminary results. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR'05).Google Scholar
- Jacobson, K., Fields, B., and Sandler, M. 2008. Using audio analysis and network structure to identify communities in on-line social networks of artists. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google Scholar
- Kim, J. H., Tomasik, B., and Turnbull, D. 2009. Using artist similarity to propagate semantic information. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).Google Scholar
- Kleedorfer, F., Knees, P., and Pohle, T. 2008. Oh Oh Oh Whoah! Towards automatic topic detection in song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08). 287--292.Google Scholar
- Knees, P., Pampalk, E., and Widmer, G. 2004. Artist classification with web-based data. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 517--524.Google Scholar
- Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2006a. Combining audio-based similarity with web-based data to accelerate automatic music playlist generation. In Proceedings of the 8th ACM SIGMM International Workshop on Multimedia Information Retrieval (MIR'06) (Santa Barbara, CA). Google Scholar
Digital Library
- Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2007. A music search engine built upon audio-based and web-based similarity measures. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'07). Google Scholar
Digital Library
- Knees, P., Schedl, M., and Pohle, T. 2008. A deeper look into web-based classification of music artists. In Proceedings of 2nd Workshop on Learning the Semantics of Audio Signals (LSAS'08).Google Scholar
- Knees, P., Schedl, M., Pohle, T., and Widmer, G. 2006b. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In Proceedings of the 14th ACM International Conference on Multimedia (MM'06). Google Scholar
Digital Library
- Laurier, C., Grivolla, J., and Herrera, P. 2008. Multimodal music mood classification using audio and lyrics. In Proceedings of the International Conference on Machine Learning and Applications. Google Scholar
Digital Library
- Law, E., von Ahn, L., Dannenberg, R., and Crawford, M. 2007. Tagatune: A game for music and sound annotation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Levy, M. and Sandler, M. 2007. A semantic space for music derived from social tags. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- L&iamcr;beks, J. and Turnbull, D. 2011. You can judge an artist by an album cover: Using images for music annotation. IEEE MultiMedia, 18, 4, 30--37. Google Scholar
Digital Library
- Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 4, 1. Google Scholar
Digital Library
- Logan, B., Ellis, D. P., and Berenzweig, A. 2003. Toward evaluation techniques for music similarity. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03): Workshop on the Evaluation of Music Information Retrieval Systems.Google Scholar
- Logan, B., Kositsky, A., and Moreno, P. 2004. Semantic analysis of song lyrics. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'04).Google Scholar
- MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, L. M. L. Cam and J. Neyman, Eds., Statistics Series, vol. I, University of California Press, Berkeley and Los Angeles, CA, 281--297.Google Scholar
- Mahedero, J. P. G., Martínez, A., Cano, P., Koppenberger, M., and Gouyon, F. 2005. Natural language processing of lyrics. In Proceedings of the 13th ACM International Conference on Multimedia (MM'05). 475--478. Google Scholar
Digital Library
- Mandel, M. I. and Ellis, D. P. 2007. A web-based game for collecting music metadata. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Mayer, R., Neumayer, R., and Rauber, A. 2008. Rhyme and style features for musical genre classification by song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google Scholar
- McFee, B. and Lanckriet, G. 2009. Heterogeneous embedding for subjective artist similarity. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).Google Scholar
- Mesnage, C. S., Rafiq, A., Dixon, S., and Brixtel, R. P. 2011. Music discovery with social networks. In Proceedings of the 2nd Workshop on Music Recommendation and Discovery (WOMRAD'11). 7--12.Google Scholar
- Nanopoulos, A., Rafailidis, D., Symeonidis, P., and Manolopoulos, Y. 2010. Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans. Audio, Speech, Lang. Process. 18, 2, 407--412. Google Scholar
Digital Library
- Pachet, F., Westerman, G., and Laigre, D. 2001. Musical data mining for electronic music distribution. In Proceedings of the 1st International Conference on Web Delivering of Music (WEDELMUSIC'01). Google Scholar
Digital Library
- Pampalk, E., Flexer, A., and Widmer, G. 2005. Hierarchical organization and description of music collections at the artist level. In Proceedings of the 9th European Conference on Research and Advanced Technology for Digital Libraries (ECDL'05). Google Scholar
Digital Library
- Pampalk, E. and Goto, M. 2007. MusicSun: A new approach to artist recommendation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Pohle, T., Knees, P., Schedl, M., Pampalk, E., and Widmer, G. 2007. “Reinventing the Wheel”: A novel approach to music player interfaces. IEEE Trans. Multimed. 9, 567--575. Google Scholar
Digital Library
- Pohle, T., Knees, P., Schedl, M., and Widmer, G. 2007. Building an interactive next-generation artist recommender based on automatically derived high-level concepts. In Proceedings of the 5th International Workshop on Content-Based Multimedia Indexing (CBMI'07).Google Scholar
- Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of 10th International Conference on World Wide Web (WWW'01). 285--295. Google Scholar
Digital Library
- Schedl, M. 2008. Automatically extracting, analyzing, and visualizing information on music artists from the world wide web. Ph.D. thesis, Johannes Kepler University, Linz, Austria.Google Scholar
- Schedl, M. and Hauger, D. 2012. Mining microblogs to infer music artist similarity and cultural listening patterns. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12). Google Scholar
Digital Library
- Schedl, M., Hauger, D., and Schnitzer, D. 2012. A model for serendipitous music retrieval. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'12): 2nd International Workshop on Context-awareness in Retrieval and Recommendation (CaRR'12). Google Scholar
Digital Library
- Schedl, M. and Knees, P. 2009. Context-based music similarity estimation. In Proceedings of the 3rd International Workshop on Learning the Semantics of Audio Signals (LSAS'09).Google Scholar
- Schedl, M. and Knees, P. 2011. Personalization in multimodal music retrieval. In Proceedings of the 9th Workshop on Adaptive Multimedia Retrieval (AMR'11). Google Scholar
Digital Library
- Schedl, M., Knees, P., Pohle, T., and Widmer, G. 2006. Towards automatic retrieval of album covers. In Proceedings of the 28th European Conference on Information Retrieval (ECIR'06). Google Scholar
Digital Library
- Schedl, M., Knees, P., and Widmer, G. 2005. A web-based approach to assessing artist similarity using co-occurrences. In Proceedings of the 4th International Workshop on Content-Based Multimedia Indexing (CBMI'05).Google Scholar
- Schedl, M., Pohle, T., Knees, P., and Widmer, G. 2011. Exploring the music similarity space on the web. ACM Trans. Info. Syst. 29, 3. Google Scholar
Digital Library
- Serra, X. 2012. Data gathering for a culture specific approach in MIR. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12). Google Scholar
Digital Library
- Shavitt, Y. and Weinsberg, U. 2009. Songs clustering using peer-to-peer co-occurrences. In Proceedings of the IEEE International Symposium on Multimedia (ISM'09): International Workshop on Advances in Music Information Research (AdMIRe'09). Google Scholar
Digital Library
- Shen, J., Meng, W., Yan, S., Pang, H., and Hua, X. 2010. Effective music tagging through advanced statistical modeling. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 635--642. Google Scholar
Digital Library
- Slaney, M. 2011. Web-scale multimedia analysis: Does content matter? IEEE MultiMedia 18, 2, 12--15. Google Scholar
Digital Library
- Slaney, M. and White, W. 2007. Similarity based on rating data. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Sordo, M., Laurier, C., and Celma, O. 2007. Annotating music collections: How content-based similarity helps to propagate labels. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07). 531--534.Google Scholar
- Turnbull, D., Barrington, L., and Lanckriet, G. 2008. Five approaches to collecting tags for music. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).Google Scholar
- Turnbull, D., Liu, R., Barrington, L., and Lanckriet, G. 2007. A game-based approach for collecting semantic annotations of music. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).Google Scholar
- Wang, X., Rosenblum, D., and Wang, Y. 2012. Context-aware mobile music recommendation for daily activities. In Proceedings of the 20th ACM International Conference on Multimedia. Google Scholar
Digital Library
- Whitman, B. and Lawrence, S. 2002. Inferring descriptions and similarity for music from community metadata. In Proceedings of the 2002 International Computer Music Conference (ICMC'02). 591--598.Google Scholar
- Yang, D., Chen, T., Zhang, W., Lu, Q., and Yu, Y. 2012. Local implicit feedback mining for music recommendation. In Proceedings of the 6th ACM Conference on Recommender Systems. 91--98. Google Scholar
Digital Library
- Zadel, M. and Fujinaga, I. 2004. Web services for music information retrieval. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04).Google Scholar
- Zangerle, E., Gassler, W., and Specht, G. 2012. Exploiting Twitter's collective knowledge for music recommendations. In Proceedings of the 21st International World Wide Web Conference (WWW'12): Making Sense of Microposts (#MSM'12). 14--17.Google Scholar
- Zhang, B., Xiang, Q., Lu, H., Shen, J., and Wang, Y. 2009. Comprehensive query-dependent fusion using regression-on-folksonomies: A case study of multimodal music search. In Proceedings of the 17th ACM International Conference on Multimedia. 213--222. Google Scholar
Digital Library
- Zhao, Z., Wang, X., Xiang, Q., Sarroff, A., Li, Z., and Wang, Y. 2010. Large-scale music tag recommendation with explicit multiple attributes. In Proceedings of the 18th ACM International Conference on Multimedia. 401--410. Google Scholar
Digital Library
- Zobel, J. and Moffat, A. 1998. Exploring the Similarity Space. ACM SIGIR Forum 32, 1, 18--34. Google Scholar
Digital Library
Index Terms
A survey of music similarity and recommendation from music context data
Recommendations
Music similarity and retrieval
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrievalThis tutorial serves as an introductory course to the field of and state-of-the-art in music information retrieval (MIR) and in particular to music similarity estimation which is an essential component of music retrieval. Apart from explaining ...
Music Retrieval and Recommendation: A Tutorial Overview
SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information RetrievalIn this tutorial, we give an introduction to the field of and state of the art in music information retrieval (MIR). The tutorial particularly spotlights the question of music similarity, which is an essential aspect in music retrieval and ...
Novel Approach for Music Search Using Music Contents and Human Perception
ICESC '14: Proceedings of the 2014 International Conference on Electronic Systems, Signal Processing and Computing TechnologiesMusic similarity can be perceived by the listeners in different ways depending on individual preferences to musical parameters. Metadata based similarity measures are used by many websites for music recommendation and retrieval. Metadata attached to ...






Comments