skip to main content
research-article

Exploiting online music tags for music emotion classification

Published:04 November 2011Publication History
Skip Abstract Section

Abstract

The online repository of music tags provides a rich source of semantic descriptions useful for training emotion-based music classifier. However, the imbalance of the online tags affects the performance of emotion classification. In this paper, we present a novel data-sampling method that eliminates the imbalance but still takes the prior probability of each emotion class into account. In addition, a two-layer emotion classification structure is proposed to harness the genre information available in the online repository of music tags. We show that genre-based grouping as a precursor greatly improves the performance of emotion classification. On the average, the incorporation of online genre tags improves the performance of emotion classification by a factor of 55% over the conventional single-layer system. The performance of our algorithm for classifying 183 emotion classes reaches 0.36 in example-based f-score.

References

  1. Alex, B. S. O. and Smola, J. 2004. A tutorial on support vector machine. Statist. Comput. 14, 3, 199--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Andrews, S., Tsochantaridis, I., and Hofmann, T. 2003. Support vector machines for multiple-instance learning. In Proceedings of Neural Information Processing Systems.Google ScholarGoogle Scholar
  3. Beard, D. and Gloag, K. 2005. Musicology: The Key Concepts., Routledge, New York.Google ScholarGoogle Scholar
  4. Bello, J. P. and Pickens, J. 2005. A robust mid-level representation for harmonic content in music signals. In Proceedings of the International Conference on Music Information Retrieval. 304--311.Google ScholarGoogle Scholar
  5. Bertin-Mahieux, T., Eck, D., Maillet, F., and Lamere, P. 2008. Autotagger: A model for predicting social tags from acoustic features on large music databases. J. New Music Resear. 37, 2, 115--135.Google ScholarGoogle ScholarCross RefCross Ref
  6. Chang, C.-C. and Lin, C.-J. 2001. LIBSVM: a library for support vector machine. http://www.csie.ntu.edu.tw/~cjlin/libsvm. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Cheng, X., Dale, C., and Liu, J. 2008. Statistics and social network of Youtube videos. In Proceedings of the International Workshop on Quality of Service. 229--238.Google ScholarGoogle Scholar
  8. Dietterich, T. G., Lathrop, R. H., and Lozano-Perez, T. 1997. Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89, 1--2, 31--71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Drummond, C. and Holte, R. C. 2003. C4.5, class imbalance, and cost-sensitivity: why under-sampling beats over-sampling. In Working Notes of the ICML'03 Workshop on Learning from Imbalanced Data Sets.Google ScholarGoogle Scholar
  10. Duan, Z.-Y., Lu, L., and Zhang, C.-S. 2008. Collective annotation of music from multiple semantic categories. In Proceedings of the International Conference on Music Information Retrieval. 237--242.Google ScholarGoogle Scholar
  11. Duda, R. O., Hart, P. E., and Stork, D. G. 2000. Pattern Classification. John Wiley & Sons, Inc., New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Ellis, D. P. W. and Poliner, G. E. 2007. Identifying ‘cover songs’ with chroma features and dynamic programming beat tracking. In Proceedings of the International Conference on Audio, Speech and Signal Processing. 1429--1432.Google ScholarGoogle Scholar
  13. Feng, Y., Zhuang, Y., and Pan, Y. 2003. Popular music retrieval by detecting mood. In Proceedings of ACM SIGIR. 375--376. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hanks, W. 1987. Discourse genres in a theory of practice, Am. Ethnol. 14, 4, 668--692.Google ScholarGoogle ScholarCross RefCross Ref
  15. He, H. and Garcia, E. A. 2009. Learning from imbalanced data. IEEE Trans. Knowl. Data Engin. 21, 9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Hu, X. and Downie, J. S. 2007. Exploring mood metadata: relationships with genre, artist and usage metadata. In Proceedings of the International Conference on Music Information Retrieval.Google ScholarGoogle Scholar
  17. Hu, X., Downie, J. S., Laurier, C., Bay, M., and Ehmann, A. F. 2008. The 2007 MIREX audio mood classification task: Lessons learned. In Proceedings of the International Conference on Music Information Retrieval. 462--467.Google ScholarGoogle Scholar
  18. Hu, X. and Downie, and J. S. 2010. Improving mood classification in music digital libraries by combining lyrics and audio. In Proceedings of the Joint Conference on Digital Libraries. 159--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Huq, A., Bello, J. P., and Rowe, R. 2010. Automated music emotion recognition: A systematic evaluation. J. New Music Res. 39, 3, 227--244.Google ScholarGoogle ScholarCross RefCross Ref
  20. Huron, D. 2000. Perceptual and cognitive applications in music information retrieval. In Proceedings of the International Conference on Music Information Retrieval.Google ScholarGoogle Scholar
  21. Joachims, T. 2004. Learning to Classify Text Using Support Vector Machines. Kluwer Academic Publishers. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Juslin, P. N. and Laukka, P. 2004. Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. J. New Music Res. 33, 3, 217--238.Google ScholarGoogle ScholarCross RefCross Ref
  23. Kim, J. and Larsen, L. 2010. Music emotion and genre recognition towards new affective music taxonomy. In Proceedings of the AES Convention.Google ScholarGoogle Scholar
  24. Kim, Y. E., Schmidt, E. M., Migneco, R., Morton, B. G., Richardson, P., Scott, J., Speck, J., and Turnbull, D. 2010. Music emotion recognition: A state of the art review. In Proceedings of International Conference on Music Information Retrieval. 255--266.Google ScholarGoogle Scholar
  25. Lamere, P. 2008. Social tagging and music information retrieval. J. New Music Res. 37, 2, 101--114.Google ScholarGoogle ScholarCross RefCross Ref
  26. Laurier, C., Grivolla, J., and Herrera, P. 2008. Multimodal music mood classification using audio and lyrics. In Proceedings of the International Conference on Machine Learning and Applications. 105--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Lee, J. H. and Downie, J. S. 2004. Survey of music information needs, uses, and seeking behaviours: Preliminary findings. In Proceedings of International Conference on Music Information Retrieval. 441--446.Google ScholarGoogle Scholar
  28. Lewis, M., Haviland-Jones, J. M., and Barrett, L. F. 2008. Handbook of Emotions, Guilford Press, New York.Google ScholarGoogle Scholar
  29. Li, T. and Ogihara, M. 2003. Detecting emotion in music. In Proceedings of the International Conference on Music Information Retrieval. 239--240.Google ScholarGoogle Scholar
  30. Liu, X.-Y., Wu, J., and Zhou, Z.-H. 2009. Exploratory under-sampling for class-imbalance learning. IEEE Trans. Syst. Man Cyber. 39, 2, 539--553. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Lu, L., Liu, D., and Zhang, H. 2006. Automatic mood detection and tracking of music audio signals. IEEE Trans. Audio, Speech Lang. Proces. 14, 1, 5--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mandel, M. and Ellis, D. P. W. 2008a. A web-based game for collecting music metadata. J. New Music Res. 37, 2, 151--165.Google ScholarGoogle ScholarCross RefCross Ref
  33. Mandel, M. and Ellis, D. P. W. 2008b. Multiple instance learning for music information retrieval. In Proceedings of the International Conference on Music Information Retrieval. 577--5.Google ScholarGoogle Scholar
  34. Meyer, L. B. 1956. Emotion and Meaning in Music. University of Chicago Press.Google ScholarGoogle Scholar
  35. Picard, R. W. 1997. Affective Computing. The MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Rentfrow, P. J. and Gosling, S. D. 2003. The Do Re Mi's of everyday life: The structure and personality correlates of music preferences. J. Person. Soc. Psych. 84, 1236--1256.Google ScholarGoogle ScholarCross RefCross Ref
  37. Scaringella, N., Zoia, G., and Mlynek, D. 2006. Automatic genre classification of music content: a survey. IEEE Sign. Process. Mag. 23, 2, 133--141.Google ScholarGoogle ScholarCross RefCross Ref
  38. Shah, C. 2008. TubeKit: A query-based Youtube crawling toolkit. In Proceedings of the IEEE/ACM Joint Conference on Digital Library. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Slaney, M., Ellis, D. P. W., Sandler, M., Goto, M., and Goodwin, M. M. 2008. Introduction to the special issue on music information retrieval. IEEE Trans. Audio Speech Lang. Proces. 16, 2, 253--254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Schuller, B., Hage, C., Schuller, D., and Rigoll, G. 2010. Mister, D.J., Cheer Me Up!': Musical and textual features for automatic mood classification. J. New Music Res. 39, 1, 13--34.Google ScholarGoogle ScholarCross RefCross Ref
  41. Trohidis, K., Tsoumakas, G., Kalliris, G., and Vlahavas, I. 2008. Multi-label classification of music into emotions. In Proceedings of the International Conference on Music Information Retrieval. 325--330.Google ScholarGoogle Scholar
  42. Tsoumakas, G. and Vlahavas, I. 2007. Random k-labelsets: an ensemble method for multilabel classification. In Proceedings of the European Conference on Machine Learning. 406--417. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Turnbull, D., Liu, R., Barrington, L., and Lanckriet, G. 2007. A game-based approach for collecting semantic annotations of music. In Proceedings of International Conference on Music Information Retrieval. 535--538.Google ScholarGoogle Scholar
  44. Tzanetakis, G. and Cook, P. 2000. Marsyas: A framework for audio analysis. Orga. Sound 4, 3, 169--175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Tzanetakis, G. and Cook, P. 2002. Musical genre classification of audio signals. IEEE Trans. Speech Audio Process. 10, 5, 293--302.Google ScholarGoogle ScholarCross RefCross Ref
  46. von Ahn, L. and Dabbish, L. 2004. Labeling images with a computer game. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. 319--326. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Wieczorkowska, A., Synak, P., and Raś, Z. W. 2006. Multi-label classification of emotions in music. In Proceedings of the Intelligent Information Processing and Web Mining. 307--315.Google ScholarGoogle Scholar
  48. Yang, Y.-H., Liu, C.-C., and Chen, H. H. 2006. Music emotion classification: a fuzzy approach. In Proceedings of the ACM International Conference on Multimedia. 81--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Yang, Y.-H., Lin, Y.-C., Su, Y.-F., and Chen, H. H. 2008. A regression approach to music emotion recognition. IEEE Trans. Audio Speech Lang. Proces. 16, 2, 448--457. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Yang, Y.-H., Lin, Y.-C., Lee, A., and Chen, H. H. 2009. Improving musical concept detection by ordinal regression and context fusion. In Proceedings of the International Conference on Music Information Retrieval. 147--152.Google ScholarGoogle Scholar
  51. Yang, Y.-H. and Chen, H. H. 2011a. Music Emotion Recognition, CRC Taylor & Francis Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Yang, Y.-H. and Chen, H. H. 2011b. Ranking-based emotion recognition for music organization and retrieval. IEEE Trans. Audio Speech Lang. Process. 19, 4, 762--774. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Exploiting online music tags for music emotion classification

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!