skip to main content
research-article

Mixtape: Using Real-Time User Feedback to Navigate Large Media Collections

Published:01 August 2017Publication History
Skip Abstract Section

Abstract

In this work, we explore the increasing demand for novel user interfaces to navigate large media collections. We implement a geometric data structure to store and retrieve item-to-item similarity information and propose a novel navigation framework that uses vector operations and real-time user feedback to direct the outcome. The framework is scalable to large media collections and is suitable for computationally constrained devices. In particular, we implement this framework in the domain of music. To evaluate the effectiveness of the navigation process, we propose an automatic evaluation framework, based on synthetic user profiles, which allows us to quickly simulate and compare navigation paths using different algorithms and datasets. Moreover, we perform a real user study. To do that, we developed and launched Mixtape, a simple web application that allows users to create playlists by providing real-time feedback through liking and skipping patterns.

References

  1. Amr Ahmed, Nino Shervashidze, Shravan Narayanamurthy, Vanja Josifovski, and Alexander J. Smola. 2013. Distributed large-scale natural graph factorization. In Proceedings of the 22nd International Conference on World Wide Web (WWW’13). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 37--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Kristina Andersen and Peter Knees. 2016. Conversations with expert users in music retrieval and research challenges for creative MIR. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16), ISMIR, 122--128. https://wp.nyu.edu/ismir2016/wp-content/uploads/sites/2294/2016/07/246_Paper.pdf.Google ScholarGoogle Scholar
  3. J.-J. Aucouturier, François Pachet, and Mark Sandler. 2005. “The way it Sounds”: Timbre models for analysis and retrieval of music signals. IEEE Trans. Multimed. 7, 6 (2005), 1028--1035. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 6 (June 2003), 1373--1396. DOI:http://dx.doi.org/10.1162/089976603321780317. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Pedro Cano, Markus Koppenberger, and Nicolas Wack. 2005. Content-based music audio recommendation. In Proceedings of the 13th Annual ACM International Conference on Multimedia. ACM, 211--212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. João Paulo V. Cardoso, Luciana Fujii Pontello, Pedro H. F. Holanda, Bruno Guilherme, Olga Goussevskaia, and Ana Paula Couto da Silva. 2016. Mixtape: Direction-based navigation in large media collections. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16). ISMIR, 454--460.Google ScholarGoogle Scholar
  7. Shuo Chen, Josh L. Moore, Douglas Turnbull, and Thorsten Joachims. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’12). ACM, New York, NY, 714--722. DOI:http://dx.doi.org/10.1145/2339530.2339643. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Shuo Chen, Jiexun Xu, and Thorsten Joachims. 2013. Multi-space probabilistic sequence modeling. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’13). ACM, New York, NY, 865--873. DOI:http://dx.doi.org/10.1145/2487575.2487632. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chia-Hao Chung, Jing-Kai Lou, and Homer Chen. 2016. A latent representation of users, sessions, and songs for listening behavior analysis. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16). ISMIR, 323--329.Google ScholarGoogle Scholar
  10. Trevor F. Cox and M. A. A. Cox. 2000. Multidimensional Scaling. Chapman and Hall/CRC.Google ScholarGoogle Scholar
  11. Vin De Silva and Joshua B. Tenenbaum. 2004. Sparse Multidimensional Scaling Using Landmark Points. Technical Report. Technical report, Stanford University.Google ScholarGoogle Scholar
  12. Katayoun Farrahi, Markus Schedl, Andreu Vall, David Hauger, and Marko Tkalcic. 2014. Impact of listening behavior on music recommendation. In Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR’14), 483--488.Google ScholarGoogle Scholar
  13. Benjamin Fields, Christophe Rhodes, Michael A. Casey, and Kurt Jacobson. 2008. Social playlists and bottleneck measurements: Exploiting musician social graphs using content-based dissimilarity and pairwise maximum flow values. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’08), Juan Pablo Bello, Elaine Chew, and Douglas Turnbull (Eds.). ISMIR, 559--564.Google ScholarGoogle Scholar
  14. Arthur Flexer. 2014. On inter-rater agreement in audio music similarity. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’14). ISMIR.Google ScholarGoogle Scholar
  15. Olga Goussevskaia, Michael Kuhn, Michael Lorenzi, and Roger Wattenhofer. 2008. From web to map: Exploring the world of music. In Proceedings of the Conference on Web Intelligence and Intelligent Agent Technology (WIIAT’08), Vol. 1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Pedro Holanda, Bruno Guilherme, Joo Paulo Cardoso, Ana Paula Couto da Silva, and Olga Goussevskaia. 2015a. Mapeando o universo da midia usando dados gerados por usuarios em redes sociais online. In Proceedings of the Brazilian Symposium on Computer Networks and Distributed Systems (SBRC’15). 1--12.Google ScholarGoogle Scholar
  17. Pedro Holanda, Bruno Guilherme, Ana Paula Couto da Silva, and Olga Goussevskaia. 2015b. TV goes social: Characterizing user interaction in an online social network for TV fans. In Proceedings of the 15th International Conference on Engineering the Web in the Big Data Era (ICWE’15), 182--199. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Ioannis Konstas, Vassilios Stathopoulos, and Joemon M Jose. 2009. On social networks and collaborative recommendation. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 195--202. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Michael Kuhn, Roger Wattenhofer, and Samuel Welten. 2010. Social audio features for advanced music retrieval interfaces. In Proceedings of the International Conference on Multimedia. ACM, 411--420. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning (ICML’14), 1188--1196.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Beth Logan. 2002. Content-based playlist generation: Exploratory experiments. In Proceedings of 3rd International Conference on Music Information Retrieval.Google ScholarGoogle Scholar
  22. Beth Logan and Ariel Salomon. 2001. A music similarity function based on signal analysis. Proceedings of the 2001 IEEE International Conference on Multimedia and Expo, 190. Google ScholarGoogle ScholarCross RefCross Ref
  23. Franois Maillet, Douglas Eck, Guillaume Desjardins, and Paul Lamere. 2009. Steerable playlist generation by learning song similarity from radio station playlists. In Proceedings of the 10th International Society for Music Information Retrieval Conference. 345--350.Google ScholarGoogle Scholar
  24. Brian McFee, Luke Barrington, and Gert Lanckriet. 2012. Learning content similarity for music recommendation. IEEE Trans. Aud. Speech Lang. Process. 20, 8 (2012), 2207--2218. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Joshua L Moore, Shuo Chen, Thorsten Joachims, and Douglas Turnbull. 2012. Learning to embed songs and tags for playlist prediction. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’12). 349--354.Google ScholarGoogle Scholar
  26. Joshua L. Moore, Shuo Chen, Douglas Turnbull, and Thorsten Joachims. 2013. Taste over time: The temporal dynamics of user preferences. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’13), Alceu de Souza Britto Jr., Fabien Gouyon, and Simon Dixon (Eds.). 401--406.Google ScholarGoogle Scholar
  27. Elias Pampalk, T. Pohle, and G. Widmer. 2005. Dynamic playlist generation based on skipping behavior. In Proceedings of the International Symposium on Music Information Retrieval.Google ScholarGoogle Scholar
  28. Maria Panteli, Emmanouil Benetos, and Simon Dixon. 2016. Learning a feature space for similarity in world music. In Proceedings of the 17th International Society for Music Information Retrieval Conference, (ISMIR’16). 538--544.Google ScholarGoogle Scholar
  29. Minsu Park, Ingmar Weber, Mor Naaman, and Sarah Vieweg. 2015. Understanding musical diversity via online social media. In Proceedings of the 9th International AAAI Conference on Weblogs and Social Media (ICWSM’15). 1--10.Google ScholarGoogle Scholar
  30. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 701--710. DOI:http://dx.doi.org/10.1145/2623330.2623732 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Andriy Shepitsen, Jonathan Gemmell, Bamshad Mobasher, and Robin Burke. 2008. Personalized recommendation in social tagging systems using hierarchical clustering. In Proceedings of the 2008 ACM Conference on Recommender Systems. ACM, 259--266. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. 1067--1077. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. J. B. Tenenbaum, V. Silva, and J. C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319--2323. Google ScholarGoogle ScholarCross RefCross Ref
  34. Douglas R. Turnbull, Justin A. Zupnick, Kristofer B. Stensland, Andrew R. Horwitz, Alexander J. Wolf, Alexander E. Spirgel, Stephen P. Meyerhofer, and Thorsten Joachims. 2014. Using personalized radio to enhance local music discovery. In Proceedings of the Conference on Extended Abstracts on Human Factors in Computing Systems (CHIEA’14). ACM, New York, NY, 2023--2028. DOI:http://dx.doi.org/10.1145/2559206.2581246 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Aaron Van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  36. Laurens J. P. Van der Maaten, Eric O. Postma, and H. Jaap van den Herik. 2009. Dimensionality reduction: A comparative review. J. Mach. Learn. Res. 10, 1--41 (2009), 66--71.Google ScholarGoogle Scholar

Index Terms

  1. Mixtape: Using Real-Time User Feedback to Navigate Large Media Collections

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!