skip to main content
research-article

Semi-supervised distance metric learning for collaborative image retrieval and clustering

Published:27 August 2010Publication History
Skip Abstract Section

Abstract

Learning a good distance metric plays a vital role in many multimedia retrieval and data mining tasks. For example, a typical content-based image retrieval (CBIR) system often relies on an effective distance metric to measure similarity between any two images. Conventional CBIR systems simply adopting Euclidean distance metric often fail to return satisfactory results mainly due to the well-known semantic gap challenge. In this article, we present a novel framework of Semi-Supervised Distance Metric Learning for learning effective distance metrics by exploring the historical relevance feedback log data of a CBIR system and utilizing unlabeled data when log data are limited and noisy. We formally formulate the learning problem into a convex optimization task and then present a new technique, named as “Laplacian Regularized Metric Learning” (LRML). Two efficient algorithms are then proposed to solve the LRML task. Further, we apply the proposed technique to two applications. One direct application is for Collaborative Image Retrieval (CIR), which aims to explore the CBIR log data for improving the retrieval performance of CBIR systems. The other application is for Collaborative Image Clustering (CIC), which aims to explore the CBIR log data for enhancing the clustering performance of image pattern clustering tasks. We conduct extensive evaluation to compare the proposed LRML method with a number of competing methods, including 2 standard metrics, 3 unsupervised metrics, and 4 supervised metrics with side information. Encouraging results validate the effectiveness of the proposed technique.

References

  1. Bar-Hillel, A., Hertz, T., Shental, N., and Weinshall, D. 2005. Learning a Mahalanobis metric from equivalence constraints. J. Mach. Learn. Resear. 6, 937--965. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Beygelzlmer, A., Kakade, S., and Langford, J. 2006. Cover trees for nearest neighbor. In Proceedings of the 23rd International Conference on Machine Learning (ICML '06). ACM, New York, 97--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Boyd, S. and Vandenberghe, L. 2003. Convex Optimization. Cambridge University Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Cox, T. and Cox, M. 1994. Multidimensional Scaling. Chapman & Hall, London.Google ScholarGoogle Scholar
  5. Dom, B. E. 2001. An information-theoretic external cluster-validity measure. Res. rep. RJ 10219, IBM.Google ScholarGoogle Scholar
  6. Fukunaga, K. 1990. Introduction to Statistical Pattern Recognition. Elsevier. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Girosi, F., Jones, M., and Poggio, T. 1995. Regularization theory and neural networks architectures. Neur. Comput. 7, 219--269. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Globerson, A. and Roweis, S. 2005. Metric learning by collapsing classes. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS'05).Google ScholarGoogle Scholar
  9. Goldberger, J. S, Roweis, G, Hinton, J, and Salakhutdinov, R. 2005. Neighbourhood components analysis. In Advances in Neural Information Processing Systems 17. MIT Press.Google ScholarGoogle Scholar
  10. He, X., Ma, W.-Y., and Zhang, H.-J. 2004. Learning an image manifold for retrieval. In Proceedings of the ACM International Conference on Multimedia. 17--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Hoi, C.-H. and Liu, M. R. 2004a. Group-based relevance feeedback with support vector machine ensembles. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR'04),' 874--877. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Hoi, C.-H. and Liu, M. R. 2004b. A novel log-based relevance feedback technique in content-based image retrieval. In Proceedings of the ACM International Conference on Multimedia. ACM Press, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Hoi, S. C., Lyu, M. R., and Jin, R. 2005. Integrating user feedback log into relevance feedback by coupled SVM for content-based image retrieval. In Proceedings of the IEEE ICDE Workshopon Managing Data for Emerging Multimedia Applications (EMMA'05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hoi, S. C., Liu, W., and Chang, S.-F. 2008. Semi-supervised distance metric learning for collaborative image retrieval. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08).Google ScholarGoogle Scholar
  15. Hoi, S. C., Liu, W., Lyu, M. R., and Ma, W.- Y. 2006. Learning distance metrics with contextual constraints for image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Hoi, S. C., Lyu, M. R., and Jin, R. 2006. A unified log-based relevance feedback scheme for image retrieval. IEEE Trans. Knowl. Data. Engin. 18, 4, 509--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jain, A, K. and Vailaya, A. 1998. Shape-based retrieval: a case study with trademark image database. Patt. Recog. 31, 1369--1390.Google ScholarGoogle ScholarCross RefCross Ref
  18. Jain, A. K., Murty, M, N., and Flynn, P. I. 1999. Data clustering: a review. ACM Comput. Surv. 31, 3, 264--323. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. King, I. and Zhong, J. 2003. Integrated probability function and its application to content-based image retrieval by relevance feedback. Patt. Recog. 36, 9, 2177--2186.Google ScholarGoogle ScholarCross RefCross Ref
  20. Kuhn, H. W. 1982. Nonlinear programming: A historical view. SIGMAP Bull. 31, 6--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Lee, J.-E., Jin, R., and Jain, A. K. 2008. Rank-based distance metric learning: An application to image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarGoogle Scholar
  22. Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimedia Comput. Commun., Appl. 2, 1, 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Liu, Y., Jin, R., and Jain, A. K. 2007. Boostcluster: Boosting clustering by pairwise constraints. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'07). 450--459. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Manjunath, B., Wu, P., Newsam, S., and Shin, H. 2001. A texture descriptor for browsing and similarity retrieval.Sign. Process. Image Commun.Google ScholarGoogle Scholar
  25. Müller, H., Pun, T., and Squire, D. 2004. Learning from user behavior in image retrieval: Application of market basket analysis. Int. J. Comput. Vis. 56, 1--2, 65--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Roweis, S. and Saul, L. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500, 2323--2326.Google ScholarGoogle Scholar
  27. Rui, Y., Huang, T., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in MARS. In Proceedings of the IEEE Conference on Image Processing. 815--818.Google ScholarGoogle Scholar
  28. Rui, Y., Huang, T., Ortega, M., and Mehrotra, S. 1998. Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans, Circ. Syst. Video. Techn. 8, 5, 644--655. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Salton, G. and Buckley, C. 1988. Term-weighting approaches in automatic text retrieval. Inform. Process. Manage. 24, 5, 513--523. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Si, L., Jin, R., Hoi, S. C., and Lyu, M. R. 2006. Collaborative image retrieval via regularized metric learning. ACM Multimedia Syst. J. 12, 1, 34--44.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 12, 1349--1380 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Strehl, E., Ghosh, J., and Mooney, R. 2000. Impact of similarity measures on web-page clustering. In Proceedings of the Workshop on Artificial Intelligence for Web Search (AAAI '00). AAAI, 58--64.Google ScholarGoogle Scholar
  33. Sturm, J. F. 1999. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones.Optimiz. Meth. Softw. 11--12, 625--653.Google ScholarGoogle Scholar
  34. Tao, D. and Tang, X. 2004. Random sampling based SVM for relevance feedback image retrieval. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Tenenbaum, J. B., de Silva, V., and Langford, J. C. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500, 2319--2323.Google ScholarGoogle Scholar
  36. Tong, S., and Chang, E. 2001. Support vector machine active learning for image retrieval. In Proceedings of the 9th ACM international conference on Multimedia. ACM Press, New York, 107--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Vapnik, V. N. 1998. Statistical Learning Theory, John Wiley & Sons.Google ScholarGoogle Scholar
  38. Weinberger, K., Blitzer, J., and Saul, L. 2006. Distance metric learning for large margin nearest neighbor classification. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 1473--1480.Google ScholarGoogle Scholar
  39. Xing, E, P., Ng, A. y., Jordan, M. I., and Russell, S. 2002. Distance metric learning with application to clustering with side-information. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  40. Yan, R., Zhang, J., Yang, J., and Hauptmann, A. G. 2006. A discriminative learning framework with pairwise constraints for video object classification. IEEE Trans. Patt. Anal. Mach. Intell. 28, 4, 578. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yang, L., Jin, R., Sukthankar, R., and Liu, Y. 2006. An efficient algorithm for local distance metric learning. In Proceedings of the Workshop on Artificial Intelligence for Web Search (AAAI '06). Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Semi-supervised distance metric learning for collaborative image retrieval and clustering

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 6, Issue 3
        August 2010
        203 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/1823746
        Issue’s Table of Contents

        Copyright © 2010 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 27 August 2010
        • Accepted: 1 May 2009
        • Revised: 1 April 2009
        • Received: 1 February 2009
        Published in tomm Volume 6, Issue 3

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!