skip to main content
research-article

Unsupervised Similarity Learning through Rank Correlation and kNN Sets

Authors Info & Claims
Published:23 October 2018Publication History
Skip Abstract Section

Abstract

The increasing amount of multimedia data collections available today evinces the pressing need for methods capable of indexing and retrieving this content. Despite the continuous advances in multimedia features and representation models, to establish an effective measure for comparing different multimedia objects still remains a challenging task. While supervised and semi-supervised techniques made relevant advances on similarity learning tasks, scenarios where labeled data are non-existent require different strategies. In such situations, unsupervised learning has been established as a promising solution, capable of considering the contextual information and the dataset structure for computing new similarity/dissimilarity measures. This article extends a recent unsupervised learning algorithm that uses an iterative re-ranking strategy to take advantage of different k-Nearest Neighbors (kNN) sets and rank correlation measures. Two novel approaches are proposed for computing the kNN sets and their corresponding top-k lists. The proposed approaches were validated in conjunction with various rank correlation measures, yielding superior effectiveness results in comparison with previous works. In addition, we also evaluate the ability of the method in considering different multimedia objects, conducting an extensive experimental evaluation on various image and video datasets.

References

  1. J. Almeida, N. J. Leite, and R. S. Torres. 2011. Comparison of video sequences with histograms of motion patterns. In Proceedings of the IEEE International Conference on Image Processing (ICIP’11). 3673--367Google ScholarGoogle Scholar
  2. J. Almeida, D. C. G. Pedronette, and O. A. B. Penatti. 2014. Unsupervised manifold learning for video genre retrieval. In Proceedings of the Iberoamerican Congress on Pattern Recognition (CIARP’14). 604--612.Google ScholarGoogle ScholarCross RefCross Ref
  3. J. Almeida, A. Rocha, R. S. Torres, and S. Goldenstein. 2008. Making colors worth more than a thousand words. In Proceedings of the ACM Symposium on Applied Computing (ACM-SAC’08). 1180--1186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Nafiz Arica and Fatos T. Yarman Vural. 2003. BAS: A perceptual shape descriptor based on the beam angle statistics. Pattern Recogn. Lett. 24, 9-10 (2003), 1627--1639. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Bai and X. Bai. 2016. Sparse contextual activation for efficient visual re-ranking. IEEE Trans. Image Processing 25, 3 (2016), 1056--1069.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Song Bai, Xiang Bai, Qi Tian, and Longin Jan Latecki. 2017. Regularized diffusion process for visual retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’17). 3967--3973.Google ScholarGoogle ScholarCross RefCross Ref
  7. S. Bai, Z. Zhou, J. Wang, X. Bai, L. J. Latecki, and Q. Tian. 2017. Ensemble diffusion for retrieval. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 774--783.Google ScholarGoogle Scholar
  8. Xiang Bai, Song Bai, and Xinggang Wang. 2015. Beyond diffusion process: Neighbor set similarity for fast re-ranking. Inf. Sci. 325 (2015), 342--354. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Xiang Bai, Bo Wang, Xinggang Wang, Wenyu Liu, and Zhuowen Tu. 2010. Co-transduction for shape retrieval. In Proceedings of the European Conference on Computer Vision (ECCV’10), Vol. 3. 328--341. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Christian Beecks. 2013. Distance-based Similarity Models for Content-based Multimedia Retrieval. Ph.D. Dissertation. Fakultat fur Mathematik, Informatik und Naturwissenschaften, RWTH Aachen University.Google ScholarGoogle Scholar
  11. C. Beecks, M. S. Uysal, and T. Seidl. 2010. A comparative study of similarity measures for content-based multimedia retrieval. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME’10). 1552--1557.Google ScholarGoogle Scholar
  12. Serge Belongie, Jitendra Malik, and Jan Puzicha. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4 (2002), 509--522. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Phil Brodatz. 1966. Textures: A Photographic Album for Artists and Designers. Dover Publications, Mineola, NY.Google ScholarGoogle Scholar
  14. Yanzhi Chen, Xi Li, Anthony Dick, and Rhys Hill. 2014. Ranking consistency for image matching and object retrieval. Pattern Recogn. 47, 3 (2014), 1349--1360. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Joost Van de Weijer and Cordelia Schmid. 2006. Coloring local feature extraction. In Proceedings of the European Conference on Computer Vision (ECCV’06). Lecture Notes in Computer Science, Vol. 3952. 334--348. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. C. Deng, R. Ji, W. Liu, D. Tao, and X. Gao. 2013. Visual reranking through weakly supervised multi-graph learning. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’13). 2600--2607. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. M. Donoser and H. Bischof. 2013. Diffusion processes for retrieval revisited. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’13). 1320--1327. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. L. A. Duarte, O. A. B. Penatti, and J. Almeida. 2016. Bag of genres for video retrieval. In Proceedings of the Conference on Graphics, Patterns and Images (SIBGRAPI’16). 257--264.Google ScholarGoogle Scholar
  19. Ronald Fagin, Ravi Kumar, and D. Sivakumar. 2003. Comparing top k lists. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms (SODA’03). 28--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Leo A. Goodman and William H. Kruskal. 1954. Measures of association for cross classifications. J. Am. Stat. Assoc. 49, 268 (1954), 732--764.Google ScholarGoogle Scholar
  21. Raghuraman Gopalan, Pavan Turaga, and Rama Chellappa. 2010. Articulation-invariant representation of non-planar shapes. In Proceedings of the European Conference on Computer Vision (ECCV’10), Vol. 3. 286--299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Jing Huang, S. Ravi Kumar, Mandar Mitra, Wei-Jing Zhu, and Ramin Zabih. 1997. Image indexing using color correlograms. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’97). 762--768. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Hervé Jegou, Cordelia Schmid, Hedi Harzallah, and Jakob Verbeek. 2010. Accurate image search using the contextual dissimilarity measure. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1 (2010), 2--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM International Conference on Multimedia (ACM-MM’14). 675--678. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jiayan Jiang, Bo Wang, and Zhuowen Tu. 2011. Unsupervised metric learning by self-smoothing operator. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’11). 794--801. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Peter Kontschieder, Michael Donoser, and Horst Bischof. 2009. Beyond pairwise shape similarity analysis. In Proceedings of the Asian Conference on Computer Vision (ACCV’09). 655--666. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Vassili Kovalev and Stephan Volmer. 1998. Color co-occurence descriptors for querying-by-example. In Proceedings of the International Conference on MultiMedia Modeling (MMM’98). 32--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Longin Jan Latecki, Rolf Lakmper, and Ulrich Eckhardt. 2000. Shape descriptors for non-rigid shapes with a single closed contour. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’00). 424--429.Google ScholarGoogle ScholarCross RefCross Ref
  29. Haibin Ling and David W. Jacobs. 2007. Shape classification using the inner-distance. IEEE Trans. Pattern Anal. Mach. Intell. 29, 2 (2007), 286--299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Haibin Ling, Xingwei Yang, and Longin Jan Latecki. 2010. Balancing deformability and discriminability for shape matching. In Proceedings of the European Conference on Computer Vision (ECCV’10), Vol. 3. 411--424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. D. G. Lowe. 1999. Object recognition from local scale-invariant features. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’99). 1150--1157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. D. Nistér and H. Stewénius. 2006. Scalable recognition with a vocabulary tree. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 2. 2161--2168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Timo Ojala, Matti Pietikäinen, and Topi Mäenpää. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 7 (2002), 971--987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. César Yugo Okada, Daniel Carlos Guimarães Pedronette, and Ricardo da S. Torres. 2015. Unsupervised distance learning by rank correlation measures for image retrieval. In Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR’15). 331--338. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Daniel Carlos Guimarães Pedronette, Jurandy Almeida, and Ricardo da S. Torres. 2014. A scalable re-ranking method for content-based image retrieval. Inf. Sci. 265, 1 (2014), 91--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. D. C. G. Pedronette, J. Almeida, and R. S. Torres. 2016. A graph-based ranked-list model for unsupervised distance learning on shape retrieval. Pattern Recogn. Lett. 83, Part 3 (2016), 357--367. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Daniel Carlos Guimarães Pedronette, Otavio A. B. Penatti, and Ricardo da S. Torres. 2014. Unsupervised manifold learning using reciprocal kNN graphs in image re-ranking and rank aggregation tasks. Image Vis. Comput. 32, 2 (2014), 120--130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Daniel Carlos Guimarães Pedronette and Ricardo da S. Torres. 2010. Shape retrieval using contour features and distance optmization. In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP’10), Vol. 1. 197--202.Google ScholarGoogle Scholar
  39. Daniel Carlos Guimarães Pedronette and Ricardo da S. Torres. 2012. Exploiting pairwise recommendation and clustering strategies for image re-ranking. Inf. Sci. 207 (2012), 19--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Daniel Carlos Guimarães Pedronette and Ricardo da S. Torres. 2013. Image re-ranking and rank aggregation based on similarity of ranked lists. Pattern Recogn. 46, 8 (2013), 2350--2360. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Otávio A. B. Penatti, Eduardo Valle, and Ricardo da S. Torres. 2012. Comparative study of global color and texture descriptors for web image retrieval. J. Vis. Commun. Image Represent. 23, 2 (2012), 359--380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Danfeng Qin, S. Gammeter, L. Bossard, T. Quack, and L. van Gool. 2011. Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’11). 777--784. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Renato O. Stehling, Mario A. Nascimento, and Alexandre X. Falcão. 2002. A compact and efficient image retrieval approach based on border/interior pixel classification. In Proceedings of the ACM International Conference on Information and Knowledge Management (CIKM’02). 102--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Michael J. Swain and Dana H. Ballard. 1991. Color indexing. Int. J. Comput. Vis. 7, 1 (1991), 11--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Bo Tao and Bradley W. Dickinson. 2000. Texture recognition and image retrieval using gradient indexing. J. Vis. Commun. Image Represent. 11, 3 (2000), 327--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Bart Thomee and Michael S. Lew. 2012. Interactive search in image retrieval: A survey. Int. J. Multimedia Inf. Retriev. 1, 2 (2012), 71--86.Google ScholarGoogle ScholarCross RefCross Ref
  47. Ricardo da S. Torres and Alexandre X. Falcão. 2007. Contour salience descriptors for effective image retrieval and analysis. Image Vis. Comput. 25, 1 (2007), 3--13.Google ScholarGoogle ScholarCross RefCross Ref
  48. L. P. Valem, D. C. G. Pedronette, R. S. Torres, E. Borin, and J. Almeida. 2015. Effective, efficient, and scalable unsupervised distance learning in image retrieval tasks. In Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR’15). 51--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Ji Wan, Dayong Wang, Steven Chu Hong Hoi, Pengcheng Wu, Jianke Zhu, Yongdong Zhang, and Jintao Li. 2014. Deep learning for content-based image retrieval: A comprehensive study. In Proceedings of the ACM International Conference on Multimedia (ACM-MM’14). 157--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Bo Wang, Jiayan Jiang, Wei Wang, Zhi-Hua Zhou, and Zhuowen Tu. 2012. Unsupervised metric fusion by cross diffusion. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’12). 3013--3020.Google ScholarGoogle ScholarCross RefCross Ref
  51. Jingyan Wang, Yongping Li, Xiang Bai, Ying Zhang, Chao Wang, and Ning Tang. 2011. Learning context-sensitive similarity by shortest path propagation. Pattern Recogn. 44, 10-11 (2011), 2367--2374. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Xiaoyu Wang, Ming Yang, T. Cour, Shenghuo Zhu, Kai Yu, and T. X. Han. 2011. Contextual weighting for vocabulary tree based image retrieval. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’11). 209--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. William Webber, Alistair Moffat, and Justin Zobel. 2010. A similarity measure for indefinite rankings. ACM Trans. Inf. Syst. 28, 4, Article 20 (2010), 38 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Lingxi Xie, Richang Hong, Bo Zhang, and Qi Tian. 2015. Image classification and retrieval are ONE. In Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR’15). 3--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Xingwei Yang, Xiang Bai, Longin Jan Latecki, and Zhuowen Tu. 2008. Improving shape retrieval by learning graph transduction. In Proceedings of the European Conference on Computer Vision (ECCV’08), Vol. 4. 788--801.Google ScholarGoogle ScholarCross RefCross Ref
  56. Xingwei Yang, Suzan Koknar-Tezel, and Longin Jan Latecki. 2009. Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’09). 357--364.Google ScholarGoogle ScholarCross RefCross Ref
  57. Xingwei Yang, L. Prasad, and L. J. Latecki. 2013. Affinity learning with diffusion on tensor product graph. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1 (2013), 28--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Shaoting Zhang, Ming Yang, T. Cour, Kai Yu, and D. N. Metaxas. 2015. Query specific rank fusion for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 37, 4 (2015), 803--815.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Liang Zheng, Shengjin Wang, Lu Tian, Fei He, Ziqiong Liu, and Qi Tian. 2015. Query-adaptive late fusion for image search and person re-identification. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’15).Google ScholarGoogle ScholarCross RefCross Ref
  60. Liang Zheng, Shengjin Wang, and Qi Tian. 2014. Lp-norm IDF for scalable image retrieval. IEEE Trans. Image Process. 23, 8 (2014), 3604--3617.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Unsupervised Similarity Learning through Rank Correlation and kNN Sets

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 4
        Special Section on Deep Learning for Intelligent Multimedia Analytics
        November 2018
        221 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3282485
        Issue’s Table of Contents

        Copyright © 2018 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 23 October 2018
        • Accepted: 1 July 2018
        • Revised: 1 March 2018
        • Received: 1 September 2017
        Published in tomm Volume 14, Issue 4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!