skip to main content
research-article

Learning Label Preserving Binary Codes for Multimedia Retrieval: A General Approach

Published:20 December 2017Publication History
Skip Abstract Section

Abstract

Learning-based hashing has been researched extensively in the past few years due to its great potential in fast and accurate similarity search among huge volumes of multimedia data. In this article, we present a novel multimedia hashing framework, called Label Preserving Multimedia Hashing (LPMH) for multimedia similarity search. In LPMH, a general optimization method is used to learn the joint binary codes of multiple media types by explicitly preserving semantic label information. Compared with existing hashing methods which are typically developed under and thus restricted to some specific objective functions, the proposed optimization strategy is not tied to any specific loss function and can easily incorporate bit balance constraints to produce well-balanced binary codes. Specifically, our formulation leads to a set of Binary Integer Programming (BIP) problems that have exact solutions both with and without bit balance constraints. These problems can be solved extremely fast and the solution can easily scale up to large-scale datasets. In the hash function learning stage, the boosted decision trees algorithm is utilized to learn multiple media-specific hash functions that can map heterogeneous data sources into a homogeneous Hamming space for cross-media retrieval. We have comprehensively evaluated the proposed method using a range of large-scale datasets in both single-media and cross-media retrieval tasks. The experimental results demonstrate that LPMH is competitive with state-of-the-art methods in both speed and accuracy.

References

  1. Ron Appel, Thomas J. Fuchs, Piotr Dollár, and Pietro Perona. 2013. Quickly boosting decision trees-pruning underachieving features early. In ICML (3). 594--602. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Artem Babenko and Victor Lempitsky. 2015. Tree quantization for large-scale similarity search and classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4240--4248.Google ScholarGoogle ScholarCross RefCross Ref
  3. Michael M. Bronstein, Alexander M. Bronstein, Fabrice Michel, and Nikos Paragios. 2010. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In CVPR’10. 3594--3601.Google ScholarGoogle Scholar
  4. Yue Cao, Mingsheng Long, and Jianmin Wang. 2016. Correlation hashing network for efficient cross-modal retrieval. ArXiv Preprint Arxiv:1602.06697.Google ScholarGoogle Scholar
  5. Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. 2017. Collective deep quantization for efficient cross-modal retrieval. AAAI. 3974--3980.Google ScholarGoogle Scholar
  6. Yue Cao, Mingsheng Long, Jianmin Wang, and Han Zhu. 2016. Correlation autoencoder hashing for supervised cross-modal search. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. ACM, 197--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mayur Datar, Nicole Immorlica, Piotr Indyk, and Vahab S. Mirrokni. 2004. Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the 20th Annual Symposium on Computational Geometry. ACM, 253--262. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). IEEE, 248--255.Google ScholarGoogle ScholarCross RefCross Ref
  9. Guiguang Ding, Yuchen Guo, and Jile Zhou. 2014. Collective matrix factorization hashing for multimodal data. In CVPR’14. 2083--2090. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. K. Ding, B. Fan, C. Huo, S. Xiang, and C. Pan. 2017. Cross-modal hashing via rank-order preserving. IEEE Transactions on Multimedia 19, 3, 571--585. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized product quantization for approximate nearest neighbor search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2946--2953. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Yunchao Gong and Svetlana Lazebnik. 2011. Iterative quantization: A procrustean approach to learning binary codes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE, 817--824. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Go Irie, Hiroyuki Arai, and Yukinobu Taniguchi. 2015. Alternating co-quantization for cross-modal hashing. In Proceedings of the IEEE International Conference on Computer Vision. 1886--1894. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 1, 117--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Qing-Yuan Jiang and Wu-Jun Li. 2016. Deep cross-modal hashing. Arxiv Preprint Arxiv:1602.02255.Google ScholarGoogle Scholar
  16. Wang-Cheng Kang, Wu-Jun Li, and Zhi-Hua Zhou. 2016. Column sampling based discrete supervised hashing. In AAAI. 1230--1236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Brian Kulis and Trevor Darrell. 2009. Learning to hash with binary reconstructive embeddings. In Advances in Neural Information Processing Systems. 1042--1050. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Shaishav Kumar and Raghavendra Udupa. 2011. Learning hash functions for cross-view similarity search. In IJCAI, Vol. 22. 1360. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15).Google ScholarGoogle ScholarCross RefCross Ref
  21. Kai Li, Guojun Qi, Jun Ye, and Kien A. Hua. 2016. Cross-modal hashing through ranking subspace learning. In IEEE International Conference on Multimedia and Expo (ICME’16). IEEE, 1--6.Google ScholarGoogle Scholar
  22. Kai Li, Guo-Jun Qi, Jun Ye, and Kien A. Hua. 2016. Linear subspace ranking hashing for cross-modal retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 99, 1--1.Google ScholarGoogle Scholar
  23. Kai Li, Guo-Jun Qi, Jun Ye, Tuoerhongjiang Yusuph, and Kien A. Hua. 2016. Supervised ranking hash for semantic similarity search. In IEEE International Symposium on Multimedia (ISM’16). IEEE, 551--558.Google ScholarGoogle Scholar
  24. Kai Li, Guo-Jun Qi, Jun Ye, Tuoerhongjiang Yusuph, and Kien A. Hua. 2017. Semantic image retrieval with feature space rankings. International Journal of Semantic Computing 11, 02, 171--192.Google ScholarGoogle ScholarCross RefCross Ref
  25. Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton van den Hengel, and David Suter. 2014. Fast supervised hashing with decision trees for high-dimensional data. In CVPR’14. IEEE, 1971--1978. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Guosheng Lin, Chunhua Shen, David Suter, and Anton van den Hengel. 2013. A general two-step approach to learning-based hashing. In IEEE International Conference on Computer Vision (ICCV’13), IEEE, 2552--2559. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. 2015. Semantics-preserving hashing for cross-view retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3864--3872.Google ScholarGoogle ScholarCross RefCross Ref
  28. Hong Liu Liu, Ji Rongrong, Wu Yongjian, and Hua Gang. Supervised matrix factorization for cross-modality hashing. In IJCAI’16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Wei Liu, Cun Mu, Sanjiv Kumar, and Shih-Fu Chang. 2014. Discrete graph hashing. In NIPS’14, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 3419--3427. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. 2012. Supervised hashing with kernels. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2074--2081. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Wei Liu, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2011. Hashing with graphs. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mingsheng Long, Yue Cao, Jianmin Wang, and Philip S. Yu. 2016. Composite correlation quantization for efficient multimodal retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 579--588. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Jonathan Masci, Michael M. Bronstein, Alexander M. Bronstein, and Jürgen Schmidhuber. 2014. Multimodal similarity-preserving hashing. IEEE Transactions On Pattern Analysis and Machine Intelligence 36, 4, 824--830. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Mohammad Norouzi and David J. Fleet. 2011. Minimal loss hashing for compact binary codes. In ICML’11. 353--360. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Mohammad Norouzi, Ali Punjani, and David J. Fleet. 2012. Fast search in hamming space with multi-index hashing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 3108--3115. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. 2015. Supervised discrete hashing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 37--45.Google ScholarGoogle ScholarCross RefCross Ref
  37. Xiaoshuang Shi, Fuyong Xing, Jinzheng Cai, Zizhao Zhang, Yuanpu Xie, and Lin Yang. 2016. Kernel-based supervised discrete hashing for image retrieval. In European Conference on Computer Vision. Springer, 419--433.Google ScholarGoogle ScholarCross RefCross Ref
  38. Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In ACM SIGMOD’13. 785--796. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Di Wang, Xinbo Gao, Xiumei Wang, and Lihuo He. 2015. Semantic topic multimodal hashing for cross-media retrieval. In Proceedings of the International Joint Conference on Artificial Intelligence. 3890--3896. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral hashing. In Advances in Neural Information Processing Systems. 1753--1760. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, and Jingdong Wang. 2015. Quantized correlation hashing for fast cross-modal search. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Dongqing Zhang and Wu-Jun Li. Large-scale supervised multimodal hashing with semantic correlation maximization. In AAAI’14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. L. Zhang, Y. Zhang, X. Gu, J. Tang, and Q. Tian. 2014. Scalable similarity search with topology preserving hashing. IEEE Transactions on Image Processing 23, 7, 3025--3039.Google ScholarGoogle ScholarCross RefCross Ref
  44. S. Zhang, J. Li, J. Guo, and B. Zhang. 2016. Scalable discrete supervised hash learning with asymmetric matrix factorization. In IEEE 16th International Conference on Data Mining (ICDM’16). 1347--1352.Google ScholarGoogle Scholar
  45. Ting Zhang, Chao Du, and Jingdong Wang. 2014. Composite quantization for approximate nearest neighbor search. In ICML. 838--846. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Ting Zhang, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. 2015. Sparse composite quantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4548--4556.Google ScholarGoogle ScholarCross RefCross Ref
  47. Ting Zhang and Jingdong Wang. 2016. Collaborative quantization for cross-modal similarity search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2036--2045.Google ScholarGoogle ScholarCross RefCross Ref
  48. Y. Zhang, L. Zhang, and Q. Tian. 2014. A prior-free weighting scheme for binary code ranking. IEEE Transactions on Multimedia 16, 4, 1127--1139. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Ziming Zhang, Yuting Chen, and Venkatesh Saligrama. 2016. Efficient training of very deep neural networks for supervised hashing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google ScholarGoogle ScholarCross RefCross Ref
  50. Yi Zhen and Dit-Yan Yeung. Co-regularized hashing for multimodal data. In NIPS’12. 1376--1384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Yi Zhen and Dit-Yan Yeung. A probabilistic model for multimodal hash function learning. In SIGKDD’12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Jile Zhou, Guiguang Ding, and Yuchen Guo. Latent semantic sparse hashing for cross-modal similarity search. In ACM SIGIR’14. 415--424. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Learning Label Preserving Binary Codes for Multimedia Retrieval: A General Approach

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!