Abstract
Learning-based hashing has been researched extensively in the past few years due to its great potential in fast and accurate similarity search among huge volumes of multimedia data. In this article, we present a novel multimedia hashing framework, called Label Preserving Multimedia Hashing (LPMH) for multimedia similarity search. In LPMH, a general optimization method is used to learn the joint binary codes of multiple media types by explicitly preserving semantic label information. Compared with existing hashing methods which are typically developed under and thus restricted to some specific objective functions, the proposed optimization strategy is not tied to any specific loss function and can easily incorporate bit balance constraints to produce well-balanced binary codes. Specifically, our formulation leads to a set of Binary Integer Programming (BIP) problems that have exact solutions both with and without bit balance constraints. These problems can be solved extremely fast and the solution can easily scale up to large-scale datasets. In the hash function learning stage, the boosted decision trees algorithm is utilized to learn multiple media-specific hash functions that can map heterogeneous data sources into a homogeneous Hamming space for cross-media retrieval. We have comprehensively evaluated the proposed method using a range of large-scale datasets in both single-media and cross-media retrieval tasks. The experimental results demonstrate that LPMH is competitive with state-of-the-art methods in both speed and accuracy.
- Ron Appel, Thomas J. Fuchs, Piotr Dollár, and Pietro Perona. 2013. Quickly boosting decision trees-pruning underachieving features early. In ICML (3). 594--602. Google Scholar
Digital Library
- Artem Babenko and Victor Lempitsky. 2015. Tree quantization for large-scale similarity search and classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4240--4248.Google Scholar
Cross Ref
- Michael M. Bronstein, Alexander M. Bronstein, Fabrice Michel, and Nikos Paragios. 2010. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In CVPR’10. 3594--3601.Google Scholar
- Yue Cao, Mingsheng Long, and Jianmin Wang. 2016. Correlation hashing network for efficient cross-modal retrieval. ArXiv Preprint Arxiv:1602.06697.Google Scholar
- Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. 2017. Collective deep quantization for efficient cross-modal retrieval. AAAI. 3974--3980.Google Scholar
- Yue Cao, Mingsheng Long, Jianmin Wang, and Han Zhu. 2016. Correlation autoencoder hashing for supervised cross-modal search. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. ACM, 197--204. Google Scholar
Digital Library
- Mayur Datar, Nicole Immorlica, Piotr Indyk, and Vahab S. Mirrokni. 2004. Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the 20th Annual Symposium on Computational Geometry. ACM, 253--262. Google Scholar
Digital Library
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). IEEE, 248--255.Google Scholar
Cross Ref
- Guiguang Ding, Yuchen Guo, and Jile Zhou. 2014. Collective matrix factorization hashing for multimodal data. In CVPR’14. 2083--2090. Google Scholar
Digital Library
- K. Ding, B. Fan, C. Huo, S. Xiang, and C. Pan. 2017. Cross-modal hashing via rank-order preserving. IEEE Transactions on Multimedia 19, 3, 571--585. Google Scholar
Digital Library
- Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized product quantization for approximate nearest neighbor search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2946--2953. Google Scholar
Digital Library
- Yunchao Gong and Svetlana Lazebnik. 2011. Iterative quantization: A procrustean approach to learning binary codes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE, 817--824. Google Scholar
Digital Library
- Go Irie, Hiroyuki Arai, and Yukinobu Taniguchi. 2015. Alternating co-quantization for cross-modal hashing. In Proceedings of the IEEE International Conference on Computer Vision. 1886--1894. Google Scholar
Digital Library
- Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 1, 117--128. Google Scholar
Digital Library
- Qing-Yuan Jiang and Wu-Jun Li. 2016. Deep cross-modal hashing. Arxiv Preprint Arxiv:1602.02255.Google Scholar
- Wang-Cheng Kang, Wu-Jun Li, and Zhi-Hua Zhou. 2016. Column sampling based discrete supervised hashing. In AAAI. 1230--1236. Google Scholar
Digital Library
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105. Google Scholar
Digital Library
- Brian Kulis and Trevor Darrell. 2009. Learning to hash with binary reconstructive embeddings. In Advances in Neural Information Processing Systems. 1042--1050. Google Scholar
Digital Library
- Shaishav Kumar and Raghavendra Udupa. 2011. Learning hash functions for cross-view similarity search. In IJCAI, Vol. 22. 1360. Google Scholar
Digital Library
- Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15).Google Scholar
Cross Ref
- Kai Li, Guojun Qi, Jun Ye, and Kien A. Hua. 2016. Cross-modal hashing through ranking subspace learning. In IEEE International Conference on Multimedia and Expo (ICME’16). IEEE, 1--6.Google Scholar
- Kai Li, Guo-Jun Qi, Jun Ye, and Kien A. Hua. 2016. Linear subspace ranking hashing for cross-modal retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 99, 1--1.Google Scholar
- Kai Li, Guo-Jun Qi, Jun Ye, Tuoerhongjiang Yusuph, and Kien A. Hua. 2016. Supervised ranking hash for semantic similarity search. In IEEE International Symposium on Multimedia (ISM’16). IEEE, 551--558.Google Scholar
- Kai Li, Guo-Jun Qi, Jun Ye, Tuoerhongjiang Yusuph, and Kien A. Hua. 2017. Semantic image retrieval with feature space rankings. International Journal of Semantic Computing 11, 02, 171--192.Google Scholar
Cross Ref
- Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton van den Hengel, and David Suter. 2014. Fast supervised hashing with decision trees for high-dimensional data. In CVPR’14. IEEE, 1971--1978. Google Scholar
Digital Library
- Guosheng Lin, Chunhua Shen, David Suter, and Anton van den Hengel. 2013. A general two-step approach to learning-based hashing. In IEEE International Conference on Computer Vision (ICCV’13), IEEE, 2552--2559. Google Scholar
Digital Library
- Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. 2015. Semantics-preserving hashing for cross-view retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3864--3872.Google Scholar
Cross Ref
- Hong Liu Liu, Ji Rongrong, Wu Yongjian, and Hua Gang. Supervised matrix factorization for cross-modality hashing. In IJCAI’16. Google Scholar
Digital Library
- Wei Liu, Cun Mu, Sanjiv Kumar, and Shih-Fu Chang. 2014. Discrete graph hashing. In NIPS’14, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 3419--3427. Google Scholar
Digital Library
- Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. 2012. Supervised hashing with kernels. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2074--2081. Google Scholar
Digital Library
- Wei Liu, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2011. Hashing with graphs. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 1--8. Google Scholar
Digital Library
- Mingsheng Long, Yue Cao, Jianmin Wang, and Philip S. Yu. 2016. Composite correlation quantization for efficient multimodal retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 579--588. Google Scholar
Digital Library
- Jonathan Masci, Michael M. Bronstein, Alexander M. Bronstein, and Jürgen Schmidhuber. 2014. Multimodal similarity-preserving hashing. IEEE Transactions On Pattern Analysis and Machine Intelligence 36, 4, 824--830. Google Scholar
Digital Library
- Mohammad Norouzi and David J. Fleet. 2011. Minimal loss hashing for compact binary codes. In ICML’11. 353--360. Google Scholar
Digital Library
- Mohammad Norouzi, Ali Punjani, and David J. Fleet. 2012. Fast search in hamming space with multi-index hashing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 3108--3115. Google Scholar
Digital Library
- Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. 2015. Supervised discrete hashing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 37--45.Google Scholar
Cross Ref
- Xiaoshuang Shi, Fuyong Xing, Jinzheng Cai, Zizhao Zhang, Yuanpu Xie, and Lin Yang. 2016. Kernel-based supervised discrete hashing for image retrieval. In European Conference on Computer Vision. Springer, 419--433.Google Scholar
Cross Ref
- Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In ACM SIGMOD’13. 785--796. Google Scholar
Digital Library
- Di Wang, Xinbo Gao, Xiumei Wang, and Lihuo He. 2015. Semantic topic multimodal hashing for cross-media retrieval. In Proceedings of the International Joint Conference on Artificial Intelligence. 3890--3896. Google Scholar
Digital Library
- Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral hashing. In Advances in Neural Information Processing Systems. 1753--1760. Google Scholar
Digital Library
- Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, and Jingdong Wang. 2015. Quantized correlation hashing for fast cross-modal search. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. Google Scholar
Digital Library
- Dongqing Zhang and Wu-Jun Li. Large-scale supervised multimodal hashing with semantic correlation maximization. In AAAI’14. Google Scholar
Digital Library
- L. Zhang, Y. Zhang, X. Gu, J. Tang, and Q. Tian. 2014. Scalable similarity search with topology preserving hashing. IEEE Transactions on Image Processing 23, 7, 3025--3039.Google Scholar
Cross Ref
- S. Zhang, J. Li, J. Guo, and B. Zhang. 2016. Scalable discrete supervised hash learning with asymmetric matrix factorization. In IEEE 16th International Conference on Data Mining (ICDM’16). 1347--1352.Google Scholar
- Ting Zhang, Chao Du, and Jingdong Wang. 2014. Composite quantization for approximate nearest neighbor search. In ICML. 838--846. Google Scholar
Digital Library
- Ting Zhang, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. 2015. Sparse composite quantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4548--4556.Google Scholar
Cross Ref
- Ting Zhang and Jingdong Wang. 2016. Collaborative quantization for cross-modal similarity search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2036--2045.Google Scholar
Cross Ref
- Y. Zhang, L. Zhang, and Q. Tian. 2014. A prior-free weighting scheme for binary code ranking. IEEE Transactions on Multimedia 16, 4, 1127--1139. Google Scholar
Digital Library
- Ziming Zhang, Yuting Chen, and Venkatesh Saligrama. 2016. Efficient training of very deep neural networks for supervised hashing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google Scholar
Cross Ref
- Yi Zhen and Dit-Yan Yeung. Co-regularized hashing for multimodal data. In NIPS’12. 1376--1384. Google Scholar
Digital Library
- Yi Zhen and Dit-Yan Yeung. A probabilistic model for multimodal hash function learning. In SIGKDD’12. Google Scholar
Digital Library
- Jile Zhou, Guiguang Ding, and Yuchen Guo. Latent semantic sparse hashing for cross-modal similarity search. In ACM SIGIR’14. 415--424. Google Scholar
Digital Library
Index Terms
Learning Label Preserving Binary Codes for Multimedia Retrieval: A General Approach
Recommendations
Label embedding semantic-guided hashing
Graphical abstractDisplay Omitted
Highlights- Proposed a novel two-step label embedding semantic-guided hashing method.
- ...
AbstractHashing technologies have been widely used for information retrieval tasks due to their efficient retrieval and storage capabilities. Generally, most of the current supervised learning only utilizes labels to construct a binary ...
Semantics-Reconstructing Hashing for Cross-Modal Retrieval
Advances in Knowledge Discovery and Data MiningAbstractRetrieval on Cross-modal data has attracted extensive attention as it enables fast searching across various data sources, such as texts, images and videos. As one of the typical techniques for cross-model searching, hashing methods project ...
Deep Supervised Hashing by Classification for Image Retrieval
Neural Information ProcessingAbstractHashing has been widely used to approximate the nearest neighbor search for image retrieval due to its high computation efficiency and low storage requirement. With the development of deep learning, a series of deep supervised methods were ...






Comments