Abstract
Deep hashing has great potential in large-scale visual similarity search due to its preferable efficiency in storage and computation. Technically, deep hashing for visual similarity search inherits the powerful representation capability of deep neural networks, and it encodes visual features into compact binary codes by preserving representative semantic visual features. Works in this field mainly focus on building the relationship between the visual and objective hash spaces, while they seldom study the triadic cross-domain semantic knowledge transfer among visual, semantic, and hashing spaces, leading to a serious semantic ignorance problem during space transformation. In this article, we propose a novel deep tripartite semantically interactive hashing framework, dubbed Semantically Cycle-consistent Hashing Networks (SCHNs), for discriminative hash code learning. Particularly, we construct a flexible semantic space and a transitive latent space, in conjunction with the visual space, to jointly deduce the privileged discriminative hash space. Specifically, a new semantic space is conceived to strengthen the flexibility and completeness of categories in the semantic feature inference phase. At the same time, a transitive latent space is formulated to explore and uncover the shared semantic interactivity embedded in visual and semantic features. Moreover, to further ensure semantic consistency across multiple spaces, we propose to build a cyclic adversarial learning module to preserve and keep their semantic concurrence during space transformation. Notably, our SCHN, for the first time, establishes the cyclic principle of deep semantic-preserving hashing by adaptive semantic parsing across different spaces in a single-modal visual similarity search. In addition, the entire learning framework is jointly optimized in an end-to-end manner. Extensive experiments performed on diverse large-scale datasets evidence the superiority of our method against other state-of-the-art deep hashing algorithms. The source codes of this article are available at https://github.com/JalinWang/SCHN.
- [1] . 2017. Hashnet: Deep learning to hash by continuation. In Proceedings of the IEEE International Conference on Computer Vision. 5608–5617.Google Scholar
Cross Ref
- [2] . 2021. Deep category-level and regularized hashing with global semantic similarity learning. IEEE Transactions on Cybernetics 51, 12 (2021), 6240–6252.Google Scholar
- [3] . 2009. NUS-WIDE: A real-world web image database from national university of singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval. ACM, 1–9.Google Scholar
Digital Library
- [4] . 2018. Triplet-based deep hashing network for cross-modal retrieval. IEEE Transactions on Image Processing 27, 8 (2018), 3893–3903.Google Scholar
Cross Ref
- [5] . 2019. Two-stream deep hashing with class-specific centers for supervised image search. IEEE Transactions on Neural Networks and Learning Systems 31, 6 (2019), 2189–2201.Google Scholar
Cross Ref
- [6] . 2019. Beyond product quantization: Deep progressive quantization for image retrieval. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 723–729.Google Scholar
Cross Ref
- [7] . 1999. Similarity search in high dimensions via hashing. In Proceedings of 25th International Conference on Very Large Data Bases. 518–529.Google Scholar
Digital Library
- [8] . 2013. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 12 (2013), 2916–2929.Google Scholar
Digital Library
- [9] . 2014. Generative adversarial nets. Advances in Neural Information Processing Systems 27 (2014), 1–9.Google Scholar
- [10] . 2008. The MIR flickr retrieval evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. 39–43.Google Scholar
Digital Library
- [11] . 2017. SuBiC: A supervised, structured binary code for image search. In Proceedings of the IEEE International Conference on Computer Vision. 833–842.Google Scholar
Cross Ref
- [12] . 2015. Scalable graph hashing with feature transformation. In Proceedings of the International Joint Conference on Artificial Intelligence. 2248–2254.Google Scholar
- [13] . 2016. Column sampling based discrete supervised hashing. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 1230–1236.Google Scholar
Cross Ref
- [14] . 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- [15] . 2009. Learning Multiple Layers of Features from Tiny Images.
Technical Report . University of Toronto.Google Scholar - [16] . 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012), 1097–1105.Google Scholar
Digital Library
- [17] . 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3270–3278.Google Scholar
Cross Ref
- [18] . 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3270–3278.Google Scholar
Cross Ref
- [19] . 2020. A general framework for deep supervised discrete hashing. International Journal of Computer Vision 128, 8 (2020), 2204–2222.Google Scholar
Digital Library
- [20] . 2016. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the International Joint Conference on Artificial Intelligence. 1711–1717.Google Scholar
- [21] . 2015. Deep learning of binary hash codes for fast image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 27–35.Google Scholar
Cross Ref
- [22] . 2016. Hierarchical clustering multi-task learning for joint human action grouping and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 1 (2016), 102–114.Google Scholar
Digital Library
- [23] . 2020. Adaptively clustering-driven learning for visual relationship detection. IEEE Transactions on Multimedia 23 (2020), 4515–4525.Google Scholar
Cross Ref
- [24] . 2018. Ordinal constraint binary coding for approximate nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 4 (2018), 941–955.Google Scholar
Digital Library
- [25] . 2022. Recent advances in monocular 2d and 3d human pose estimation: A deep learning perspective. Computing Surveys (2022).
DOI: Google ScholarDigital Library
- [26] . 2012. Supervised hashing with kernels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2074–2081.Google Scholar
Digital Library
- [27] . 2011. Hashing with graphs. In Proceedings of the 28th International Conference on Machine Learning. 1–8.Google Scholar
Digital Library
- [28] . 2016. Large-scale vehicle re-identification in urban surveillance videos. In 2016 IEEE International Conference on Multimedia and Expo. IEEE, 1–6.Google Scholar
Cross Ref
- [29] . 2017. Provid: Progressive and multimodal vehicle reidentification for large-scale urban surveillance. IEEE Transactions on Multimedia 20, 3 (2017), 645–658.Google Scholar
Digital Library
- [30] . 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, (November2008), 2579–2605.Google Scholar
- [31] . 2015. Semi-supervised hashing with semantic confidence for large scale visual search. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 53–62.Google Scholar
Digital Library
- [32] . 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32 (2019), 8026–8037.Google Scholar
- [33] . 2020. Deep reinforcement learning for image hashing. IEEE Transactions on Multimedia 22, 8 (2020), 2061–2073.Google Scholar
Cross Ref
- [34] . 2017. Deep semantic hashing with generative adversarial networks. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 225–234.Google Scholar
Digital Library
- [35] . 2016. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision. Springer, 525–542.Google Scholar
Cross Ref
- [36] . 2017. Classification by retrieval: Binarizing data and classifiers. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 595–604.Google Scholar
Digital Library
- [37] . 2015. Supervised discrete hashing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 37–45.Google Scholar
Cross Ref
- [38] . 2021. Transductive semi-supervised deep hashing. IEEE Transactions on Neural Networks and Learning Systems (2021), 1–14.
DOI: Google ScholarCross Ref
- [39] . 2018. A survey on learning to hash. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 4 (2018), 769–790.Google Scholar
Cross Ref
- [40] . 2020. Deep collaborative discrete hashing with semantic-invariant structure construction. IEEE Transactions on Multimedia 23 (2020), 1274–1286.Google Scholar
Digital Library
- [41] . 2009. Spectral hashing. In Proceedings of the Neural Information Processing Systems. 1753–1760.Google Scholar
- [42] . 2014. Supervised hashing for image retrieval via image representation learning. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2156–2162.Google Scholar
Cross Ref
- [43] . 2019. Multi-level policy and reward-based deep reinforcement learning framework for image captioning. IEEE Transactions on Multimedia 22, 5 (2019), 1372–1383.Google Scholar
Cross Ref
- [44] . 2016. Discrete collaborative filtering. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 325–334.Google Scholar
Digital Library
- [45] . 2014. Supervised hashing with latent factor models. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 173–182.Google Scholar
Digital Library
- [46] . 2019. Scalable supervised asymmetric hashing with semantic and latent factor embedding. IEEE Transactions on Image Processing 28, 10 (2019), 4803–4818.Google Scholar
Cross Ref
- [47] . 2021. Inductive structure consistent hashing via flexible semantic calibration. IEEE Transactions on Neural Networks and Learning Systems 32, 10 (2021), 4514–4528.Google Scholar
Cross Ref
- [48] . 2019. Binary multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 7 (2019), 1774–1782.Google Scholar
Cross Ref
- [49] . 2022. Modality-invariant asymmetric networks for cross-modal hashing. IEEE Transactions on Knowledge and Data Engineering (2022).
DOI: Google ScholarDigital Library
- [50] . 2021. Towards discriminative visual search via semantically cycle-consistent hashing networks. In ACM Multimedia Asia. 1–7.Google Scholar
- [51] . 2015. Scalable person re-identification: A benchmark. In Proceedings of the IEEE International Conference on Computer Vision. 1116–1124.Google Scholar
Digital Library
- [52] . 2017. Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. In Proceedings of the IEEE International Conference on Computer Vision. 3754–3762.Google Scholar
Cross Ref
- [53] . 2016. Deep hashing network for efficient similarity retrieval. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2415–2421.Google Scholar
Cross Ref
Index Terms
Discriminative Visual Similarity Search with Semantically Cycle-consistent Hashing Networks
Recommendations
Towards Discriminative Visual Search via Semantically Cycle-consistent Hashing Networks
MMAsia '21: ACM Multimedia AsiaDeep hashing has shown great potentials in large-scale visual similarity search due to preferable storage and computation efficiency. Typically, deep hashing encodes visual features into compact binary codes by preserving representative semantic visual ...
Deep Semantic Asymmetric Hashing
Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural ComputationAbstractDeep hashing, which combines binary codes learning and convolutional neural network, has achieved promising performance for highly efficient image retrieval. Asymmetric deep hashing methods, which treat query points and database points in an ...
Sparse Matrix Based Hashing for Approximate Nearest Neighbor Search
PCM 2016: 17th Pacific-Rim Conference on Advances in Multimedia Information Processing - Volume 9916Binary hashing has been widely studied for approximate nearest neighbor ANN search with its compact representation and efficient comparison. Many existing hashing methods aim at improving the accuracy of ANN search, but ignore the complexity of ...






Comments