skip to main content
research-article

Deep Scalable Supervised Quantization by Self-Organizing Map

Authors Info & Claims
Published:20 August 2019Publication History
Skip Abstract Section

Abstract

Approximate Nearest Neighbor (ANN) search is an important research topic in multimedia and computer vision fields. In this article, we propose a new deep supervised quantization method by Self-Organizing Map to address this problem. Our method integrates the Convolutional Neural Networks and Self-Organizing Map into a unified deep architecture. The overall training objective optimizes supervised quantization loss as well as classification loss. With the supervised quantization objective, we minimize the differences on the maps between similar image pairs and maximize the differences on the maps between dissimilar image pairs. By optimization, the deep architecture can simultaneously extract deep features and quantize the features into suitable nodes in self-organizing map. To make the proposed deep supervised quantization method scalable for large datasets, instead of constructing a larger self-organizing map, we propose to divide the input space into several subspaces and construct self-organizing map in each subspace. The self-organizing maps in all the subspaces implicitly construct a large self-organizing map, which costs less memory and training time than directly constructing a self-organizing map with equal size. The experiments on several public standard datasets prove the superiority of our approaches over the existing ANN search methods. Besides, as a by-product, our deep architecture can be directly applied to visualization with little modification, and promising performance is demonstrated in the experiments.

References

  1. Yue Cao, Bin Liu, Mingsheng Long, Jianmin Wang, and MOE Kliss. 2018. HashGAN: Deep learning to hash with pair conditional wasserstein GAN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1287--1296.Google ScholarGoogle ScholarCross RefCross Ref
  2. Yue Cao, Mingsheng Long, Bin Liu, Jianmin Wang, and MOE Kliss. 2018. Deep cauchy hashing for hamming space retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1229--1237.Google ScholarGoogle ScholarCross RefCross Ref
  3. Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, and Qingfu Wen. 2016. Deep quantization network for efficient image retrieval. In Association for the Advancement of Artificial Intelligence. 3457--3463. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the devil in the details: Delving deep into convolutional nets. In Proceedings of the Association for the Advancement of Artificial Intelligence (2014).Google ScholarGoogle ScholarCross RefCross Ref
  5. Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: A real-world web image database from National University of Singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval. 48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Thanh-Toan Do, Dang-Khoa Le Tan, Trung T. Pham, and Ngai-Man Cheung. 2017. Simultaneous feature aggregating and hashing for large-scale image search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6618--6627.Google ScholarGoogle ScholarCross RefCross Ref
  7. Yueqi Duan, Jiwen Lu, Ziwei Wang, Jianjiang Feng, and Jie Zhou. 2017. Learning deep binary descriptor with multi-quantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1183--1192.Google ScholarGoogle ScholarCross RefCross Ref
  8. Kamran Ghasedi Dizaji, Feng Zheng, Najmeh Sadoughi, Yanhua Yang, Cheng Deng, and Heng Huang. 2018. Unsupervised deep generative adversarial hashing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3664--3673.Google ScholarGoogle ScholarCross RefCross Ref
  9. Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35, 12 (2013), 2916--2929. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Yen-Chang Hsu and Zsolt Kira. 2015. Neural network-based clustering using pairwise constraints. arXiv preprint arXiv:1511.06321 (2015).Google ScholarGoogle Scholar
  11. Yen-Chang Hsu, Zhaoyang Lv, and Zsolt Kira. 2016. Deep image category discovery using a transferred similarity function. arXiv preprint arXiv:1612.01253 (2016).Google ScholarGoogle Scholar
  12. Mengqiu Hu, Yang Yang, Fumin Shen, Ning Xie, Richang Hong, and Heng Tao Shen. 2019. Collective reconstructive embeddings for cross-modal hashing. IEEE Trans. Image Process. 28, 6 (2019), 2770--2784.Google ScholarGoogle ScholarCross RefCross Ref
  13. Mengqiu Hu, Yang Yang, Fumin Shen, Ning Xie, and Heng Tao Shen. 2018. Hashing with angular reconstructive embeddings. IEEE Trans. Image Process. 27, 2 (2018), 545--555.Google ScholarGoogle ScholarCross RefCross Ref
  14. Himalaya Jain, Joaquin Zepeda, Patrick Pérez, and Rémi Gribonval. 2017. SUBIC: A supervised, structured binary code for image search. In Proceedings of the IEEE International Conference on Computer Vision, Vol. 1. 3.Google ScholarGoogle ScholarCross RefCross Ref
  15. Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1 (2011), 117--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Qing-Yuan Jiang and Wu-Jun Li. 2018. Asymmetric deep supervised hashing. In Proceedings of the Association for the Advancement of Artificial Intelligence (2018).Google ScholarGoogle Scholar
  17. Yu-Gang Jiang, Jun Wang, Xiangyang Xue, and Shih-Fu Chang. 2013. Query-adaptive image search with hash codes. IEEE Trans. Multimedia 15, 2 (2013), 442--453. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Teuvo Kohonen. 1982. Self-organized formation of topologically correct feature maps. Biol. Cybernet. 43, 1 (1982), 59--69.Google ScholarGoogle ScholarCross RefCross Ref
  19. Teuvo Kohonen and Timo Honkela. 2007. Kohonen network. Scholarpedia 2, 1 (2007), 1568.Google ScholarGoogle Scholar
  20. Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. (2009). Technical Report.Google ScholarGoogle Scholar
  21. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3270--3278.Google ScholarGoogle ScholarCross RefCross Ref
  23. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.Google ScholarGoogle ScholarCross RefCross Ref
  24. Qi Li, Zhenan Sun, Ran He, and Tieniu Tan. 2017. Deep supervised discrete hashing. In Advances in Neural Information Processing Systems. 2482--2491. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2016. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the International Joint Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Renjie Liao, Alex Schwing, Richard Zemel, and Raquel Urtasun. 2016. Learning deep parsimonious representations. In Advances in Neural Information Processing Systems. 5076--5084. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2064--2072.Google ScholarGoogle ScholarCross RefCross Ref
  28. Zhen Liu, Houqiang Li, Wengang Zhou, Ruizhen Zhao, and Qi Tian. 2014. Contextual hashing for large-scale image search. IEEE Trans. Image Process. 23, 4 (2014), 1606--1614. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jiwen Lu, Venice Erin Liong, and Jie Zhou. 2017. Deep hashing for scalable image search. IEEE Trans. Image Process. 26, 5 (2017), 2352--2367. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Yadan Luo, Yang Yang, Fumin Shen, Zi Huang, Pan Zhou, and Heng Tao Shen. 2018. Robust discrete code modeling for supervised hashing. Pattern Recogn. 75, 1 (2018), 128--135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Mohammad Norouzi and David J. Fleet. 2013. Cartesian k-means. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3017--3024. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. 2013. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013).Google ScholarGoogle Scholar
  33. Fumin Shen, Xin Gao, Li Liu, Yang Yang, and Heng Tao Shen. 2017. Deep asymmetric pairwise hashing. In Proceedings of the ACM International Conference on Multimedia. ACM, 1522--1530. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. 2018. Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE Trans. Pattern Anal. Mach. Intell. 40, 12 (2018), 3034--3044.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Andrea Vedaldi and Karel Lenc. 2015. Matconvnet: Convolutional neural networks for matlab. In Proceedings of the ACM International Conference on Multimedia. ACM, New York, NY, 689--692. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Daixin Wang, Peng Cui, Mingdong Ou, and Wenwu Zhu. 2015. Learning compact hash codes for multimodal representations using orthogonal deep structure. IEEE Trans. Multimedia 17, 9 (2015), 1404--1416.Google ScholarGoogle ScholarCross RefCross Ref
  37. Jianfeng Wang, Jingdong Wang, Nenghai Yu, and Shipeng Li. 2013. Order preserving hashing for approximate nearest neighbor search. In Proceedings of the ACM International Conference on Multimedia. 133--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Min Wang, Wengang Zhou, Qi Tian, and Houqiang Li. 2018. A general framework for linear distance preserving hashing. IEEE Trans. Image Process. 27, 2 (2018), 907--922.Google ScholarGoogle ScholarCross RefCross Ref
  39. Min Wang, Wengang Zhou, Qi Tian, Junfu Pu, and Houqiang Li. 2017. Deep supervised quantization by self-organizing map. In Proceedings of the ACM International Conference on Multimedia. ACM, 1707--1715. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Min Wang, Wengang Zhou, Qi Tian, Zhengjun Zha, and Houqiang Li. 2016. Linear distance preserving pseudo-supervised and unsupervised hashing. In Proceedings of the ACM International Conference on Multimedia. ACM, 1257--1266. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Xiaojuan Wang, Ting Zhang, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. 2016. Supervised quantization for similarity search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018--2026.Google ScholarGoogle ScholarCross RefCross Ref
  42. Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral hashing. In Advances in Neural Information Processing Systems. 1753--1760. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised hashing for image retrieval via image representation learning. In Association for the Advancement of Artificial Intelligence, Vol. 1. 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Litao Yu, Yongsheng Gao, and Jun Zhou. 2018. Generative adversarial product quantisation. In Proceedings of the ACM Conference on Multimedia. ACM, 861--869. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Tan Yu, Junsong Yuan, Chen Fang, and Hailin Jin. 2018. Product quantization network for fast image retrieval. In Proceedings of the European Conference on Computer Vision. 186--201.Google ScholarGoogle ScholarCross RefCross Ref
  46. Haofeng Zhang, Li Liu, Yang Long, and Ling Shao. 2018. Unsupervised deep hashing with pseudo labels for scalable image retrieval. IEEE Trans. Image Process. 27, 4 (2018), 1626--1638. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Lei Zhang, Yongdong Zhang, Jinhui Tang, Xiaoguang Gu, Jintao Li, and Qi Tian. 2013. Topology preserving hashing for similarity search. In Proceedings of the ACM International Conference on Multimedia. 123--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, and Lei Zhang. 2015. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans. Image Process. 24, 12 (2015), 4766--4779.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Ting Zhang, Chao Du, and Jingdong Wang. 2014. Composite quantization for approximate nearest neighbor search. In Proceedings of the International Conference on Machine Learning. 838--846. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1556--1564.Google ScholarGoogle Scholar
  51. Wengang Zhou, Houqiang Li, Richang Hong, Yijuan Lu, and Qi Tian. 2015. BSIFT: Toward data-independent codebook for large scale image search. IEEE Trans. Image Process. 24, 3 (2015), 967--979.Google ScholarGoogle ScholarCross RefCross Ref
  52. Wengang Zhou, Yijuan Lu, Houqiang Li, and Qi Tian. 2012. Scalar quantization for large scale image search. In Proceedings of the ACM International Conference on Multimedia. 169--178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Wengang Zhou, Ming Yang, Houqiang Li, Xiaoyu Wang, Yuanqing Lin, and Qi Tian. 2014. Towards codebook-free: Scalable cascaded hashing for mobile image search. IEEE Trans. Multimedia 16, 3 (2014), 601--611. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Wengang Zhou, Ming Yang, Xiaoyu Wang, Houqiang Li, Yuanqing Lin, and Qi Tian. 2016. Scalable feature matching by dual cascaded scalar quantization for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1 (2016), 159--171. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Hao Zhu and Shenghua Gao. 2017. Locality-constrained deep supervised hashing for image retrieval. In Proceedings of the International Joint Conference on Artificial Intelligence. 3567--3573. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Weijia Zhu, Wenpeng Ding, Jizheng Xu, Yunhui Shi, and Baocai Yin. 2015. Hash-based block matching for screen content coding. IEEE Trans. Multimedia 17, 7 (2015), 935--944.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Deep Scalable Supervised Quantization by Self-Organizing Map

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!