Deep Local Binary Coding for Person Re-Identification by Delving into the Details

ABSTRACT

Person re-identification (ReID) has recently received extensive research interests due to its diverse applications in multimedia analysis and computer vision. However, the majority of existing works focus on improving matching accuracy, while ignoring matching efficiency. In this work, we present a novel binary representation learning framework for efficient person ReID, namely Deep Local Binary Coding (DLBC). Different from existing deep binary ReID approaches, DLBC attempts to learn discriminative binary codes by explicitly interacting with local visual details. Specifically, DLBC first extracts a set of local features from spatially salient regions of pedestrian images. Subsequently, DLBC formulates a new binary-local semantic mutual information (BSMI) maximization term, based on which a self-lifting (SL) block is built to further exploit the semantic importance of local features. The BSMI term together with the SL block simultaneously enhances the dependency of binary codes on selected local features as well as their robustness to cross-view visual inconsistency. In addition, an efficient optimizing method is developed to train the proposed deep models with orthogonal and binary constraints. Extensive experiments reveal that DLBC significantly minimizes the accuracy gap between binary ReID methods and the state-of-the-art real-valued ones, whilst remarkably reducing query time and memory cost.

References

1. E. Ahmed, M. Jones, and T. K. Marks. 2015. An improved deep learning architecture for person re-identification. In CVPR.
2. Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. 2018. Mine: mutual information neural estimation. In Proceedings of the International Conference on Machine Learning.
3. Binghui Chen, Weihong Deng, and Jiani Hu. 2019 a. Mixed High-Order Attention Network for Person Re-Identification. In Proceedings of International Conference on Computer Vision. 371--381.
4. Binghui Chen, Weihong Deng, and Jiani Hu. 2019 b. Mixed High-Order Attention Network for Person Re-Identification. In Proceedings of the International Conference on Computer Vision.
5. Binghui Chen, Weihong Deng, and Jiani Hu. 2019 c. Mixed High-Order Attention Network for Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
6. Guangyi Chen, Chunze Lin, Liangliang Ren, Jiwen Lu, and Jie Zhou. 2019 e. Self-Critical Attention Learning for Person Re-Identification. In Proceedings of International Conference on Computer Vision. 9637--9646.
7. Guangyi Chen, Chunze Lin, Liangliang Ren, Jiwen Lu, and Jie Zhou. 2019 f. Self-Critical Attention Learning for Person Re-Identification. In Proceedings of the International Conference on Computer Vision.
8. Jiaxin Chen, Jie Qin, Li Liu, Fan Zhu, Fumin Shen, Jin Xie, and Ling Shao. 2019 g. Deep sketch-shape hashing with segmented 3d stochastic viewing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 791--800.
9. Jiaxin Chen, Yunhong Wang, Jie Qin, Li Liu, and Ling Shao. 2017. Fast person re-identification via cross-camera semantic binary transformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3873--3882.
10. Jiaxin Chen, Yunhong Wang, and Yuan Yan Tang. 2016a. Person re-identification by exploiting spatio-temporal cues and multi-view metric learning. IEEE Signal Processing Letters, Vol. 23, 7 (2016), 998--1002.
11. Jiaxin Chen, Yunhong Wang, and Rui Wu. 2016b. Person re-identification by distance metric learning to discrete hashing. In 2016 IEEE International Conference on Image Processing (ICIP). IEEE, 789--793.
12. Jiaxin Chen, Zhaoxiang Zhang, and Yunhong Wang. 2014. Relevance metric learning for person re-identification by exploiting global similarities. In 2014 22nd International Conference on Pattern Recognition. IEEE, 1657--1662.
13. Jiaxin Chen, Zhaoxiang Zhang, and Yunhong Wang. 2015. Relevance metric learning for person re-identification by exploiting listwise similarities. IEEE Transactions on Image Processing, Vol. 24, 12 (2015), 4741--4755.
14. Tianlong Chen, Shaojin Ding, Jingyi Xie, Ye Yuan, Wuyang Chen, Yang Yang, Zhou Ren, and Zhangyang Wang. 2019 d. ABD-Net: Attentive but Diverse Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
15. Zuozhuo Dai, Mingqiang Chen, Xiaodong Gu, Siyu Zhu, and Ping Tan. 2019. Batch DropBlock Network for Person Re-Identification and Beyond. In The IEEE International Conference on Computer Vision (ICCV).
16. Zan Gao, Li-Shuai Gao, Hua Zhang, Zhiyong Cheng, and Richang Hong. 2019. Deep Spatial Pyramid Features Collaborative Reconstruction for Partial Person ReID. In Proceedings of the 27th ACM International Conference on Multimedia. 1879--1887.
17. Shaogang Gong, Marco Cristani, Chen Change Loy, and Timothy M Hospedales. 2014. The re-identification challenge. In Person re-identification. Springer, 1--20.
18. Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2012. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 12 (2012), 2916--2929.
19. Yuan Gong, Yizhe Zhang, Christian Poellabauer, et al. 2019. Second-order Non-local Attention Networks for Person Re-identification. In Proceedings of the International Conference on Computer Vision.
20. D. Gray and H. Tao. 2008. Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. In ECCV.
21. Jianyuan Guo, Yuhui Yuan, Lang Huang, Chao Zhang, Jin-Ge Yao, and Kai Han. 2019. Beyond Human Parts: Dual Part-Aligned Representations for Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
23. R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In Proceedings of the International Conference on Learning Representations.
24. Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, and Xilin Chen. 2019. Interaction-And-Aggregation Network for Person Re-Identification. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
25. Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.
26. Lei Huang, Xianglong Liu, Bo Lang, Adams Wei Yu, Yongliang Wang, and Bo Li. 2018. Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks. In Thirty-Second AAAI Conference on Artificial Intelligence.
27. Yukun Huang, Zheng-Jun Zha, Xueyang Fu, and Wei Zhang. 2019 b. Illumination-Invariant Person Re-Identification. In Proceedings of the 27th ACM International Conference on Multimedia. 365--373.
28. Ziling Huang, Zheng Wang, Wei Hu, Chia-Wen Lin, and Shin'ichi Satoh. 2019 a. DoT-GNN: Domain-Transferred Graph Neural Network for Group Re-identification. In Proceedings of the 27th ACM International Conference on Multimedia. 1888--1896.
29. S. Karanam, Y. Li, and R. J Radke. 2015. Person re-identification with discriminatively trained viewpoint invariant dictionaries. In ICCV.
30. M. Kostinger, M. Hirzer, P. Wohlhart, P. Roth, and H. Bischof. 2012. Large scale metric learning from equivalence constraints. In CVPR.
31. I. Kviatkovsky, A. Adam, and E. Rivlin. 2013. Color invariants for person reidentification. TPAMI, Vol. 35, 7 (2013), 1622--1634.
32. Diangang Li, Yihong Gong, De Cheng, Weiwei Shi, Xiaoyu Tao, and Xinyuan Chang. 2019. Consistency-Preserving deep hashing for fast person re-identification. Pattern Recognition, Vol. 94 (2019), 207--217.
33. W. Li, R. Zhao, T. Xiao, and X. Wang. 2014a. Deepreid: Deep filter pairing neural network for person re-identification. In CVPR.
34. Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. 2014b. Deepreid: Deep filter pairing neural network for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition. 152--159.
35. S. Liao, Y. Hu, X. Zhu, and S. Z. Li. 2015. Person re-identification by local maximal occurrence representation and metric learning. In CVPR.
36. S. Liao and S. Z. Li. 2015. Efficient PSD Constrained Asymmetric Metric Learning for Person Re-identification. In ICCV.
37. Tsung-Yu Lin and Subhransu Maji. 2017. Improved bilinear pooling with cnns. arXiv preprint arXiv:1707.06772 (2017).
38. G. Lisanti, I. Masi, A. D. Bagdanov, and A. Del Bimbo. 2013. Person re-identification by iterative re-weighted sparse ranking. TPAMI, Vol. 37, 8 (2013), 1629--1642.
39. Jiawei Liu, Zheng-Jun Zha, Richang Hong, Meng Wang, and Yongdong Zhang. 2019 b. Deep adversarial graph attention convolution network for text-based person search. In Proceedings of the 27th ACM International Conference on Multimedia. 665--673.
40. Zheng Liu, Jiaxin Chen, and Yunhong Wang. 2016. A fast adaptive spatio-temporal 3d feature for video-based person re-identification. In 2016 IEEE international conference on image processing (ICIP). IEEE, 4294--4298.
41. Zheng Liu, Jie Qin, Annan Li, Yunhong Wang, and Luc Van Gool. 2019 a. Adversarial binary coding for efficient person re-identification. In 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 700--705.
42. C. C. Loy, C. Liu, and S. Gong. 2013. Person re-identification by manifold ranking. In ICIP.
43. Chuanchen Luo, Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. 2019 a. Spectral Feature Transformation for Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
44. Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. 2019 b. Bag of tricks and a strong baseline for deep person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 0--0.
45. S. Paisitkriangkrai, C. Shen, and A. Hengel. 2015. Learning to rank in person re-identification with metric ensembles. In CVPR.
46. Fang Pengfei, Zhou Jieming, Kumar Roy Soumava, Petersson Lars, and Harandi Mehrtash. 2019. Bilinear Attention Networks for Person Retrieval. In Proceedings of the IEEE international conference on computer vision. 8030--8039.
47. B. Prosser, W. S. Zheng, S. Gong, T. Xiang, and Q. Mary. 2010. Person Re-Identification by Support Vector Ranking. In BMVC.
48. Jie Qin, Li Liu, Ling Shao, Fumin Shen, Bingbing Ni, Jiaxin Chen, and Yunhong Wang. 2017. Zero-shot action recognition with error-correcting output codes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2833--2842.
49. Jie Qin, Yunhong Wang, Li Liu, Jiaxin Chen, and Ling Shao. 2016. Beyond semantic attributes: Discrete latent attributes learning for zero-shot recognition. IEEE Signal Processing Letters, Vol. 23, 11 (2016), 1667--1671.
50. Ruijie Quan, Xuanyi Dong, Yu Wu, Linchao Zhu, and Yi Yang. 2019. Auto-ReID: Searching for a Part-Aware ConvNet for Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
51. Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. 2016. Performance measures and a data set for multi-target, multi-camera tracking. In European Conference on Computer Vision. Springer, 17--35.
52. Yuming Shen, Jie Qin, Jiaxin Chen, Mengyang Yu, Li Liu, Fan Zhu, Fumin Shen, and Ling Shao. 2020. Auto-Encoding Twin-Bottleneck Hashing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2818--2827.
53. Yantao Shen, Tong Xiao, Hongsheng Li, Shuai Yi, and Xiaogang Wang. 2018. End-to-End Deep Kronecker-Product Matching for Person Re-Identification. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
54. C. Su, S. Zhang, J. Xing, W. Gao, and Q. Tian. 2016. Deep Attributes Driven Multi-Camera Person Re-identification. In ECCV.
55. Shupeng Su, Chao Zhang, Kai Han, and Yonghong Tian. 2018. Greedy Hash: Towards Fast Optimization for Accurate Hash Coding in CNN. In Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). Curran Associates, Inc., 798--807.
56. Arulkumar Subramaniam, Athira Nambiar, and Anurag Mittal. 2019. Second-order Non-local Attention Networks for Person Re-identification. In Proceedings of International Conference on Computer Vision. 562--572.
57. Yifan Sun, Qin Xu, Yali Li, Chi Zhang, Yikang Li, Shengjin Wang, and Jian Sun. 2019. Perceive Where to Focus: Learning Visibility-Aware Part-Level Features for Partial Person Re-Identification. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR. 393--402.
58. Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Proceedings of the European Conference on Computer Vision (ECCV). 480--496.
59. Chiat-Pin Tay, Sharmili Roy, and Kim-Hui Yap. 2019. AANet: Attribute Attention Network for Person Re-Identifications. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
60. Petar Velivc ković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep graph infomax. In Proceedings of the International Conference on Learning Representations.
61. Bing Wang, Gang Wang, Kap Luk Chan, and Li Wang. 2014. Tracklet association with online target-specific metric learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1234--1241.
62. G. Wang, L. Lin, S. Ding, Y. Li, and Q. Wang. 2016b. DARI: Distance Metric and Representation Integration for Person Verification. In AAAI.
63. Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. 2018. Learning discriminative features with multiple granularities for person re-identification. In 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 274--282.
64. Xiaogang Wang, Kinh Tieu, and Eric L Grimson. 2008. Correspondence-free activity analysis and scene modeling in multiple camera views. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, 1 (2008), 56--71.
65. Z. Wang, R. Hu, C. Liang, Y. Yu, J. Jiang, M. Ye, J. Chen, and Q Leng. 2016a. Zero-shot person re-identification via cross-view consistency. IEEE Trans. on Multimedia, Vol. 18, 2 (2016), 260--272.
66. Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. 2018. Person Transfer GAN to Bridge Domain Gap for Person Re-Identification. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
67. Longhui Wei, Shiliang Zhang, Hantao Yao, Wen Gao, and Qi Tian. 2017. Glad: Global-local-alignment descriptor for pedestrian retrieval. In Proceedings of the 25th ACM international conference on Multimedia. 420--428.
68. Bryan Xia, Yuan Gong, Yizhe Zhang, and Christian Poellabauer. 2019 a. Second-order Non-local Attention Networks for Person Re-identification. In Proceedings of International Conference on Computer Vision. 3760--3769.
69. Bryan (Ning) Xia, Yuan Gong, Yizhe Zhang, and Christian Poellabauer. 2019 b. Second-Order Non-Local Attention Networks for Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
70. T. Xiao, H. Li, W. Ouyang, and X. Wang. 2016. Learning deep feature representations with domain guided dropout for person re-identification. In CVPR.
71. Jing Xu, Rui Zhao, Feng Zhu, Huaming Wang, and Wanli Ouyang. 2018. Attention-Aware Compositional Network for Person Re-Identification. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
72. Yichao Yan, Jie Qin, Jiaxin Chen, Li Liu, Fan Zhu, Ying Tai, and Ling Shao. 2020. Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2899--2908.
73. Wenjie Yang, Houjing Huang, Zhang Zhang, Xiaotang Chen, Kaiqi Huang, and Shu Zhang. 2019. Towards Rich Feature Discovery With Class Activation Maps Augmentation for Person Re-Identification. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
74. Y. Yang, J. Yang, J. Yan, S. Liao, D. Yi, and S. Z. Li. 2014. Salient color names for person re-identification. In ECCV.
75. Mang Ye, Xiangyuan Lan, and Qingming Leng. 2019. Modality-aware Collaborative Learning for Visible Thermal Person Re-Identification. In Proceedings of the 27th ACM International Conference on Multimedia. 347--355.
76. Mang Ye, Chao Liang, Zheng Wang, Qingming Leng, and Jun Chen. 2015. Ranking optimization for person re-identification via similarity and dissimilarity. In Proceedings of the 23rd ACM international conference on Multimedia. 1239--1242.
77. J. You, A. Wu, X. Li, and W. S. Zheng. 2016. Top-push Video-based Person Re-identification. In CVPR.
78. Mingyong Zeng, Chang Tian, and Zemin Wu. 2018. Person re-identification with hierarchical deep learning feature and efficient xqda metric. In Proceedings of the 26th ACM international conference on Multimedia. 1838--1846.
79. L. Zhang, T. Xiang, and S. Gong. 2016b. Learning a Discriminative Null Space for Person Re-identification. In CVPR.
80. Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, and Lei Zhang. 2015a. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing, Vol. 24, 12 (2015), 4766--4779.
81. R. Zhang, L. Lin, R. Zhang, W. Zuo, and L. Zhang. 2015b. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. TIP, Vol. 24, 12 (2015), 4766--4779.
82. Y. Zhang, X. Li, L. Zhao, and Z. Zhang. 2016a. Semantics-Aware Deep Correspondence Structure Learning for Robust Person Re-identification. In IJCAI.
83. Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1556--1564.
84. R. Zhao, W. Ouyang, and X. Wang. 2013a. Person re-identification by salience matching. In ICCV.
85. R. Zhao, W. Ouyang, and X. Wang. 2013b. Unsupervised salience learning for person re-identification. In CVPR.
86. R. Zhao, W. Ouyang, and X. Wang. 2014. Learning mid-level filters for person re-identification. In CVPR.
87. Feng Zheng, Cheng Deng, Xing Sun, Xinyang Jiang, Xiaowei Guo, Zongqiao Yu, Feiyue Huang, and Rongrong Ji. 2019 a. Pyramidal Person Re-IDentification via Multi-Loss Dynamic Training. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8514--8522.
88. Feng Zheng and Ling Shao. 2016. Learning Cross-View Binary Identities for Fast Person Re-Identification.. In IJCAI. 2399--2406.
89. Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015. Scalable person re-identification: A benchmark. In Proceedings of the IEEE international conference on computer vision. 1116--1124.
90. W. S. Zheng, S. Gong, and T. Xiang. 2013. Reidentification by relative distance comparison. TPAMI, Vol. 35, 3 (2013), 653--668.
91. Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, and Jan Kautz. 2019 b. Joint discriminative and generative learning for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2138--2147.
92. Zhun Zhong, Liang Zheng, Donglin Cao, and Shaozi Li. 2017. Re-ranking person re-identification with k-reciprocal encoding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1318--1327.
93. Jieming Zhou, Soumava Kumar Roy, Lars Petersson, and Mehrtash Harandi. 2019 a. Bilinear Attention Networks for Person Retrieval. In Proceedings of International Conference on Computer Vision. 8030--8039.
94. Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. 2019 b. Omni-Scale Feature Learning for Person Re-Identification. In The IEEE International Conference on Computer Vision (ICCV).
95. Fuqing Zhu, Xiangwei Kong, Liang Zheng, Haiyan Fu, and Qi Tian. 2017. Part-based deep hashing for large-scale person re-identification. IEEE Transactions on Image Processing, Vol. 26, 10 (2017), 4806--4817.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision representations

        • Image representations

      • Computer vision tasks

        • Visual content-based indexing and retrieval