Abstract
Visible-infrared person re-identification (Re-ID) has received increasing research attention for its great practical value in night-time surveillance scenarios. Due to the large variations in person pose, viewpoint, and occlusion in the same modality, as well as the domain gap brought by heterogeneous modality, this hybrid modality person matching task is quite challenging. Different from the metric learning methods for visible person re-ID, which only pose similarity constraints on class level, an efficient metric learning approach for visible-infrared person Re-ID should take both the class-level and modality-level similarity constraints into full consideration to learn sufficiently discriminative and robust features. In this article, the hybrid modality is divided into two types, within modality and cross modality. We first fully explore the variations that hinder the ranking results of visible-infrared person re-ID and roughly summarize them into three types: within-modality variation, cross-modality modality-related variation, and cross-modality modality-unrelated variation. Then, we propose a comprehensive metric learning framework based on four kinds of paired-based similarity constraints to address all the variations within and cross modality. This framework focuses on both class-level and modality-level similarity relationships between person images. Furthermore, we demonstrate the compatibility of our framework with any paired-based loss functions by giving detailed implementation of combing it with triplet loss and contrastive loss separately. Finally, extensive experiments of our approach on SYSU-MM01 and RegDB demonstrate the effectiveness and superiority of our proposed metric learning framework for visible-infrared person Re-ID.
- [1] . 2020. Hetero-Center loss for cross-modality person Re-identification. Neurocomputing 386 (2020), 97–109.Google Scholar
Cross Ref
- [2] . 2017. Beyond triplet loss: A deep quadruplet network for person re-identification. In Proceedings of the Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
- [3] . 2016. Person re-identification by multi-channel parts-based CNN with improved triplet loss function. In Computer Vision and Pattern Recognition.Google Scholar
- [4] . 2020. Hi-CMD: Hierarchical cross-modality disentanglement for visible-infrared person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20). IEEE.Google Scholar
Cross Ref
- [5] . 2018. Cross-modality person re-identification with generative adversarial training. In Proceedings of the 27th International Joint Conference on Artificial Intelligence IJCAI-18. Google Scholar
Digital Library
- [6] . 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’05). Google Scholar
Digital Library
- [7] . 2017. Person recognition system based on a combination of body images from visible light and thermal cameras. Sensors 17, 3 (2017), 605.
DOI: DOI: https://doi.org/10.3390/s17030605Google ScholarCross Ref
- [8] Weijian Deng, Liang Zheng, Guoliang Kang, Yi Yang, Qixiang Ye, and Jianbin Jiao. 2018. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE.Google Scholar
- [9] Zhangxiang Feng, Jianhuang Lai, and Xiaohua Xie. 2019. Learning modality-specific representations for visible-infrared person re-identification. IEEE Transactions on Image Processing 29 (2019), 579–590.Google Scholar
- [10] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Y. Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS’14). 2672–2680. Google Scholar
Digital Library
- [11] Wang Guan’An, Zhang Tianzhu, Cheng Jian, Liu Si, Yang Yang, and Hou Zengguang. 2020. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment. In IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE.Google Scholar
- [12] . 2019. A survey on deep learning based person re-identification. Acta Automat. Sin. 45, 11 (2019), 2032–2049.
DOI: DOI: https://doi.org/10.16383/j.aas.c180154Google Scholar - [13] . 2019. HSME: Hypersphere manifold embedding for visible thermal person re-identification. Proc. AAAI Conf. Artif. Intell. 33 (2019), 8385–8392.
DOI: DOI: https://doi.org/10.1609/aaai.v33i01.33018385 Google ScholarCross Ref
- [14] . 2017. In defense of the triplet loss for person re-identification. CoRR abs/1703.07737 (2017).
DOI: DOI: https://doi.org/1703.07737Google Scholar - [15] Yan Huang, Jingsong Xu, Qiang Wu, Zhedong Zheng, Zhaoxiang Zhang, and Jian Zhang. 2018. Multi-pseudo regularized label for generated data in person re-identification. IEEE Transactions on Image Processing PP (2018), 1–1. https://doi.org/10.1109/TIP.2018.2874715Google Scholar
- [16] . 2020. A similarity inference metric for RGB-infrared cross-modality person re-identification. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization. Google Scholar
Digital Library
- [17] . 2020. Infrared-visible cross-modal person re-identification with an X modality. Proc. AAAI Conf. Artif. Intell. 34, 4 (2020), 4610–4617.Google Scholar
- [18] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z. Li. 2015. Person re-identification by Local Maximal Occurrence representation and metric learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). IEEE.Google Scholar
- [19] . 2020. Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing 398 (2020), 11–19.Google Scholar
Cross Ref
- [20] . 2017. End-to-end comparative attention networks for person re-identification. IEEE Trans. Image Process. 26, 99 (2017), 3492–3506.Google Scholar
Digital Library
- [21] . 2020. Cross-modality person re-identification with shared-specific feature transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20). IEEE.Google Scholar
Cross Ref
- [22] Ye Mang, Shen Jianbing, Lin Gaojie, Xiang Tao, Shao Ling, and Steven C. H. Hoi. 2021. Deep learning for person re-identification: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 99 (2021), 1–1.Google Scholar
- [23] Ye Mang, Lan Xiangyuan, and Leng Qingming. 2019. Modality-aware Collaborative Learning for Visible Thermal Person Re-Identification. In 27th ACM International Conference. ACM. Google Scholar
Digital Library
- [24] Xuelin Qian, Yanwei Fu, Wenxuan Wang, Tao Xiang, Yang Wu, Yu-Gang Jiang, and Xiangyang Xue. 2018. Pose-normalized image generation for person re-identification. In Proceedings of the 15th European Conference, Munich, Germany. Springer, Cham.Google Scholar
- [25] Ergys Ristani and Carlo Tomasi. 2018. Features for multi-target multi-camera tracking and re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6036–6046.Google Scholar
- [26] Hailin Shi, Yang Yang, Xiangyu Zhu, Shengcai Liao, Zhen Lei, and Stan Z. Li. 2016. Embedding deep metric for person re-identification: A study against large variations. In European Conference on Computer Vision. Springer, Cham, 732–748.Google Scholar
- [27] . 2016. Improved deep metric learning with multi-class n-pair loss objective. In Advances in Neural Information Processing Systems, , , , , and (Eds.), Vol. 29. Curran Associates, Inc., 1857–1865. Google Scholar
Digital Library
- [28] . 2016. Gated siamese convolutional neural network architecture for human re-identification. In Proceedings of the European Conference on Computer Vision.Google Scholar
Cross Ref
- [29] . 2016. A siamese long short-term memory architecture for human re-identification. In Proceedings of the European Conference on Computer Vision.Google Scholar
Cross Ref
- [30] Wang Guan’An, Zhang Tianzhu, Yang Yang, Cheng Jian, Chang Jianlong, Liang Xu, and Hou Zengguang. 2020. Cross-modality paired-images generation for RGB-infrared person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence 34, 7 (2020), 12144–12151.Google Scholar
- [31] . 2018. Person re-identification with cascaded pairwise convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Reconigtion.Google Scholar
Cross Ref
- [32] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. 2018. Person transfer gan to bridge domain gap for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 79–88.Google Scholar
- [33] . 2017. RGB-infrared cross-modality person re-identification. In Proceedings of the International Conference on Computer Vision (ICCV’17). IEEE, Los Alamitos, CA.Google Scholar
Cross Ref
- [34] . 2018. Hierarchical discriminative learning for visible thermal person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI.Google Scholar
- [35] . 2018. Visible thermal person re-identification via dual-constrained top-ranking. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). Google Scholar
Digital Library
- [36] Shizhou Zhang, Yifei Yang, Peng Wang, Xiuwei Zhang, and Yanning Zhang. 2021. Attend to the difference: Cross-modality person re-identification via contrastive correlation. IEEE Transactions on Image Processing 30 (2021), 8861–8872.Google Scholar
- [37] . 2020. HPILN: A feature learning framework for cross-modality person re-identification. IET Image Process. 13, 14 (2020), 2897–2904.
DOI: DOI: https://doi.org/10.1049/iet-ipr.2019.0699Google ScholarCross Ref
- [38] . 2016. Person re-identification: Past, present and future.Google Scholar
- [39] . 2019. Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19).Google Scholar
- [40] . 2018. Camera style adaptation for person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
Index Terms
Hybrid Modality Metric Learning for Visible-Infrared Person Re-Identification
Recommendations
Class-Aware Modality Mix and Center-Guided Metric Learning for Visible-Thermal Person Re-Identification
MM '20: Proceedings of the 28th ACM International Conference on MultimediaVisible thermal person re-identification (VT-REID) is an important and challenging task in that 1) weak lighting environments are inevitably encountered in real-world settings and 2) the inter-modality discrepancy is serious. Most existing methods ...
Serialized Local Feature Representation Learning for Infrared-Visible Person Re-identification
Intelligent Computing Theories and ApplicationAbstractInfrared-Visible person re-identification is a kind of cross-modality person re-identification. The purpose of the task is that given a person image we need to find another image on the same person from gallery. The query images and gallery images ...
Modality-aware Collaborative Learning for Visible Thermal Person Re-Identification
MM '19: Proceedings of the 27th ACM International Conference on MultimediaVisible thermal person re-identification (VT-ReID) is a cross-modality pedestrian retrieval problem, which automatically searches persons between day-time visible images and night-time thermal images. Despite the extensive progress in single-modality ...






Comments