Abstract
Person search is a time-consuming computer vision task that entails locating and recognizing query people in scenic pictures. Body components are commonly mismatched during matching due to position variation, occlusions, and partially absent body parts, resulting in unsatisfactory person search results. Existing approaches for extracting local characteristics of the human body using keypoint information are unable to handle the search job when distinct body parts are misaligned, ignoring to exploit multiple granularities, which is crucial in the person search process. Moreover, the alignment learning methods learn body part features with fixed and equal weights, ignoring the beneficial contextual information, e.g., the umbrella carried by the pedestrian, which supplements compelling clues for identifying the person. In this paper, we propose a Coarse-to-Fine Adaptive Alignment Representation (CFA2R) network for learning multiple granular features in misaligned person search in the coarse-to-fine perspective. To exploit more beneficial body parts and related context of the cropped pedestrians, we design a Part-Attentional Progressive Module (PAPM) to guide the network to focus on informative body parts and positive accessorial regions. Besides, we propose a Re-weighting Alignment Module (RAM) shedding light on more contributive parts instead of treating them equally. Specifically, adaptive re-weighted but not fixed part features are reconstructed by Re-weighting Reconstruction module, considering that different parts serve unequally during image matching. Extensive experiments conducted on CUHK-SYSU and PRW datasets demonstrate competitive performance of our proposed method.
- [1] . 2020. Enforcing affinity feature learning through self-attention for person re-identification. ACM Trans. Multimedia Comput. Commun. Appl. 16, 1 (2020), 16:1–16:22. Google Scholar
Digital Library
- [2] . 2021. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1 (2021), 172–186. Google Scholar
Digital Library
- [3] . 2018. RCAA: Relational context-aware agents for person search. In Proc. Springer Eur. Conf. Comput. Vis., Vol. 11213. 86–102.Google Scholar
- [4] . 2020. Hierarchical online instance matching for person search. In Proc. AAAI Conf. Artif. Intell.10518–10525.Google Scholar
- [5] . 2018. Person search via a mask-guided two-stream CNN model. In Proc. Springer Eur. Conf. Comput. Vis., Vol. 11211. 764–781.Google Scholar
- [6] . 2020. Person search by separated modeling and A mask-guided two-stream CNN model. IEEE Trans. Image Process. 29 (2020), 4669–4682. Google Scholar
Digital Library
- [7] . 2021. Norm-aware embedding for efficient person search and tracking. Int. J. Comput. Vis. 129, 11 (2021), 3154–3168. Google Scholar
Digital Library
- [8] . 2017. Deformable convolutional networks. In Proc. IEEE/CVF Int. Conf. Comput. Vis.764–773.Google Scholar
- [9] . 2020. Dynamic imposter based online instance matching for person search. Pattern Recognit. 100 (2020), 107120.Google Scholar
Digital Library
- [10] . 2014. Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36, 8 (2014), 1532–1545. Google Scholar
Digital Library
- [11] . 2009. Integral channel features. In Proc. BMVA Brit. Mach. Vis. Conf.1–11.Google Scholar
- [12] . 2020. Bi-directional interaction network for person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.2836–2845.Google Scholar
- [13] . 2020. Instance guided proposal network for person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.2582–2591.Google Scholar
- [14] . 2010. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 9 (2010), 1627–1645. Google Scholar
Digital Library
- [15] . 2021. Decoupled and memory-reinforced networks: Towards effective feature learning for one-step person search. In Proc. AAAI Conf. Artif. Intell.1505–1512.Google Scholar
- [16] . 2020. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2 (2020), 386–397. Google Scholar
Cross Ref
- [17] . 2018. End-to-end detection and re-identification integrated net for person search. In Proc. Springer Asian Conf. Comput. Vis., Vol. 11362. 349–364.Google Scholar
- [18] . 2018. Squeeze-and-excitation networks. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.7132–7141. Google Scholar
Cross Ref
- [19] . 2021. Occluded suspect search via channel-guided mechanism. Neural Comput. Appl. 33, 3 (2021), 961–971. Google Scholar
Digital Library
- [20] . 2022. Complementary data augmentation for cloth-changing person re-identification. IEEE Trans. Image Process. 31 (2022), 4227–4239. Google Scholar
Digital Library
- [21] . 2021. Decomposition makes better rain removal: An improved attention-guided deraining network. IEEE Trans. Circuits Syst. Video Technol. 31, 10 (2021), 3981–3995.Google Scholar
Cross Ref
- [22] . 2017. Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning. IEEE Trans. Image Process. 26, 3 (2017), 1363–1378. Google Scholar
Digital Library
- [23] . 2021. Prototype-guided saliency feature learning for person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.4865–4874.Google Scholar
- [24] . 2018. Person search by multi-scale matching. In Proc. Springer Eur. Conf. Comput. Vis., Vol. 11205. 553–569.Google Scholar
- [25] . 2019. Fast person search pipeline. In Proc. IEEE Int. Conf. Multimedia Expo. 1114–1119. Google Scholar
Cross Ref
- [26] . 2018. Multilevel collaborative attention network for person search. In Proc. Springer Asian Conf. Comput. Vis., Vol. 11361. 467–482.Google Scholar
- [27] . 2021. Hierarchical distillation learning for scalable person search. Pattern Recognit. 114 (2021), 107862. Google Scholar
Cross Ref
- [28] . 2015. Multi-scale learning for low-resolution person re-identification. In Proc. IEEE/CVF Int. Conf. Comput. Vis.3765–3773.Google Scholar
- [29] . 2015. Person re-identification by local maximal occurrence representation and metric learning. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.2197–2206.Google Scholar
- [30] . 2015. Efficient PSD constrained asymmetric metric learning for person re-identification. In Proc. IEEE/CVF Int. Conf. Comput. Vis.3685–3693.Google Scholar
- [31] . 2014. Microsoft COCO: Common objects in context. In Proc. Springer Eur. Conf. Comput. Vis., Vol. 8693. 740–755.Google Scholar
- [32] . 2015. Bilinear CNN models for fine-grained visual recognition. In Proc. IEEE/CVF Int. Conf. Comput. Vis.1449–1457. Google Scholar
Digital Library
- [33] . 2021. Graph similarity rectification for person search. Neurocomputing 465 (2021), 184–194. Google Scholar
Digital Library
- [34] . 2017. Neural person search machines. In Proc. IEEE/CVF Int. Conf. Comput. Vis.493–501.Google Scholar
- [35] . 2020. Dual context-aware refinement network for person search. In Proc. ACM Int. Conf. Multimedia. 3450–3459.Google Scholar
- [36] . 2012. Local descriptors encoded by Fisher vectors for person re-identification. In Proc. Springer Eur. Conf. Comput. Vis. Workshops, Vol. 7583. 413–422.Google Scholar
- [37] . 2016. Hierarchical Gaussian descriptor for person re-identification. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.1363–1372.Google Scholar
- [38] . 2019. Query-guided end-to-end person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.811–820.Google Scholar
- [39] . 2019. Mask-guided attention network for occluded pedestrian detection. In Proc. IEEE/CVF Int. Conf. Comput. Vis.4966–4974.Google Scholar
- [40] . 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39, 6 (2017), 1137–1149. Google Scholar
Digital Library
- [41] . 2018. Dual attention matching network for context-aware feature sequence based person re-identification. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.5363–5372. Google Scholar
Cross Ref
- [42] . 2018. Mask-guided contrastive attention model for person re-identification. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.1179–1188.Google Scholar
- [43] . 2018. Part-aligned bilinear representations for person re-identification. In Proc. Springer Eur. Conf. Comput. Vis., Vol. 11218. 418–437.Google Scholar
- [44] . 2015. Going deeper with convolutions. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.1–9.Google Scholar
- [45] . 2018. Eliminating background-bias for robust person re-identification. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.5794–5803.Google Scholar
- [46] . 2020. Listen, look, and find the one: Robust person search with multimodality index. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 47:1–47:20. Google Scholar
Digital Library
- [47] . 2021. Consistency-constancy bi-knowledge learning for pedestrian detection in night surveillance. In Proc. ACM Int. Conf. Multimedia. 4463–4471. Google Scholar
Digital Library
- [48] . 2016. Knowledge-based coding of objects for multisource surveillance video data. IEEE Trans. Multim. 18, 9 (2016), 1691–1706. Google Scholar
Digital Library
- [49] . 2017. Joint detection and identification feature learning for person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.3376–3385.Google Scholar
- [50] . 2021. Rethinking data collection for person re-identification: Active redundancy reduction. Pattern Recognit. 113 (2021), 107827. Google Scholar
Cross Ref
- [51] . 2021. Exploring image enhancement for salient object detection in low light images. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1s (2021), 1–19. Google Scholar
Digital Library
- [52] . 2021. Anchor-free person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.7690–7699.Google Scholar
- [53] . 2019. Learning context graph for person search. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.2158–2167.Google Scholar
- [54] . 2015. Convolutional channel features. In Proc. IEEE/CVF Int. Conf. Comput. Vis.82–90.Google Scholar
- [55] . 2018. SKEPRID: Pose and illumination change-resistant skeleton-based person re-identification. ACM Trans. Multimedia Comput. Commun. Appl. 14, 4 (2018), 82:1–82:24. Google Scholar
Digital Library
- [56] . 2016. Is faster R-CNN doing well for pedestrian detection? In Proc. Springer Eur. Conf. Comput. Vis.443–457.Google Scholar
- [57] . 2016. Learning a discriminative null space for person re-identification. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.1239–1248.Google Scholar
- [58] . 2021. Diverse knowledge distillation for end-to-end person search. In Proc. AAAI Conf. Artif. Intell.3412–3420.Google Scholar
- [59] . 2016. Sample-specific SVM learning for person re-identification. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.1278–1287.Google Scholar
- [60] . 2017. Pyramid scene parsing network. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.6230–6239.Google Scholar
- [61] . 2017. Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.907–915.Google Scholar
- [62] . 2017. Person re-identification in the wild. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.3346–3355.Google Scholar
- [63] . 2018. A discriminatively learned CNN embedding for person reidentification. ACM Trans. Multimedia Comput. Commun. Appl. 14, 1 (2018), 13:1–13:20. Google Scholar
Digital Library
- [64] . 2019. Pedestrian alignment network for large-scale person re-identification. IEEE Trans. Circuits Syst. Video Technol. 29, 10 (2019), 3037–3045. Google Scholar
Cross Ref
- [65] . 2021. Part-aligned network with background for misaligned person search. In Proc. IEEE Int. Conf. Acoustics Speech Signal Process.4250–4254.Google Scholar
- [66] . 2021. Unsupervised vehicle search in the wild: A new benchmark. In Proc. ACM Int. Conf. Multimedia. 5316–5325.Google Scholar
- [67] . 2020. Robust partial matching for person search in the wild. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.6826–6834.Google Scholar
- [68] . 2021. Cross-view similarity exploration for unsupervised cross-domain person re-identification. Neural Comput. Appl. 33, 9 (2021), 4001–4011. Google Scholar
Digital Library
Index Terms
Beyond the Parts: Learning Coarse-to-Fine Adaptive Alignment Representation for Person Search
Recommendations
Person resolution in person search results: WebHawk
CIKM '05: Proceedings of the 14th ACM international conference on Information and knowledge managementFinding information about people on the Web using a search engine is difficult because there is a many-to-many mapping between person names and specific persons (i.e. referents). This paper describes a person resolution system, called WebHawk. Given a ...
Person Search in a Scene by Jointly Modeling People Commonness and Person Uniqueness
MM '14: Proceedings of the 22nd ACM international conference on MultimediaThis paper presents a novel framework for a multimedia search task: searching a person in a scene using human body appearance. Existing works mostly focus on two independent problems related to this task, i.e., people detection and person re-...
Making person search enjoy the merits of person re-identification
Highlights- A knowledge transfer framework to make the one-step person search model enjoy the merits of powerful person re-identification models.
AbstractPerson search is an extended task of person re-identification (Re-ID). However, most existing one-step person search works do not study how to employ existing Re-ID models to improve the one-step person search. To address this issue, ...






Comments