skip to main content
research-article

3D Skeleton and Two Streams Approach to Person Re-identification Using Optimized Region Matching

Authors Info & Claims
Published:06 October 2022Publication History
Skip Abstract Section

Abstract

Person re-identification (Re-ID) is a challenging and arduous task due to non-overlapping views, complex background, and uncontrollable occlusion in video surveillance. An existing method for capturing pedestrian local region information is to divide person regions into horizontal stripes, which may lead to invalid features and erroneous learning. To solve this problem, this paper proposes a 3D skeleton and a two-stream approach to person Re-ID. The first stream of the method uses the 3D skeleton for background filtering and region segmentation. The second stream uses Siamese net to extract the global descriptor. The features of the two streams are fused to preserve the integrity of the person. An optimized region matching method for metric learning is designed. Extensive comparing experiments were conducted with state-of-the-art Re-ID methods on the Market-1501, CUHK03, and DukeMTMC-reID datasets. Experimental results show that the proposed method outperforms the existing methods in recognition accuracy.

REFERENCES

  1. [1] Shen Chen, Qi Guo-Jun, Jiang Rongxin, Jin Zhongming, Yong Hongwei, Chen Yaowu, and Hua Xian-Sheng. 2018. Sharp attention network via adaptive sampling for person re-identification. IEEE Trans. Circuits Syst. Video Technol. 29, 10 (2018), 30163027. Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Wu Yu, Lin Yutian, Dong Xuanyi, Yan Yan, Bian Wei, and Yang Yi. 2019. Progressive learning for person re-identification with one example. IEEE Trans. Image Process. 28, 6 (2019), 28722881. Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Xuan Shiyu and Zhang Shiliang. 2021. Intra-inter camera similarity for unsupervised person re-identification. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’21). IEEE, Nashville, TN, USA, 1192611935. Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Zhu Jianqing, Zeng Huanqiang, Liao Shengcai, Lei Zhen, Cai Canhui, and Zheng LiXin. 2017. Deep hybrid similarity learning for person re-identification. IEEE Trans. Circuits Syst. Video Technol. 28, 11 (2017), 31833193. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Lin Yutian, Wu Yu, Yan Chenggang, Xu Mingliang, and Yang Yi. 2020. Unsupervised person re-identification via cross-camera similarity exploration. IEEE Trans. Image Process. 29 (2020), 54815490. Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Ye Mang, Shen Jianbing, and Shao Ling. 2020. Visible-infrared person re-identification via homogeneous augmented tri-modal learning. IEEE Trans. Inf. Forensic Secur. 16 (2020), 728739. Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Zhou Linghua, Min Weidong, Lin Deyu, Han Qing, and Liu Ruikang. 2020. Detecting motion blurred vehicle logo in IoV using Filter-DeblurGAN and VL-YOLO. IEEE Trans. Veh. Technol. 69, 4 (2020), 36043614. Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Zhou Kaiyang, Yang Yongxin, Cavallaro Andrea, and Xiang Tao. 2019. Omni-scale feature learning for person re-identification. In 2019 IEEE/CVF International Conference on Computer Vision (CVPR’19). IEEE, Seoul, Korea (South), 37023712. Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Cheng De, Li Zhihui, Gong Yihong, and Zhan Dingwen. 2018. Fusion of multiple person re-id methods with model and data-aware abilities. IEEE T. Cybern. 50, 2 (2018), 561571. Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Luo Jinghao, Liu Yaohua, Gao Changxin, and Sang Nong. 2019. Learning what and where from attributes to improve person re-identification. In 2019 IEEE International Conference on Image Processing (ICIP’19). IEEE, Taipei, Taiwan, 165169. Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Zheng Zhedong, Zheng Liang, and Yang Yi. 2017. A discriminatively learned CNN embedding for person re-identification. ACM Trans. Multimed. Comput. Commun. 14, 1 (2017), 120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Varior Rahul Rama, Shuai Bing, Lu Jiwen, Xu Dong, and Wang Gang. 2016. A Siamese long short-term memory architecture for human re-identification. In 2016 European Conference on Computer Vision (ECCV’16). Springer, Amsterdam, The Netherlands, Springer, Cham, 135153. Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Wu Shangxuan, Chen Ying-Cong, Li Xiang, Wu An-Cong, You Jin-Jie, and Zheng Wei-Shi. 2016. An enhanced deep feature representation for person re-identification. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV’16). IEEE, Lake Placid, NY, USA, 18. Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Zheng Liang, Huang Yujia, Lu Huchuan, and Yang Yi. 2019. Pose-invariant embedding for deep person re-identification. IEEE Trans. Image Process 28, 9 (2019), 45004509. Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Wei Longhui, Zhang Shiliang, Yao Hantao, Gao Wen, and Tian Qi. 2018. GLAD: Global–local-alignment descriptor for scalable person re-identification. IEEE Trans. Multimedia 21, 4 (2018), 986999. Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Varior Rahul Rama, Haloi Mrinal, and Wang Gang. 2016. Gated Siamese convolutional neural network architecture for human re-identification. In 2016 European Conference on Computer Vision (ECCV’16). Springer, Amsterdam, The Netherlands, Springer, Cham, 791808. Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Cao Zhe, Hidalgo Gines, Simon Tomas, Wei Shih-En, and Sheikh Yaser. 2019. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1 (2019), 172186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Yang Hao, Liu Li, Min Weidong, Yang Xiaosong, and Xiong Xin. 2020. Driver yawning detection based on subtle facial action recognition. IEEE Trans. Multimedia 23, (2020), 572583. Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Lin Yutian, Dong Xuanyi, Zheng Liang, Yan Yan, and Yang Yi. 2019. A bottom-up clustering approach to unsupervised person re-identification. In 2019 AAAI Conference on Artificial Intelligence (AAAI’19). AAAI, Hawaii, USA, 87388745. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Min Weidong, Fan Mengdan, Guo Xiaoguang, and Han Qing. 2017. A new approach to track multiple vehicles with the combination of robust detection and two classifiers. IEEE Trans. Intell. Transp. Syst. 19, 1 (2017), 174186. Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Luo Hao, Jiang Wei, Fan Xing, and Zhang Chi. 2020. STNReID: Deep convolutional networks with pairwise spatial transformer networks for partial person re-identification. IEEE Trans. Multimedia 22, 11 (2020), 29052913. Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Kang Jin Kyu, Hoang Toan Minh, and Park Kang Ryoung. 2019. Person re-identification between visible and thermal camera images based on deep residual CNN using single input. IEEE Access 7 (2019), 5797257984. Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Ma Fei, Jing Xiao-Yuan, Zhu Xiaoke, Tang Zhenmin, and Peng Zhiping. 2019. True-color and grayscale video person re-identification. IEEE Trans. Inf. Forensic Secur. 15 (2019), 115129. Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Yang Yang, Lei Zhen, Wang Jinqiao, and Li Stan Z.. 2019. In defense of color names for small-scale person re-identification. In 2019 International Conference on Biometrics (ICB’19). IEEE, Crete, Greece, 16. Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Wang Qi, Min Weidong, He Daojing, Zou Song, Huang Tiemei, Zhang Yu, and Liu Ruikang. 2020. Discriminative fine-grained network for vehicle re-identification using two-stage re-ranking. Sci. China-Inf. Sci. 63, 11 (2020), 112. Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Liao Shengcai, Hu Yang, Zhu Xiangyu, and Li Stan Z.. 2015. Person re-identification by local maximal occurrence representation and metric learning. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). IEEE, Boston, MA, USA, 21972206. Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Hong Xing Yu, Shi Zheng Wei, Wu Ancong, Guo Xiaowei, Gong Shaogang, and Lai JianHuang. 2019. Unsupervised person re-identification by soft multi label learning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19). IEEE, Seoul, Korea (South), 21482157. Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Xiao Tong, Li Shuang, Wang Bochao, Lin Liang, and Wang Xiaogang. 2017. Joint detection and identification feature learning for person search. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, Honolulu, HI, USA, 34153424. Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Li Wei, Zhao Rui, Xiao Tong, and Wang Xiaogang. 2014. DeepReID: Deep filter pairing neural network for person re-identification. In 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, Columbus, OH, USA, 152159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Wu Lin, Shen Chunhua, and van den Hengel Anton. 2016. PersonNet: Person re-identification with deep convolutional neural networks. (2016) arXiv:1601.07255. Retrieved from https://arxiv.org/abs/1601.07255Google ScholarGoogle Scholar
  31. [31] Zhong Zhun, Zheng Liang, Luo Zhiming, Li Shaozi, and Yang Yi. 2019. Invariance matters: Exemplar memory for domain adaptive person re-identification. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19). IEEE, Seoul, Korea (South), 598607. Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Wei Long, Wei Zhenyong, Jin Zhongming, Yu Zhengxu, Huang Jianqiang, Cai Deng, He Xiaofei, and Hua Xian-Sheng. 2020. SIF: Self-inspirited feature learning for person re-identification. IEEE Trans. Image Process. 29 (2020), 4942—4951. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. [33] Lin Yutian, Zheng Liang, Zheng Zhedong, Wu Yu, Hu Zhilan, Yan Chenggang, and Yang Yi. 2019. Improving person re-identification by attribute and identity learning. Pattern Recognit. 95 (2019), 151161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Bai Xiang, Yang Mingkun, Huang Tengteng, Dou Zhiyong, Yu Rui, and Xu Yongchao. 2019. Deep-person: Learning discriminative deep features for person re-identification. Pattern Recognit. 98 (2019), 107036. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Sun Yifan, Zheng Liang, Yang Yi, Tian Qi, and Wang Shengjin. 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In 2018 IEEE Conference on the European Conference on Computer Vision (CVPR’18). IEEE, Salt Lake City, UT, USA, 480496. Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Cho Yeong-Jun and Yoon Kuk-Jin. 2016. Improving person re-identification via pose-aware multi-shot matching. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE, Las Vegas, NV, USA, 13541362. Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Zheng Zhedong, Zheng Liang, and Yang Yi. 2018. Pedestrian alignment network for large-scale person re-identification. IEEE Trans. Circuits Syst. Video Technol. 29, 10 (2018), 30373045. Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Baltieri Davide, Vezzani Roberto, and Cucchiara Rita. 2015. Mapping appearance descriptors on 3D body models for people re-identification. Int. J. Comput. Vis. 111, 3 (2015), 345364. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Ding Guodong, Khan Salman, Tang Zhenmin, and Porikli Fatih. 2020. Feature mask network for person re-identification. Pattern Recognit. Letters 137 (2020), 9198. Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Li Wei, Zhu Xiatian, and Gong Shaogang. 2018. Harmonious attention network for person re-identification. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, Salt Lake City, UT, USA, 22852294. Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Miao Jiaxu, Wu Yu, Liu Ping, Ding Yuhang, and Yang Yi. 2019. Pose-guided feature alignment for occluded person re-identification. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE, Seoul, Korea (South), 542551. Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Si Jianlou, Zhang Honggang, Li Chun-Guang, Kuen Jason, Kong Xiangfei, Kot Alex C., and Wang Gang. 2018. Dual attention matching network for context-aware feature sequence based person re-identification. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, Salt Lake City, UT, USA, 53635372. Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Lin Tsung-Yu, RoyChowdhury Aruni, and Maji Subhransu. 2015. Bilinear CNN models for fine-grained visual recognition. In 2015 IEEE International Conference on Computer Vision (ICCV’15). IEEE, Santiago, Chile, 14491457. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Simon Kai and Lausen Georg. 2005. ViPER: Augmenting automatic information extraction with visual perceptions. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management (CIKM’05). ACM, Bremen, Germany, 381388. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Hu Hai-Miao, Fang Wen, Li Bo, and Tian Qi. 2018. An adaptive multi-projection metric learning for person re-identification across non-overlapping cameras. IEEE Trans. Circuits Syst. Video Technol. 29, 9 (2018), 28092821. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Perwiaz Nazia, Fraz Muhammad Moazam, and Shanzad Muhammad. 2018. Person re-identification using hybrid representation reinforced by metric learning. IEEE Access 6 (2018), 7733477349. Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Zhang Li, Xiang Tao, and Gong Shaogang. 2016. Learning a discriminative null space for person re-identification. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE, Las Vegas, NV, USA, 12391248. Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Zhong Zhun, Zheng Liang, Cao Donglin, and Li Shaozi. 2017. Re-ranking person re-identification with k-reciprocal encoding. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, Honolulu, HI, USA, 13181327. Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Russakovsky Olga, Deng Jia, Su Hao, Krause Jonathan, Satheesh Sanjeev, Ma Sean, Huang Zhiheng, Karpathy Andrej, Khosla Aditya, Bernstein Michael, Berg Alexander C., and Fei-Fei Li. 2015. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 3 (2015), 211252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Sun Yifan, Zheng Liang, Deng Weijian, and Wang Shengjin. 2017. SVDNet for pedestrian retrieval. In 2017 IEEE International Conference on Computer Vision (ICCV’17). IEEE, Venice, Italy, 38003808. Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Zheng Liang, Shen Liyue, Tian Lu, Wang Shengjin, Wang Jingdong, and Tian Qi. 2015. Scalable person re-identification: A benchmark. In 2015 IEEE International Conference on Computer Vision (ICCV’15). IEEE, Santiago, Chile, 11161124. Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Ristani Ergys, Solera Francesco, Zou Roger, Cucchiara Rita, and Tomasi Carlo. 2016. Performance measures and a data set for multi-target, multi-camera tracking. In 2016 European Conference on Computer Vision (ECCV’16). Springer, Amsterdam, The Netherlands, 1735. Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Hermans Alexander, Beyer Lucas, and Leibe Bastian. 2017. In defense of the triplet loss for person re-identification. (2017), arXiv:1703.07737. Retrieved from https://arxiv.org/abs/1703.07737Google ScholarGoogle Scholar
  54. [54] Hu Jie, Shen Li, and Sun Gang. 2018. Squeeze-and-excitation networks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, Salt Lake City, UT, USA, 71327141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Yang Xun, Wang Meng, and Tao Dacheng. 2017. Person re-identification with metric learning using privileged information. IEEE Trans. Image Process. 27, 2 (2017), 791805. Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Yang Xun, Wang Meng, Hong Richang, Tian Qi, and Rui Yong. 2017. Enhancing person re-identification in a self-trained subspace. ACM Trans. Multimed. Comput. Commun. 13, 3 (2017), 1—23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. [57] Wang Meng, Li Hao, Tao Dacheng, Lu Ke, and Wu Xindong. 2012. Multimodal graph-based reranking for web image search. IEEE Trans. Image Process. 21, 11 (2012), 46494661. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. [58] Yang Xun, Du Xiaoyu, and Wang Meng. 2020. Learning to match on graph for fashion compatibility modeling. In 2020 AAAI Conference on Artificial Intelligence (AAAI’20). AAAI, New York, NY, USA, 287294. Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Quispe Rodolfo and Pedrini Helio. 2019. Improved person re-identification based on saliency and semantic parsing with deep neural network models. Image Vis. Comput. 92 (2019), 103809. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. [60] Ge Yixiao, Li Zhuowan, Zhao Haiyu, Yin Guojun, Yi Shuai, Wang Xiaogang, and Li Hongsheng. 2018. FD-GAN: Pose-guided feature distilling GAN for robust person re-identification. NIPS, Montréal, Canada. arXiv:1810.02936. Retrieved from https://arxiv.org/abs/1810.02936Google ScholarGoogle Scholar
  61. [61] Suh Yumin, Wang Jingdong, Tang Siyu, Mei Tao, and Lee Kyoung Mu. 2018. Part-aligned bilinear representations for person re-identification. In Proceedings of European Conference on Computer Vision (ECCV’18). Springer, Munich, Germany, 402419. Google ScholarGoogle ScholarCross RefCross Ref
  62. [62] Chang Xiaobin, Hospedales Timothy M., and Xiang Tao. 2018. Multi-level factorisation net for person re-identification. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, Salt Lake City, UT, USA, 21092118. Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. 3D Skeleton and Two Streams Approach to Person Re-identification Using Optimized Region Matching

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 2s
      June 2022
      383 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3561949
      • Editor:
      • Abdulmotaleb El Saddik
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 6 October 2022
      • Online AM: 25 May 2022
      • Accepted: 6 May 2022
      • Revised: 11 April 2022
      • Received: 4 October 2021
      Published in tomm Volume 18, Issue 2s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!