skip to main content
research-article
Public Access

SKEPRID: Pose and Illumination Change-Resistant Skeleton-Based Person Re-Identification

Published:10 October 2018Publication History
Skip Abstract Section

Abstract

Currently, the surveillance camera-based person re-identification is still challenging because of diverse factors such as people’s changing poses and various illumination. The various poses make it hard to conduct feature matching across images, and the illumination changes make color-based features unreliable. In this article, we present SKEPRID,1 a skeleton-based person re-identification method that handles strong pose and illumination changes jointly. To reduce the impacts of pose changes on re-identification, we estimate the joints’ positions of a person based on the deep learning technique and thus make it possible to extract features on specific body parts with high accuracy. Based on the skeleton information, we design a set of local color comparison-based cloth-type features, which are resistant to various lighting conditions. Moreover, to better evaluate SKEPRID, we build the PO8LI2 dataset, which has large pose and illumination diversity. Our experimental results show that SKEPRID outperforms state-of-the-art approaches in the case of strong pose and illumination variation.

References

  1. S. M. Safdarnejad, X. Liu, L. Udpa, B. Andrus, J. Wood, and D. Craven. 2015. Sports videos in the wild (SVW): A video dataset for sports analysis. In Proceedings of the IEEE FG, vol. 1. 1--7.Google ScholarGoogle Scholar
  2. J. Liu, J. Luo, and M. Shah. 2009. Recognizing realistic actions from videos in the wild. In Proceedings of the IEEE CVPR. 1996--2003.Google ScholarGoogle Scholar
  3. S. Ramagiri, R. Kavi, and V. Kulathumani. 2011. Real-time multi-view human action recognition using a wireless camera network. In Proceedings of the ACM/IEEE ICDSC. 1--6.Google ScholarGoogle Scholar
  4. W. Li, R. Zhao, and X. Wang. 2012. Human re-identification with transferred metric learning. In Proceedings of the ACCV. 31--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. D. S. Cheng, M. Cristani, M. Stoppa, L. Bazzani, and V. Murino. 2011. Custom pictorial structures for re-identification. In Proceedings of the BMVC, vol. 5. 6.Google ScholarGoogle Scholar
  6. M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. 2010. Person re-identification by symmetry-driven accumulation of local features. In Proceedings of the IEEE CVPR. 2360--2367.Google ScholarGoogle Scholar
  7. C. Weinrich, M. Volkhardt, and H.-M. Gross. 2013. Appearance-based 3D upper-body pose estimation and person re-identification on mobile robots. In Proceedings of the IEEE SMC. 4384--4390. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. S. Bak, F. Martins, and F. Bremond. 2015. Person re-identification by pose priors. In Proceedings of the SPIE/IS8T Electronic Imaging, International Society for Optics and Photonics. 93990H--93990H.Google ScholarGoogle Scholar
  9. Y.-J. Cho and K.-J. Yoon. 2016. Improving person re-identification via pose-aware multi-shot matching. In Proceedings of the IEEE CVPR. 1354--1362.Google ScholarGoogle ScholarCross RefCross Ref
  10. R. Satta, F. Pala, G. Fumera, and F. Roli. 2013. Real-time appearance-based person re-identification over multiple KinectTM cameras. In Proceedings of the VISAPP, vol. 2. 407--410.Google ScholarGoogle Scholar
  11. I. B. Barbosa, M. Cristani, A. D. Bue, L. Bazzani, and V. Murino. 2012. Re-identification with RGB-D sensors. In Proceedings of the ECCV. 433--442. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Munaro, S. Ghidoni, D. T. Dizmen, and E. Menegatti. 2014. A feature-based approach to people re-identification using skeleton keypoints. In Proceedings of the IEEE ICRA. 5644--5651.Google ScholarGoogle Scholar
  13. S. Yuan, Y. Tian, and A. Arditi. 2011. Clothing matching for visually impaired persons. Technol. Disabil. 23, 2, 75--85.Google ScholarGoogle ScholarCross RefCross Ref
  14. R. Layne, T. M. Hospedales, S. Gong, and Q. Mary. 2012. Person re-identification by attributes. In Proceedings of the BMVC, vol. 2(3), p. 8.Google ScholarGoogle Scholar
  15. E. Ahmed, M. Jones, and T. K. Marks. 2015. An improved deep learning architecture for person re-identification. In Proceedings of the IEEE CVPR. 3908--3916.Google ScholarGoogle Scholar
  16. D. Yi, Z. Lei, S. Liao, and S. Z. Li. 2014. Deep metric learning for person re-identification. In Proceedings of the IEEE ICPR. 34--39. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. L. Zheng, Y. Huang, H. Lu, and Y. Yang. 2017. Pose invariant embedding for deep person re-identification, arXiv preprint arXiv:1701.07732.Google ScholarGoogle Scholar
  18. H. Zhao, M. Tian, S. Sun, J. Shao, J. Yan, S. Yi, X. Wang, and X. Tang. 2017. Spindle Net: Person re-identification with human body region guided feature decomposition and fusion. In Proceedings of the CVPR.Google ScholarGoogle Scholar
  19. C. Su, J. Li, S. Zhang, J. Xing, W. Gao, and Q. Tian. 2017. Pose-driven Deep Convolutional Model for Person Re-identification. In Proceedings of the IEEE ICCV. 3980--3989.Google ScholarGoogle Scholar
  20. W. Li, R. Zhao, T. Xiao, and X. Wang. 2014. Deepreid: Deep filter pairing neural network for person re-identification. In Proceedings of the IEEE CVPR. 152--159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian. 2015. Scalable person re-identification: A benchmark. In Proceedings of the IEEE ICCV. 1116--1124. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S. Khamis, C.-H. Kuo, V. K. Singh, V. D. Shet, and Larry S. Davis. 2014. Joint learning for attribute-consistent person re-identification. In Proceedings of the ECCV. 134--146.Google ScholarGoogle Scholar
  23. F. Xiong, M. Gou, O. Camps, and M. Sznaier. 2014. Person re-identification using kernel-based metric learning methods. In Proceedings of the ECCV. 1--16.Google ScholarGoogle Scholar
  24. M. Koestinger, M. Hirzer, P. Wohlhart, P. Roth, and H. Bischof. 2012. Large scale metric learning from equivalence constraints. In Proceedings of theIEEE CVPR. 2288--2295. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. N. Martinel, C. Micheloni, and G. Feresti. 2014. Saliency weighted features for person re-identification. In Proceedings of the ECCV Workshop on Visual Surveillance and Re-identification.Google ScholarGoogle Scholar
  26. S. Pedagadi, J. Orwell, S. Velastin, and B. Boghossian. 2013. Local Fisher discriminant analysis for pedestrian re-identification. In Proceedings of the IEEE CVPR. 3318--3325. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. Yan, D. Xu, B. Zhang, H. J. Zhang, Q. Yang, and S. Lin. 2007. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE TPAMI 29, 1, 40--51. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. K. Q. Weinberger, J. Blitzer, and L. K. Saul. 2006. Distance metric learning for large margin nearest-neighbor classification. In Proceedings of the NIPS, vol. 18. 1473. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. A. Bhattacharyya. 1943. On a measure of divergence between two statistical population defined by their population distributions. Bull. Calcutta Math. Soc. 35, 99--109.Google ScholarGoogle Scholar
  30. A. Toshev and C. Szegedy. 2014. Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE CVPR. 1653--1660. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. 2016. Convolutional pose machines. In Proceedings of the IEEE CVPR. 4724--4732.Google ScholarGoogle Scholar
  32. J. Zhu, H. Zeng, S. Liao, Z. Lei, C. Cai, and L. Zheng. 2017. Deep hybrid similarity learning for person re-identification. arXiv preprint arXiv:1702.04858.Google ScholarGoogle Scholar
  33. S. S. Negi and Y. S. Bhandari. 2014. A hybrid approach to image enhancement using contrast stretching on image sharpening and the analysis of various cases arising using histogram. In Proceedings of the IEEE ICRAIE. 1--6.Google ScholarGoogle Scholar
  34. B. Xu, Y. Zhuang, H. Tang, and L. Zhang. 2010. Object-based multilevel contrast stretching method for image enhancement. IEEE Trans. Consum. Electron. 56, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. N. Dalal and B. Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE CVPR, vol. 1. 886--893. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele. 2014. 2D human pose estimation: New benchmark and state of the art analysis. In Proceedings of the IEEE CVPR. 3686--3693. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. S. Johnson and M. Everingham. 2010. Clustered pose and nonlinear appearance models for human pose estimation. In Proceedings of the BMVC.Google ScholarGoogle Scholar
  38. J. Canny. 1986. A computational approach to edge detection. IEEE TPAMI 6, 679--698. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. R. M. Haralick, K. Shanmugam, and I. Dinstein. 1973. Textural features of image classification. IEEE Trans. Syst., Man Cybernetics. SMC-3, no. 6.Google ScholarGoogle ScholarCross RefCross Ref
  40. H. B. Kekre, S. D. Thepade, T. K. Sarode, and V. Suryawanshi. 2010. Image retrieval using texture features extracted from GLCM, LBG and KPE. IJCTE 2, 5, 695.Google ScholarGoogle Scholar
  41. D. G. Lowe. 1999. Object recognition from local scale-invariant features. In Proceedings of the IEEE ICCV, vol. 2. 1150--1157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. D. Gray, S. Brennan, and H. Tao. 2007. Evaluating appearance models for recognition, reacquisition, and tracking, IEEE PETS 3, 5, 1--7.Google ScholarGoogle Scholar
  43. R. Zhao, W. Ouyang, and X. Wang. 2013. Unsupervised salience learning for person re-identification. In Proceedings of the IEEE CVPR. 3586--3593. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. S. Paisitkriangkrai, C. Shen, and A. van den Hengel. 2015. Learning to rank in person re-identification with metric ensembles. In Proceedings of the IEEE CVPR. 1846--1855.Google ScholarGoogle Scholar
  45. S. Liao, Y. Hu, X. Zhu, and S. Z. Li. 2015. Person re-identification by local maximal occurrence representation and metric learning. In Proceedings of the CVPR. 2197--2206.Google ScholarGoogle Scholar
  46. C. Jose and F. Fleuret. 2016. Scalable metric learning via weighted approximate rank component analysis. In Proceedings of the ECCV. 875--890.Google ScholarGoogle Scholar
  47. N. Martinel, A. Das, C. Micheloni, and A. K. RoyChowdhury. 2016. Temporal model adaptation for person re-identification. In Proceedings of the ECCV. 858--877.Google ScholarGoogle Scholar
  48. L. Zhang, T. Xiang, and S. Gong. 2016. Learning a discriminative null space for person re-identification. In Proceedings of the CVPR. 1239--1248.Google ScholarGoogle Scholar
  49. H. Wang, S. Gong, X. Zhu, and T. Xiang. 2016. Human-in-the-loop person re-identification. In Proceedings of the ECCV. 405--422.Google ScholarGoogle Scholar
  50. L. Wu, C. Shen, and A. V. D. Hengel. 2016. Personnet: Person re-identification with deep convolutional neural networks. arXiv preprint arXiv:1601.07255.Google ScholarGoogle Scholar
  51. T. Xiao, H. Li, W. Ouyang, and X. Wang. 2016. Learning deep feature representations with domain guided dropout for person re-identification. In Proceedings of the IEEE CVPR. 1249--1258.Google ScholarGoogle Scholar
  52. R. R. Varior, M. Haloi, and G. Wang. 2016. Gated siamese convolutional neural network architecture for human re-identification. In Proceedings of the ECCV. 791--808.Google ScholarGoogle Scholar
  53. R. R. Varior, B. Shuai, J. Lu, D. Xu, and G. Wang. 2016. A siamese long short-term memory architecture for human re-identification. In Proceedings of the ECCV. 135--153.Google ScholarGoogle Scholar
  54. S. Liao and S. Z. Li. 2015. Efficient PSD constrained asymmetric metric learning for person re-identification. In Proceedings of the ICCV. 3685--3693. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. F. Wang, W. Zuo, L. Lin, D. Zhang, and L. Zhang. 2016. Joint learning of single-image and cross-image representations for person re-identification. In Proceedings of the CVPR.Google ScholarGoogle Scholar
  56. C. Su, F. Yang, S. Zhang, Q. Tian, L. S. Davis, and W. Gao. 2017. Multi-task learning with low rank attribute embedding for multi-camera person re-identification. In Proceedings of the IEEE TPAMI.Google ScholarGoogle Scholar
  57. R. N. Rodrigues, G. N. Schroeder, J. J. Corso, and V. Govindaraju. 2009. Unconstrained face recognition using MRF priors and manifold traversing. In Proceedings of the IEEE BTAS. 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. X. He, S. Yan, Y. Hu, P. Niyogi, and H.-J. Zhang. 2005. Face recognition using Laplacian faces. IEEE TPAMI 27, 3, 328--340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. X. Wang, Tony X. Han, and S. Yan. 2009. An HOG-LBP human detector with partial occlusion handling. In Proceedings of the IEEE ICCV. 32--39.Google ScholarGoogle Scholar
  60. W. Chen and J. J. Corso. 2015. Action detection by implicit intentional motion clustering. In Proceedings of the IEEE ICCV. 3298--3306. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE TPAMI 32, 9, 1627--1645. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. D. Tao, X. Li, S. J. Maybank, and X. Wu. 2006. Human carrying status in visual surveillance. In Proceedings of the IEEE CVPR, vol. 2. 1670--1677. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. E. Zhang, Y. Zhao, and W. Xiong. 2010. Active energy image plus 2DLPP for gait recognition. Signal Process. 90, 7, 2295--2302. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. D. Damen and D. Hogg. 2012. Detecting carried objects from sequences of walking pedestrians. IEEE TPAMI 34, 6, 1056--1067. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. J. Man and B. Bhanu. 2006. Individual recognition using gait energy image. IEEE TPAMI 28, 2, 316--322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. L. Lee and W. Eric L. Grimson. 2002. Gait analysis for recognition and classification. In Proceedings of the IEEE FG. 155--162. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. SKEPRID: Pose and Illumination Change-Resistant Skeleton-Based Person Re-Identification

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 4
          Special Section on Deep Learning for Intelligent Multimedia Analytics
          November 2018
          221 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3282485
          Issue’s Table of Contents

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 10 October 2018
          • Accepted: 1 July 2018
          • Revised: 1 May 2018
          • Received: 1 January 2018
          Published in tomm Volume 14, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!