skip to main content
research-article

An Image Cues Coding Approach for 3D Human Pose Estimation

Published:16 December 2019Publication History
Skip Abstract Section

Abstract

Although Deep Convolutional Neural Networks (DCNNs) facilitate the evolution of 3D human pose estimation, ambiguity remains the most challenging problem in such tasks. Inspired by the Human Perception Mechanism (HPM), we propose an image-to-pose coding method to fill the gap between image cues and 3D poses, thereby alleviating the ambiguity of 3D human pose estimation. First, in 3D pose space, we divide the whole 3D pose space into multiple subregions named pose codes, turning a disambiguation problem into a classification problem. The proposed coding mechanism covers multiple camera views and provides a complete description for 3D pose space. Second, it is noteworthy that the articulated structure of the human body lies on a sophisticated product manifold and the error accumulation in the chain structure will undoubtedly affect the coding performance. Therefore, in image space, we extract the image cues from independent local image patches rather than the whole image. The mapping relationship between image cues and 3D pose codes is established by a set of DCNNs. The image-to-pose coding method transforms the implicit image cues into explicit constraints. Finally, the image-to-pose coding method is integrated into a linear matching mechanism to construct a 3D pose estimation method that effectively alleviates the ambiguity. We conduct extensive experiments on widely used public benchmarks. The experimental results show that our method effectively alleviates the ambiguity in 3D pose recovery and is robust to the variations of view.

References

  1. Ijaz Akhter and Michael J. Black. 2015. Pose-conditioned joint angle limits for 3D human pose reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1446--1455.Google ScholarGoogle Scholar
  2. Sikandar Amin, Mykhaylo Andriluka, Marcus Rohrbach, and Bernt Schiele. 2013. Multi-view pictorial structures for 3D human pose estimation. In Bmvc. Citeseer.Google ScholarGoogle Scholar
  3. Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2014. 2D human pose estimation: New benchmark and state of the art analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 3686--3693.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mykhaylo Andriluka, Stefan Roth, and Bernt Schiele. 2010. Monocular 3D pose estimation and tracking by detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). IEEE, 623--630.Google ScholarGoogle ScholarCross RefCross Ref
  5. Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black. 2016. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In European Conference on Computer Vision. Springer, 561--578.Google ScholarGoogle Scholar
  6. Stephen Boyd and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Ching-Hang Chen and Deva Ramanan. 2017. 3D human pose estimation= 2D pose estimation+ matching. In Conference on Computer Vision and Pattern Recognition. IEEE, 7035--7043.Google ScholarGoogle Scholar
  8. Mingliang Chen, Weiyao Lin, and Bing Zhou. 2015. A real-time virtual dressing system with RGB-D camera. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA’15). IEEE, 1041--1044.Google ScholarGoogle ScholarCross RefCross Ref
  9. Xiaochuan Fan, Kang Zheng, Youjie Zhou, and Song Wang. 2014. Pose locality constrained representation for 3D human pose reconstruction. In European Conference on Computer Vision. Springer, 174--188.Google ScholarGoogle ScholarCross RefCross Ref
  10. Haoshu Fang, Yuanlu Xu, Wenguan Wang, Xiaobai Liu, and Song-Chun Zhu. 2018. Learning pose grammar to encode human body configuration for 3D pose estimation. In The National Conference on Artificial Intelligence. AAAI, 6821--6828.Google ScholarGoogle ScholarCross RefCross Ref
  11. Albert Haque, Boya Peng, Zelun Luo, Alexandre Alahi, Serena Yeung, and Li Fei-Fei. 2016. Towards viewpoint invariant 3D human pose estimation. In European Conference on Computer Vision. Springer, 160--177.Google ScholarGoogle ScholarCross RefCross Ref
  12. Michael Hofmann and Dariu M. Gavrila. 2012. Multi-view 3D human pose estimation in complex environment. International Journal of Computer Vision 96, 1, 103--124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. 2014. Human3. 6m: Large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 7, 1325--1339.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Sam Johnson and Mark Everingham. 2010. Clustered pose and nonlinear appearance models for human pose estimation. In The British Machine Vision Conference, Vol. 2. Springer, 5 pages.Google ScholarGoogle ScholarCross RefCross Ref
  15. Ilya Kostrikov and Juergen Gall. 2014. Depth sweep regression forests for estimating 3D human pose from images. In The British Machine Vision Conference, Vol. 1. Springer, 5 pages.Google ScholarGoogle ScholarCross RefCross Ref
  16. Hsi-Jian Lee, Chen Zen, et al. 1985. Determination of 3D human-body postures from a single view. Computer Vision Graphics and Image Processing 30, 2, 148--168.Google ScholarGoogle ScholarCross RefCross Ref
  17. Sijin Li and Antoni B. Chan. 2014. 3D human pose estimation from monocular images with deep convolutional neural network. In Asian Conference on Computer Vision. Springer, 332--347.Google ScholarGoogle Scholar
  18. Jun Liu, Henghui Ding, Amir Shahroudy, Ling-Yu Duan, Xudong Jiang, Gang Wang, and Alex Kot Chichung. 2019. Feature boosting network for 3D pose estimation. In Transactions on Pattern Analysis and Machine Intelligence. IEEE, 1--11.Google ScholarGoogle Scholar
  19. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics 34, 248 (2015), 6 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Julieta Martinez, Rayat Hossain, Javier Romero, and James J. Little. 2017. A simple yet effective baseline for 3D human pose estimation. In International Conference on Computer Vision. IEEE, 2640--2649.Google ScholarGoogle Scholar
  21. Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, and Christian Theobalt. 2017. Monocular 3D human pose estimation in the wild using improved CNN supervision. In International Conference on 3D Vision (3DV’17). IEEE, 506--516.Google ScholarGoogle ScholarCross RefCross Ref
  22. Francesc Moreno-Noguer. 2017. 3D human pose estimation from a single image via distance matrix regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, 1561--1570.Google ScholarGoogle ScholarCross RefCross Ref
  23. Alejandro Newell, Kaiyu Yang, and Jia Deng. 2016. Stacked hourglass networks for human pose estimation. In European Conference on Computer Vision. Springer, 483--499.Google ScholarGoogle ScholarCross RefCross Ref
  24. Bruce Xiaohan Nie, Ping Wei, and Song-Chun Zhu. 2017. Monocular 3D human pose estimation by predicting depth on joints. In IEEE International Conference on Computer Vision. IEEE, 3467--3475.Google ScholarGoogle ScholarCross RefCross Ref
  25. Vasu Parameswaran and Rama Chellappa. 2004. View independent human body pose estimation from a single perspective image. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04). Vol. 2. IEEE, II--II.Google ScholarGoogle ScholarCross RefCross Ref
  26. G. Pavlakos, X. Zhou, K. G. Derpanis, et al. 2017. Coarse-to-fine volumetric prediction for single-image 3D human pose. In Conference on Computer Vision and Pattern Recognition. IEEE, 7025--7034.Google ScholarGoogle ScholarCross RefCross Ref
  27. Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2012. Reconstructing 3D human pose from 2D image landmarks. In European Conference on Computer Vision. Springer, 573--586.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Grégory Rogez and Cordelia Schmid. 2016. Mocap-guided data augmentation for 3D pose estimation in the wild. In Advances in Neural Information Processing Systems. MIT Press, 3108--3116.Google ScholarGoogle Scholar
  29. Leonid Sigal, Alexandru O. Balan, and Michael J. Black. 2010. Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision 87, 1--2, 4.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Leonid Sigal, Michael Isard, Horst Haussecker, and Michael J. Black. 2012. Loose-limbed people: Estimating 3D human pose and motion using non-parametric belief propagation. International Journal of Computer Vision 98, 1, 15--48.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Edgar Simo-Serra, Arnau Ramisa, Guillem Alenyà, Carme Torras, and Francesc Moreno-Noguer. 2012. Single image 3D human pose estimation from noisy observations. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2673--2680.Google ScholarGoogle ScholarCross RefCross Ref
  32. Yong Su, Zhiyong Feng, Jianhai Zhang, Weilong Peng, and Meng Xing. 2018. Sequential articulated motion reconstruction from a monocular image sequence. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 1s, 23.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Camillo J. Taylor. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Computer Vision and Image Understanding 80, 3, 349--363.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Bugra Tekin, Artem Rozantsev, Vincent Lepetit, and Pascal Fua. 2016. Direct prediction of 3D body poses from motion compensated sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 991--1000.Google ScholarGoogle ScholarCross RefCross Ref
  35. Denis Tome, Christopher Russell, and Lourdes Agapito. 2017. Lifting from the deep: Convolutional 3D pose estimation from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2500--2509.Google ScholarGoogle ScholarCross RefCross Ref
  36. Xiaolin K. Wei and Jinxiang Chai. 2009. Modeling 3D human poses from uncalibrated monocular images. In IEEE 12th International Conference on Computer Vision. IEEE, 1873--1880.Google ScholarGoogle Scholar
  37. Jiajun Wu, Tianfan Xue, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, and William T. Freeman. 2016. Single image 3D interpreter network. In European Conference on Computer Vision. Springer, 365--382.Google ScholarGoogle Scholar
  38. Wei Yang, Wanli Ouyang, Xiaolong Wang, Jimmy Ren, Hongsheng Li, and Xiaogang Wang. 2018. 3D human pose estimation in the wild by adversarial learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 5255--5264.Google ScholarGoogle ScholarCross RefCross Ref
  39. Hashim Yasin, Umar Iqbal, Bjorn Kruger, Andreas Weber, and Juergen Gall. 2016. A dual-source approach for 3D pose estimation from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4948--4956.Google ScholarGoogle ScholarCross RefCross Ref
  40. Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, and Yichen Wei. 2017. Towards 3D human pose estimation in the wild: A weakly-supervised approach. In IEEE International Conference on Computer Vision. IEEE, 398--407.Google ScholarGoogle ScholarCross RefCross Ref
  41. Xiaowei Zhou, Menglong Zhu, Spyridon Leonardos, Konstantinos G. Derpanis, and Kostas Daniilidis. 2016. Sparseness meets deepness: 3D human pose estimation from monocular video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4966--4975.Google ScholarGoogle ScholarCross RefCross Ref
  42. Xiaowei Zhou, Menglong Zhu, Georgios Pavlakos, Spyridon Leonardos, Konstantinos G. Derpanis, and Kostas Daniilidis. 2019. Monocap: Monocular human motion capture using a cnn coupled with a geometric prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 4 (2019), 901--914.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An Image Cues Coding Approach for 3D Human Pose Estimation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!