Abstract
Although Deep Convolutional Neural Networks (DCNNs) facilitate the evolution of 3D human pose estimation, ambiguity remains the most challenging problem in such tasks. Inspired by the Human Perception Mechanism (HPM), we propose an image-to-pose coding method to fill the gap between image cues and 3D poses, thereby alleviating the ambiguity of 3D human pose estimation. First, in 3D pose space, we divide the whole 3D pose space into multiple subregions named pose codes, turning a disambiguation problem into a classification problem. The proposed coding mechanism covers multiple camera views and provides a complete description for 3D pose space. Second, it is noteworthy that the articulated structure of the human body lies on a sophisticated product manifold and the error accumulation in the chain structure will undoubtedly affect the coding performance. Therefore, in image space, we extract the image cues from independent local image patches rather than the whole image. The mapping relationship between image cues and 3D pose codes is established by a set of DCNNs. The image-to-pose coding method transforms the implicit image cues into explicit constraints. Finally, the image-to-pose coding method is integrated into a linear matching mechanism to construct a 3D pose estimation method that effectively alleviates the ambiguity. We conduct extensive experiments on widely used public benchmarks. The experimental results show that our method effectively alleviates the ambiguity in 3D pose recovery and is robust to the variations of view.
- Ijaz Akhter and Michael J. Black. 2015. Pose-conditioned joint angle limits for 3D human pose reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1446--1455.Google Scholar
- Sikandar Amin, Mykhaylo Andriluka, Marcus Rohrbach, and Bernt Schiele. 2013. Multi-view pictorial structures for 3D human pose estimation. In Bmvc. Citeseer.Google Scholar
- Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2014. 2D human pose estimation: New benchmark and state of the art analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 3686--3693.Google Scholar
Digital Library
- Mykhaylo Andriluka, Stefan Roth, and Bernt Schiele. 2010. Monocular 3D pose estimation and tracking by detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). IEEE, 623--630.Google Scholar
Cross Ref
- Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J. Black. 2016. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In European Conference on Computer Vision. Springer, 561--578.Google Scholar
- Stephen Boyd and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press.Google Scholar
Digital Library
- Ching-Hang Chen and Deva Ramanan. 2017. 3D human pose estimation= 2D pose estimation+ matching. In Conference on Computer Vision and Pattern Recognition. IEEE, 7035--7043.Google Scholar
- Mingliang Chen, Weiyao Lin, and Bing Zhou. 2015. A real-time virtual dressing system with RGB-D camera. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA’15). IEEE, 1041--1044.Google Scholar
Cross Ref
- Xiaochuan Fan, Kang Zheng, Youjie Zhou, and Song Wang. 2014. Pose locality constrained representation for 3D human pose reconstruction. In European Conference on Computer Vision. Springer, 174--188.Google Scholar
Cross Ref
- Haoshu Fang, Yuanlu Xu, Wenguan Wang, Xiaobai Liu, and Song-Chun Zhu. 2018. Learning pose grammar to encode human body configuration for 3D pose estimation. In The National Conference on Artificial Intelligence. AAAI, 6821--6828.Google Scholar
Cross Ref
- Albert Haque, Boya Peng, Zelun Luo, Alexandre Alahi, Serena Yeung, and Li Fei-Fei. 2016. Towards viewpoint invariant 3D human pose estimation. In European Conference on Computer Vision. Springer, 160--177.Google Scholar
Cross Ref
- Michael Hofmann and Dariu M. Gavrila. 2012. Multi-view 3D human pose estimation in complex environment. International Journal of Computer Vision 96, 1, 103--124.Google Scholar
Digital Library
- Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. 2014. Human3. 6m: Large scale datasets and predictive methods for 3D human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 7, 1325--1339.Google Scholar
Digital Library
- Sam Johnson and Mark Everingham. 2010. Clustered pose and nonlinear appearance models for human pose estimation. In The British Machine Vision Conference, Vol. 2. Springer, 5 pages.Google Scholar
Cross Ref
- Ilya Kostrikov and Juergen Gall. 2014. Depth sweep regression forests for estimating 3D human pose from images. In The British Machine Vision Conference, Vol. 1. Springer, 5 pages.Google Scholar
Cross Ref
- Hsi-Jian Lee, Chen Zen, et al. 1985. Determination of 3D human-body postures from a single view. Computer Vision Graphics and Image Processing 30, 2, 148--168.Google Scholar
Cross Ref
- Sijin Li and Antoni B. Chan. 2014. 3D human pose estimation from monocular images with deep convolutional neural network. In Asian Conference on Computer Vision. Springer, 332--347.Google Scholar
- Jun Liu, Henghui Ding, Amir Shahroudy, Ling-Yu Duan, Xudong Jiang, Gang Wang, and Alex Kot Chichung. 2019. Feature boosting network for 3D pose estimation. In Transactions on Pattern Analysis and Machine Intelligence. IEEE, 1--11.Google Scholar
- Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics 34, 248 (2015), 6 pages.Google Scholar
Digital Library
- Julieta Martinez, Rayat Hossain, Javier Romero, and James J. Little. 2017. A simple yet effective baseline for 3D human pose estimation. In International Conference on Computer Vision. IEEE, 2640--2649.Google Scholar
- Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, and Christian Theobalt. 2017. Monocular 3D human pose estimation in the wild using improved CNN supervision. In International Conference on 3D Vision (3DV’17). IEEE, 506--516.Google Scholar
Cross Ref
- Francesc Moreno-Noguer. 2017. 3D human pose estimation from a single image via distance matrix regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). IEEE, 1561--1570.Google Scholar
Cross Ref
- Alejandro Newell, Kaiyu Yang, and Jia Deng. 2016. Stacked hourglass networks for human pose estimation. In European Conference on Computer Vision. Springer, 483--499.Google Scholar
Cross Ref
- Bruce Xiaohan Nie, Ping Wei, and Song-Chun Zhu. 2017. Monocular 3D human pose estimation by predicting depth on joints. In IEEE International Conference on Computer Vision. IEEE, 3467--3475.Google Scholar
Cross Ref
- Vasu Parameswaran and Rama Chellappa. 2004. View independent human body pose estimation from a single perspective image. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04). Vol. 2. IEEE, II--II.Google Scholar
Cross Ref
- G. Pavlakos, X. Zhou, K. G. Derpanis, et al. 2017. Coarse-to-fine volumetric prediction for single-image 3D human pose. In Conference on Computer Vision and Pattern Recognition. IEEE, 7025--7034.Google Scholar
Cross Ref
- Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2012. Reconstructing 3D human pose from 2D image landmarks. In European Conference on Computer Vision. Springer, 573--586.Google Scholar
Digital Library
- Grégory Rogez and Cordelia Schmid. 2016. Mocap-guided data augmentation for 3D pose estimation in the wild. In Advances in Neural Information Processing Systems. MIT Press, 3108--3116.Google Scholar
- Leonid Sigal, Alexandru O. Balan, and Michael J. Black. 2010. Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision 87, 1--2, 4.Google Scholar
Digital Library
- Leonid Sigal, Michael Isard, Horst Haussecker, and Michael J. Black. 2012. Loose-limbed people: Estimating 3D human pose and motion using non-parametric belief propagation. International Journal of Computer Vision 98, 1, 15--48.Google Scholar
Digital Library
- Edgar Simo-Serra, Arnau Ramisa, Guillem Alenyà, Carme Torras, and Francesc Moreno-Noguer. 2012. Single image 3D human pose estimation from noisy observations. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2673--2680.Google Scholar
Cross Ref
- Yong Su, Zhiyong Feng, Jianhai Zhang, Weilong Peng, and Meng Xing. 2018. Sequential articulated motion reconstruction from a monocular image sequence. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 1s, 23.Google Scholar
Digital Library
- Camillo J. Taylor. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Computer Vision and Image Understanding 80, 3, 349--363.Google Scholar
Digital Library
- Bugra Tekin, Artem Rozantsev, Vincent Lepetit, and Pascal Fua. 2016. Direct prediction of 3D body poses from motion compensated sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 991--1000.Google Scholar
Cross Ref
- Denis Tome, Christopher Russell, and Lourdes Agapito. 2017. Lifting from the deep: Convolutional 3D pose estimation from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2500--2509.Google Scholar
Cross Ref
- Xiaolin K. Wei and Jinxiang Chai. 2009. Modeling 3D human poses from uncalibrated monocular images. In IEEE 12th International Conference on Computer Vision. IEEE, 1873--1880.Google Scholar
- Jiajun Wu, Tianfan Xue, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, and William T. Freeman. 2016. Single image 3D interpreter network. In European Conference on Computer Vision. Springer, 365--382.Google Scholar
- Wei Yang, Wanli Ouyang, Xiaolong Wang, Jimmy Ren, Hongsheng Li, and Xiaogang Wang. 2018. 3D human pose estimation in the wild by adversarial learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 5255--5264.Google Scholar
Cross Ref
- Hashim Yasin, Umar Iqbal, Bjorn Kruger, Andreas Weber, and Juergen Gall. 2016. A dual-source approach for 3D pose estimation from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4948--4956.Google Scholar
Cross Ref
- Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, and Yichen Wei. 2017. Towards 3D human pose estimation in the wild: A weakly-supervised approach. In IEEE International Conference on Computer Vision. IEEE, 398--407.Google Scholar
Cross Ref
- Xiaowei Zhou, Menglong Zhu, Spyridon Leonardos, Konstantinos G. Derpanis, and Kostas Daniilidis. 2016. Sparseness meets deepness: 3D human pose estimation from monocular video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4966--4975.Google Scholar
Cross Ref
- Xiaowei Zhou, Menglong Zhu, Georgios Pavlakos, Spyridon Leonardos, Konstantinos G. Derpanis, and Kostas Daniilidis. 2019. Monocap: Monocular human motion capture using a cnn coupled with a geometric prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 4 (2019), 901--914.Google Scholar
Digital Library
Index Terms
An Image Cues Coding Approach for 3D Human Pose Estimation
Recommendations
Image-Based Synthesis for Deep 3D Human Pose Estimation
This paper addresses the problem of 3D human pose estimation in the wild. A significant challenge is the lack of training data, i.e., 2D images of humans annotated with 3D poses. Such data is necessary to train state-of-the-art CNN architectures. Here, ...
3D Human pose estimation
Review of the recent literature in 3D human pose estimation from RGB images and videos.Release of a challenging, publicly available, 3D pose estimation synthetic dataset.Extensive experimental evaluation of some representative state-of-the-art methods. ...
Bayesian 3d human body pose tracking from depth image sequences
ACCV'09: Proceedings of the 9th Asian conference on Computer Vision - Volume Part IIThis paper addresses the problem of accurate and robust tracking of 3D human body pose from depth image sequences. Recovering the large number of degrees of freedom in human body movements from depth image sequence is challenging due to the need to ...






Comments