Abstract
Previous image parsing methods usually model the problem in a conditional random field which describes a statistical model learned from a training dataset and then processes a query image using the conditional probability. However, for clothing images, fashion items have a large variety of layering and configuration, and it is hard to learn a certain statistical model of features that apply to general cases. In this article, we take fashion images as an example to show how Markov Random Fields (MRFs) can outperform Conditional Random Fields when the application does not follow a certain statistical model learned from the training data set. We propose a new method for automatically parsing fashion images in high processing efficiency with significantly less training time by applying a modification of MRFs, named reweighted MRF (RW-MRF), which resolves the problem of over smoothing infrequent labels. We further enhance RW-MRF with occlusion prior and background prior to resolve two other common problems in clothing parsing, occlusion, and background spill. Our experimental results indicate that our proposed clothing parsing method significantly improves processing time and training time over state-of-the-art methods, while ensuring comparable parsing accuracy and improving label recall rate.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Enhanced Reweighted MRFs for Efficient Fashion Image Parsing
- Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 11 (Nov 2012), 2274--2282. Google Scholar
Digital Library
- Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 5 (May 2011), 898--916. Google Scholar
Digital Library
- Yuri Boykov and Vladimir Kolmogorov. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9 (Sept. 2004), 1124--1137. Google Scholar
Digital Library
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In CVPR09.Google Scholar
- Jian Dong, Qiang Chen, Xiaohui Shen, Jianchao Yang, and Shuicheng Yan. 2014. Towards unified human parsing and pose estimation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14).IEEE, Washington, DC, 843--850. Google Scholar
Digital Library
- Jian Dong, Qiang Chen, Wei Xia, Zhongyang Huang, and Shuicheng Yan. 2013. A deformable mixture parsing model with parselets. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV’13). IEEE Computer Society, Washington, DC, 3408--3415. Google Scholar
Digital Library
- Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res. 9 (June 2008), 1871--1874. Google Scholar
Digital Library
- Li Fei-Fei, R. Fergus, and P. Perona. 2006. One-shot learning of object categories. IEEE Trans. Pattern Anal. Machine Intell. 28, 4 (April 2006), 594--611. Google Scholar
Digital Library
- Basela Hasan and David Hogg. 2010. Segmentation using deformable spatial priors with application to clothing. In Proceedings of the British Machine Vision Conference. BMVA Press, 83.1--83.11.Google Scholar
Cross Ref
- Xuming He, Richard S. Zemel, and Miguel Á. Carreira-Perpiñán. 2004. Multiscale conditional random fields for image labeling. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004 (CVPR 2004), Vol. 2. IEEE, Washington, DC, II--695. Google Scholar
Digital Library
- Yannis Kalantidis, Lyndon Kennedy, and Li-Jia Li. 2013. Getting the Look: Clothing recognition and segmentation for automatic product suggestions in everyday photos. In Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval (ICMR’13). ACM, New York, NY, 105--112. Google Scholar
Digital Library
- E. Kim, XiaoLei Huang, and Gang Tan. 2011. Markup SVG: An online content-aware image abstraction and annotation tool. IEEE Trans. Multimed. 13, 5 (Oct 2011), 993--1006. Google Scholar
Digital Library
- Thomas Leung and Jitendra Malik. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vision 43, 1 (2001), 29--44. Google Scholar
Digital Library
- Xiaodan Liang, Si Liu, Xiaohui Shen, Jianchao Yang, Luoqi Liu, Jian Dong, Liang Lin, and Shuicheng Yan. 2015a. Deep human parsing with active template regression. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 12 (2015), 2402--2414. Google Scholar
Digital Library
- Xiaodan Liang, Chunyan Xu, Xiaohui Shen, Jianchao Yang, Si Liu, Jinhui Tang, Liang Lin, and Shuicheng Yan. 2015b. Human parsing with contextualized convolutional neural network. In Proceedings of the IEEE International Conference on Computer Vision. 1386--1394.Google Scholar
Digital Library
- Si Liu, Jiashi Feng, C. Domokos, Hui Xu, Junshi Huang, Zhenzhen Hu, and Shuicheng Yan. 2014. Fashion parsing with weak color-category labels. IEEE Trans. Multimed. 16, 1 (Jan 2014), 253--265.Google Scholar
Cross Ref
- Si Liu, Xiaodan Liang, Luoqi Liu, Xiaohui Shen, Jianchao Yang, Changsheng Xu, Liang Lin, Xiaochun Cao, and Shuicheng Yan. 2015. Matching-cnn meets KNN: Quasi-parametric human parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1419--1427.Google Scholar
Cross Ref
- Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, and William T. Freeman. 2008. LabelMe: A database and web-based tool for image annotation. Int. J. Comput. Vision 77, 1--3 (May 2008), 157--173. Google Scholar
Digital Library
- János Schanda. 2007. Colorimetry: Understanding the CIE System. John Wiley & Sons, New York, NY.Google Scholar
Cross Ref
- Jamie Shotton, John Winn, Carsten Rother, and Antonio Criminisi. 2006. TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In Proceedings of the 9th European Conference on Computer Vision—Volume Part I (ECCV’06). Springer-Verlag, Berlin, 1--15. DOI:http://dx.doi.org/10.1007/11744023_1 Google Scholar
Digital Library
- Joseph Tighe and Svetlana Lazebnik. 2010. Superparsing: Scalable nonparametric image parsing with superpixels. In Computer Vision—ECCV 2010. Springer, Berlin, 352--365. Google Scholar
Digital Library
- Zhuowen Tu, Xiangrong Chen, Alan L. Yuille, and Song-Chun Zhu. 2005. Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vision 63, 2 (2005), 113--140. Google Scholar
Digital Library
- Yichen Wei, Fang Wen, Wangjiang Zhu, and Jian Sun. 2012. Geodesic saliency using background priors. In Proceedings of the 12th European Conference on Computer Vision—Volume Part III (ECCV’12). Springer-Verlag, Berlin, 29--42. Google Scholar
Digital Library
- John Winn and Nebojsa Jojic. 2005. Locus: Learning object classes with unsupervised segmentation. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). Vol. 1. IEEE, Washigton, DC, 756--763. Google Scholar
Digital Library
- Qiong Wu, Rui Gao, Xida Chen, and Pierre Boulanger. 2014. Tagging driven by interactive image discovery: Tagging-tracking-learning. In Proceedings of the 2014 IEEE International Symposium on Multimedia (ISM’14). IEEE, Washington, DC, 179--186. Google Scholar
Digital Library
- K. Yamaguchi, M. H. Kiapour, and T. L. Berg. 2013. Paper doll parsing: Retrieving similar styles to parse clothing items. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV’13). IEEE, Washington, DC, 3519--3526. Google Scholar
Digital Library
- Kota Yamaguchi, M. Hadi Kiapour, Luis E. Ortiz, and Tamara L. Berg. 2012. Parsing clothing in fashion photographs. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, Washington, DC, 3570--3577. Google Scholar
Digital Library
- Wei Yang, Ping Luo, and Liang Lin. 2014. Clothing co-parsing by joint image segmentation and labeling. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, Washington, DC, 3182--3189. Google Scholar
Digital Library
- Yi Yang and D. Ramanan. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE Computer Society, Washington, DC, 1385--1392. Google Scholar
Digital Library
- Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip Torr. 2015. Conditional random fields as recurrent neural networks. arXiv:1502.03240 (2015).Google Scholar
Index Terms
Enhanced Reweighted MRFs for Efficient Fashion Image Parsing
Recommendations
A new localized superpixel Markov random field for image segmentation
ICME'09: Proceedings of the 2009 IEEE international conference on Multimedia and ExpoIn this paper, we present a novel localized Markov random field (MRF) method based on superpixels for region segmentation. Early vision problems could be formulated as pixel labeling using MRF. But the local interaction in MRF is limited to pixel label ...
Learning CRFs for Image Parsing with Adaptive Subgradient Descent
ICCV '13: Proceedings of the 2013 IEEE International Conference on Computer VisionWe propose an adaptive sub gradient descent method to efficiently learn the parameters of CRF models for image parsing. To balance the learning efficiency and performance of the learned CRF models, the parameter learning is iteratively carried out by ...
An HMM/MRF-based stochastic framework for robust vehicle tracking
Shadows of moving objects often obstruct robust visual tracking. In this paper, we present a car tracker based on a hidden Markov model/Markov random field (HMM/MRF)-based segmentation method that is capable of classifying each small region of an image ...






Comments