skip to main content
research-article

Enhanced Reweighted MRFs for Efficient Fashion Image Parsing

Published:08 March 2016Publication History
Skip Abstract Section

Abstract

Previous image parsing methods usually model the problem in a conditional random field which describes a statistical model learned from a training dataset and then processes a query image using the conditional probability. However, for clothing images, fashion items have a large variety of layering and configuration, and it is hard to learn a certain statistical model of features that apply to general cases. In this article, we take fashion images as an example to show how Markov Random Fields (MRFs) can outperform Conditional Random Fields when the application does not follow a certain statistical model learned from the training data set. We propose a new method for automatically parsing fashion images in high processing efficiency with significantly less training time by applying a modification of MRFs, named reweighted MRF (RW-MRF), which resolves the problem of over smoothing infrequent labels. We further enhance RW-MRF with occlusion prior and background prior to resolve two other common problems in clothing parsing, occlusion, and background spill. Our experimental results indicate that our proposed clothing parsing method significantly improves processing time and training time over state-of-the-art methods, while ensuring comparable parsing accuracy and improving label recall rate.

Skip Supplemental Material Section

Supplemental Material

References

  1. Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 11 (Nov 2012), 2274--2282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 5 (May 2011), 898--916. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Yuri Boykov and Vladimir Kolmogorov. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9 (Sept. 2004), 1124--1137. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In CVPR09.Google ScholarGoogle Scholar
  5. Jian Dong, Qiang Chen, Xiaohui Shen, Jianchao Yang, and Shuicheng Yan. 2014. Towards unified human parsing and pose estimation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14).IEEE, Washington, DC, 843--850. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jian Dong, Qiang Chen, Wei Xia, Zhongyang Huang, and Shuicheng Yan. 2013. A deformable mixture parsing model with parselets. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV’13). IEEE Computer Society, Washington, DC, 3408--3415. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res. 9 (June 2008), 1871--1874. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Li Fei-Fei, R. Fergus, and P. Perona. 2006. One-shot learning of object categories. IEEE Trans. Pattern Anal. Machine Intell. 28, 4 (April 2006), 594--611. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Basela Hasan and David Hogg. 2010. Segmentation using deformable spatial priors with application to clothing. In Proceedings of the British Machine Vision Conference. BMVA Press, 83.1--83.11.Google ScholarGoogle ScholarCross RefCross Ref
  10. Xuming He, Richard S. Zemel, and Miguel Á. Carreira-Perpiñán. 2004. Multiscale conditional random fields for image labeling. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004 (CVPR 2004), Vol. 2. IEEE, Washington, DC, II--695. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Yannis Kalantidis, Lyndon Kennedy, and Li-Jia Li. 2013. Getting the Look: Clothing recognition and segmentation for automatic product suggestions in everyday photos. In Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval (ICMR’13). ACM, New York, NY, 105--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. E. Kim, XiaoLei Huang, and Gang Tan. 2011. Markup SVG: An online content-aware image abstraction and annotation tool. IEEE Trans. Multimed. 13, 5 (Oct 2011), 993--1006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Thomas Leung and Jitendra Malik. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vision 43, 1 (2001), 29--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Xiaodan Liang, Si Liu, Xiaohui Shen, Jianchao Yang, Luoqi Liu, Jian Dong, Liang Lin, and Shuicheng Yan. 2015a. Deep human parsing with active template regression. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 12 (2015), 2402--2414. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Xiaodan Liang, Chunyan Xu, Xiaohui Shen, Jianchao Yang, Si Liu, Jinhui Tang, Liang Lin, and Shuicheng Yan. 2015b. Human parsing with contextualized convolutional neural network. In Proceedings of the IEEE International Conference on Computer Vision. 1386--1394.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Si Liu, Jiashi Feng, C. Domokos, Hui Xu, Junshi Huang, Zhenzhen Hu, and Shuicheng Yan. 2014. Fashion parsing with weak color-category labels. IEEE Trans. Multimed. 16, 1 (Jan 2014), 253--265.Google ScholarGoogle ScholarCross RefCross Ref
  17. Si Liu, Xiaodan Liang, Luoqi Liu, Xiaohui Shen, Jianchao Yang, Changsheng Xu, Liang Lin, Xiaochun Cao, and Shuicheng Yan. 2015. Matching-cnn meets KNN: Quasi-parametric human parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1419--1427.Google ScholarGoogle ScholarCross RefCross Ref
  18. Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, and William T. Freeman. 2008. LabelMe: A database and web-based tool for image annotation. Int. J. Comput. Vision 77, 1--3 (May 2008), 157--173. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. János Schanda. 2007. Colorimetry: Understanding the CIE System. John Wiley & Sons, New York, NY.Google ScholarGoogle ScholarCross RefCross Ref
  20. Jamie Shotton, John Winn, Carsten Rother, and Antonio Criminisi. 2006. TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In Proceedings of the 9th European Conference on Computer Vision—Volume Part I (ECCV’06). Springer-Verlag, Berlin, 1--15. DOI:http://dx.doi.org/10.1007/11744023_1 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Joseph Tighe and Svetlana Lazebnik. 2010. Superparsing: Scalable nonparametric image parsing with superpixels. In Computer Vision—ECCV 2010. Springer, Berlin, 352--365. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Zhuowen Tu, Xiangrong Chen, Alan L. Yuille, and Song-Chun Zhu. 2005. Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vision 63, 2 (2005), 113--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yichen Wei, Fang Wen, Wangjiang Zhu, and Jian Sun. 2012. Geodesic saliency using background priors. In Proceedings of the 12th European Conference on Computer Vision—Volume Part III (ECCV’12). Springer-Verlag, Berlin, 29--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. John Winn and Nebojsa Jojic. 2005. Locus: Learning object classes with unsupervised segmentation. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). Vol. 1. IEEE, Washigton, DC, 756--763. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Qiong Wu, Rui Gao, Xida Chen, and Pierre Boulanger. 2014. Tagging driven by interactive image discovery: Tagging-tracking-learning. In Proceedings of the 2014 IEEE International Symposium on Multimedia (ISM’14). IEEE, Washington, DC, 179--186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. K. Yamaguchi, M. H. Kiapour, and T. L. Berg. 2013. Paper doll parsing: Retrieving similar styles to parse clothing items. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV’13). IEEE, Washington, DC, 3519--3526. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Kota Yamaguchi, M. Hadi Kiapour, Luis E. Ortiz, and Tamara L. Berg. 2012. Parsing clothing in fashion photographs. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, Washington, DC, 3570--3577. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Wei Yang, Ping Luo, and Liang Lin. 2014. Clothing co-parsing by joint image segmentation and labeling. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, Washington, DC, 3182--3189. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Yi Yang and D. Ramanan. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE Computer Society, Washington, DC, 1385--1392. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip Torr. 2015. Conditional random fields as recurrent neural networks. arXiv:1502.03240 (2015).Google ScholarGoogle Scholar

Index Terms

  1. Enhanced Reweighted MRFs for Efficient Fashion Image Parsing

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!