Abstract
Taking the shoe as a concrete example, we present an innovative product retrieval system that leverages object detection and retrieval techniques to support a brand-new online shopping experience in this article. The system, called Circle & Search, enables users to naturally indicate any preferred product by simply circling the product in images as the visual query, and then returns visually and semantically similar products to the users. The system is characterized by introducing attributes in both the detection and retrieval of the shoe. Specifically, we first develop an attribute-aware part-based shoe detection model. By maintaining the consistency between shoe parts and attributes, this shoe detector has the ability to model high-order relations between parts and thus the detection performance can be enhanced. Meanwhile, the attributes of this detected shoe can also be predicted as the semantic relations between parts. Based on the result of shoe detection, the system ranks all the shoes in the repository using an attribute refinement retrieval model that takes advantage of query-specific information and attribute correlation to provide an accurate and robust shoe retrieval. To evaluate this retrieval system, we build a large dataset with 17,151 shoe images, in which each shoe is annotated with 10 shoe attributes e.g., heel height, heel shape, sole shape, etc.). According to the experimental result and the user study, our Circle & Search system achieves promising shoe retrieval performance and thus significantly improves the users' online shopping experience.
- Relja Arandjelovic and Andrew Zisserman. 2011. Smooth object retrieval using a bag of boundaries. In Proceedings of the International Conference on Computer Vision (ICCV'11). IEEE, 375--382. Google Scholar
Digital Library
- Tamara L. Berg, Alexander C. Berg, and Jonathan Shih. 2010. Automatic attribute discovery and characterization from noisy web data. In Proceedings of the European Conference on Computer Vision (ECCV'10). Springer, 663--676. Google Scholar
Digital Library
- Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. 2011. Describing people: A poselet-based approach to attribute classification. In Proceedings of the International Conference on Computer Vision (ICCV'11). IEEE, 1543--1550. Google Scholar
Digital Library
- Huizhong Chen, Andrew Gallagher, and Bernd Girod. 2012. Describing clothing by semantic attributes. In Proceedings of the European Conference on Computer Vision (ECCV'12). Springer, 609--623. Google Scholar
Digital Library
- Navneet Dalal and Bill Triggs. 2005. INRIA person dataset. http://pascal.inrialpes.fr/data/human.Google Scholar
- Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09). 1778--1785.Google Scholar
Cross Ref
- Pedro Felzenszwalb and Daniel Huttenlocher. 2004. Distance transforms of sampled functions. Tech. rep., Department of Computing and Information Science, Cornell. http://www.cs.cornell.edu/~dph/papers/dt.pdf.Google Scholar
- Pedro Felzenszwalb, Ross B. Girshick, David Mcallester, and Deva Ramanan. 2010. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intel. 32, 9, 1627--1645. Google Scholar
Digital Library
- Vittorio Ferrari and Andrew Zisserman. 2008. Learning visual attributes. In Proceedings of the Neural Information Processing Systems Conference (NIPS'08).Google Scholar
- Junfeng He, Jinyuan Feng, Xianglong Liu, Tao Cheng, Tai-Hsu Lin, Hyunjin Chung, and Shih-Fu Chang. 2012. Mobile product search with bag of hash bits and boundary reranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12). 3005--3012. Google Scholar
Digital Library
- Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intel. 33, 1, 117--128. Google Scholar
Digital Library
- Hongwen Kang, Martial Hebert, Alexei A. Efros, and Takeo Kanade. 2012. Connecting missing links: Object discovery from sparse observations using 5 million product images. In Proceedings of the European Conference on Computer Vision (ECCV'12). Springer, 794--807. Google Scholar
Digital Library
- Adriana Kovashka, Devi Parikh, and Kristen Grauman. 2012. WhittleSearch: Image search with relative attribute feedback. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12). 2973--2980. Google Scholar
Digital Library
- Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar. 2009. Attribute and simile classifiers for face verification. In Proceedings of the International Conference on Computer Vision (ICCV'09). IEEE, 365--372.Google Scholar
- Si Liu, Zheng Song, Guangcan Liu, Changsheng Xu, Hanqing Lu, and Shuicheng Yan. 2012. Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12). 3330--3337. Google Scholar
Digital Library
- Shiyang Lu, Tao Mei, Jingdong Wang, Jian Zhang, Zhiyong Wang, David Dagan Feng, Jian-Tao Sun, and Shipeng Li. 2012. Browse-to-search. In Proceedings of the 20th ACM International Conference on Multimedia (ACM-MM'12). ACM Press, New York, 1323--1324. Google Scholar
Digital Library
- Devi Parikh and Kristen Grauman. 2011. Interactively building a discriminative vocabulary of nameable attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'11). 1681--1688. Google Scholar
Digital Library
- Xiaohui Shen, Zhe Lin, Jonathan Brandt, and Ying Wu. 2012. Mobile product image search by automatic query object extraction. In Proceedings of the European Conference on Computer Vision (ECCV'12). Springer, 114--127. Google Scholar
Digital Library
- Behjat Siddiquie, Rogerio Schmidt Feris, and Larry S. Davis. 2011. Image ranking and retrieval based on multi-attribute queries. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'11). 801--808. Google Scholar
Digital Library
- Yang Wang and Greg Mori. 2010. A discriminative latent model of object classes and attributes. In Proceedings of the European Conference on Computer Vision (ECCV'10). Springer, 155--168. Google Scholar
Digital Library
- Yi Yang and Deva Ramanan. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'11). 1385--1392. Google Scholar
Digital Library
Index Terms
Circle & Search: Attribute-Aware Shoe Retrieval
Recommendations
DeepShoe: An improved Multi-Task View-invariant CNN for street-to-shop shoe retrieval
AbstractThe difficulty of describing a shoe item seeing on street with text for online shopping demands an image-based retrieval solution. We call this problem street-to-shop shoe retrieval, whose goal is to find exactly the same shoe in the ...
Possibility of guiding arm movement in circle drawing
SMC'09: Proceedings of the 2009 IEEE international conference on Systems, Man and CyberneticsWe tried to guide human action using galvanic vestibular stimulation (GVS). GVS has a possibility of human behavior guidance without any attention. We tried to guide the trajectory of the subjects' hands when as the continuously drew circles. Previous ...
Attribute Conjunction Learning with Recurrent Neural Network
ECML PKDD 2016: European Conference on Machine Learning and Knowledge Discovery in Databases - Volume 9851Searching images with multi-attribute queries shows practical significance in various real world applications. The key problem in this task is how to effectively and efficiently learn from the conjunction of query attributes. In this paper, we propose ...






Comments