Abstract
Face-top coordination, which exists in most clothes-fitting scenarios, is challenging due to varieties of attributes, implicit correlations, and tradeoffs between general preferences and individual preferences. We present a Deep-Based Self-Refined (DBSR) system to simulate face-top coordination based on intuition evaluation. To this end, we first establish a well-coordinated face-top (WCFT) dataset from fashion databases and communities. Then, we use a jointly trained CNN Deep Canonical Correlation Analysis (DCCA) method to bridge the semantic face-top gap based on the WCFT dataset to deal with general preferences. Subsequently, an irrelevance-based Optimum-path Forest (OPF) method is developed to adapt the results to individual preferences iteratively. Experimental results and user study demonstrate the effectiveness of our method.
- Kaori Abe, Teppei Suzuki, Shunya Ueta, Akio Nakamura, Yutaka Satoh, and Hirokatsu Kataoka. 2017.Changing fashion cultures. arXiv preprint arXiv:1703.07920 (2017).Google Scholar
- Stuti Ajmani, Hiranmay Ghosh, Anupama Mallik, and Santanu Chaudhury. 2013.An ontology based personalized garment recommendation system. In Proceedings of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT’13), Vol. 3. IEEE, 17–20. Google Scholar
Digital Library
- Shotaro Akaho. 2006.A kernel method for canonical correlation analysis. arXiv preprint cs/0609071 (2006).Google Scholar
- Ziad Al-Halah, Rainer Stiefelhagen, and Kristen Grauman. 2017.Fashion forward: Forecasting visual style in fashion. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). IEEE, 388–397.Google Scholar
Cross Ref
- Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. 2013.Deep canonical correlation analysis. In Proceedings of the International Conference on Machine Learning (ICML’13). ACM, 1247–1255. Google Scholar
Digital Library
- Jean-Yves Baudouin and Guy Tiberghien. 2004.Symmetry, averageness, and feature size in the facial attractiveness of women. Acta Psychol. 117, 3 (2004), 313–332.Google Scholar
Cross Ref
- Andrea Bottino and Aldo Laurentini. 2010.The analysis of facial beauty: An emerging area of research in pattern analysis. In Proceedings of the International Conference Image Analysis and Recognition. Springer, 425–435. Google Scholar
Digital Library
- Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, and Binqiang Zhao. 2019.POG: Personalized outfit generation for fashion recommendation at Alibaba iFashion. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2662–2670. Google Scholar
Digital Library
- Xu Chen, Hanxiong Chen, Hongteng Xu, Yongfeng Zhang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2019.Personalized fashion recommendation with visual explanations based on multimodal attention network: Towards visually explainable recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 765–774. Google Scholar
Digital Library
- Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005.Learning a similarity metric discriminatively, with application to face verification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1. IEEE, 539–546. Google Scholar
Digital Library
- Andre Tavares Da Silva, Alexandre Xavier Falcão, and Léo Pini Magalhães. 2011.Active learning paradigms for CBIR systems based on optimum-path forest classification. Pattern Recog. 44, 12 (2011), 2971–2978. Google Scholar
Digital Library
- Yael Eisenthal, Gideon Dror, and Eytan Ruppin. 2006.Facial attractiveness: Beauty and the machine. Neural Comput. 18, 1 (2006), 119–142. Google Scholar
Digital Library
- Ruining He, Chunbin Lin, Jianguo Wang, and Julian McAuley. 2016.Sherlock: Sparse hierarchical embeddings for visually-aware one-class collaborative filtering. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI’16). Morgan Kaufmann, 3740–3746. Google Scholar
Digital Library
- Ruining He and Julian McAuley. 2016.VBPR: Visual Bayesian Personalized Ranking from implicit feedback. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. AAAI, 403–410. Google Scholar
Digital Library
- Ruining He, Charles Packer, and Julian McAuley. 2016.Learning compatibility across categories for heterogeneous item recommendation. In Proceedings of the IEEE 16th International Conference on Data Mining (ICDM’16). IEEE, 937–942.Google Scholar
Cross Ref
- Tong He and Yang Hu. 2018.FashionNet: Personalized outfit recommendation with deep neural network. arXiv preprint arXiv:1810.02443 (2018).Google Scholar
- Wanjia He, Weiran Wang, and Karen Livescu. 2017.Multi-view recurrent neural acoustic word embeddings. In Proceedings of the 5th International Conference on Learning Representations (ICLR’17).Google Scholar
- Shintami Chusnul Hidayati, Cheng-Chun Hsu, Yu-Ting Chang, Kai-Lung Hua, Jianlong Fu, and Wen-Huang Cheng. 2018.What dress fits me best? Fashion recommendation on the clothing style for personal body shape. In Proceedings of the 26th ACM International Conference on Multimedia (ACM MM’18). ACM, 438–446. Google Scholar
Digital Library
- Harold Hotelling. 1992.Relations between two sets of variates. Biometrika 28, 3/4 (1936), 321–377.Google Scholar
Cross Ref
- Min Hou, Le Wu, Enhong Chen, Zhi Li, Vincent W. Zheng, and Qi Liu. 2019. Explainable fashion recommendation: A semantic attribute region guided approach. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). Macao, 4681–4688. Google Scholar
Digital Library
- Wei-Lin Hsiao and Kristen Grauman. 2017.Learning the latent “look”: Unsupervised discovery of a style-coherent embedding from fashion images. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). IEEE, 4213–4222.Google Scholar
Cross Ref
- Wei-Lin Hsiao and Kristen Grauman. 2018.Creating capsule wardrobes from fashion images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, 7161–7170.Google Scholar
Cross Ref
- Chia-Wei Hsieh, Chieh-Yun Chen, Chien-Lung Chou, Hong-Han Shuai, Jiaying Liu, and Wen-Huang Cheng. 2019.FashionOn: Semantic-guided image-based virtual try-on with detailed human and clothing information. In Proceedings of the 27th ACM International Conference on Multimedia (ACM MM’19). ACM, 275–283. Google Scholar
Digital Library
- Yang Hu, Xi Yi, and Larry S. Davis. 2015.Collaborative fashion recommendation: A functional tensor factorization approach. In Proceedings of the 23rd ACM International Conference on Multimedia (ACM MM’15). ACM, 129–138. Google Scholar
Digital Library
- Tomoharu Iwata, Shinji Wanatabe, and Hiroshi Sawada. 2011.Fashion coordinates recommender system using photographs from fashion magazines. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI’11), Vol. 3. Morgan Kaufmann, 2262–2267. Google Scholar
Digital Library
- Vignesh Jagadeesh, Robinson Piramuthu, Anurag Bhardwaj, Wei Di, and Neel Sundaresan. 2014.Large scale visual recommendations from street fashion images. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1925–1934. Google Scholar
Digital Library
- Wang-Cheng Kang, Eric Kim, Jure Leskovec, Charles Rosenberg, and Julian McAuley. 2019.Complete the look: Scene-based complementary product recommendation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). IEEE, 10532–10541.Google Scholar
Cross Ref
- M. Hadi Kiapour, Kota Yamaguchi, Alexander C. Berg, and Tamara L. Berg. 2014.Hipster wars: Discovering elements of fashion styles. In Proceedings of the European Conference on Computer Vision (ECCV’14). Springer, 472–488.Google Scholar
- Dmitry Kornilov. 2019. Recommendation system based on a user’s physical features. US Patent App. 15/826,533.Google Scholar
- Sudhir Kumar and Mithun Das Gupta. 2019. GAN: Complementary fashion item recommendation. arXiv preprint arXiv:1906.05596 (2019).Google Scholar
- Honglin Li, Masahiro Toyoura, Kazumi Shimizu, Wei Yang, and Xiaoyang Mao. 2016.Retrieval of clothing images based on relevance feedback with focus on collar designs. Vis. Comput. 32, 10 (2016), 1351–1363. Google Scholar
Digital Library
- Yuncheng Li, Liangliang Cao, Jiang Zhu, and Jiebo Luo. 2017.Mining fashion outfit composition using an end-to-end deep learning approach on set data. IEEE Trans. Multim. 19, 8 (2017), 1946–1955.Google Scholar
Digital Library
- Lizi Liao, Xiangnan He, Bo Zhao, Chong-Wah Ngo, and Tat-Seng Chua. 2018.Interpretable multimodal retrieval for fashion products. In Proceedings of the 26th ACM International Conference on Multimedia (ACM MM’18). ACM, 1571–1579. Google Scholar
Digital Library
- Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2020.Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Trans. Knowl. Data Eng. 32, 8 (2020), 1502–1516.Google Scholar
Cross Ref
- Jian Liu, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Fuzheng Zhuang, Jiajie Xu, Xiaofang Zhou, and Hui Xiong. 2019.Deep cross networks with aesthetic preference for cross-domain recommendation. arXiv preprint arXiv:1905.13030 (2019).Google Scholar
- Luoqi Liu, Junliang Xing, Si Liu, Hui Xu, Xi Zhou, and Shuicheng Yan. 2014.Wow! you are so beautiful today!ACM Trans. Multim. Comput. Commun. Applic. 11, 1s (2014), 1–20. Google Scholar
Digital Library
- Si Liu, Jiashi Feng, Zheng Song, Tianzhu Zhang, Hanqing Lu, Changsheng Xu, and Shuicheng Yan. 2012.Hi, magic closet, tell me what to wear! In Proceedings of the 20th ACM International Conference on Multimedia (ACM MM’12). ACM, 619–628. Google Scholar
Digital Library
- Zhi Lu, Yang Hu, Yunchao Jiang, Yan Chen, and Bing Zeng. 2019.Learning binary code for personalized fashion recommendation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19). IEEE, 10562–10570.Google Scholar
Cross Ref
- Yihui Ma, Jia Jia, Suping Zhou, Jingtian Fu, Yejun Liu, and Zijian Tong. 2017.Towards better understanding the clothing fashion styles: A multimodal deep learning approach. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. AAAI, 38–44. Google Scholar
Digital Library
- Yunshan Ma, Xun Yang, Lizi Liao, Yixin Cao, and Tat-Seng Chua. 2019.Who, where, and what to wear? extracting fashion knowledge from social media. In Proceedings of the 27th ACM International Conference on Multimedia (ACM MM’19). ACM, 257–265. Google Scholar
Digital Library
- Utkarsh Mall, Kevin Matzen, Bharath Hariharan, Noah Snavely, and Kavita Bala. 2019.GeoStyle: Discovering fashion trends and events. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE, 411–420.Google Scholar
Cross Ref
- Kevin Matzen, Kavita Bala, and Noah Snavely. 2017.StreetStyle: Exploring world-wide clothing styles from millions of photos. arXiv preprint arXiv:1706.01869 (2017).Google Scholar
- Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015.Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 43–52. Google Scholar
Digital Library
- Mahesh Chandra Mukkamala and Matthias Hein. 2017.Variants of RMSProp and Adagrad with logarithmic regret bounds. In Proceedings of the 34th International Conference on Machine Learning (ICML’17), Vol. 70. ACM, 2545–2553. Google Scholar
Digital Library
- Tam V. Nguyen, Si Liu, Bingbing Ni, Jun Tan, Yong Rui, and Shuicheng Yan. 2012.Sense beauty via face, dressing, and/or voice. In Proceedings of the 20th ACM International Conference on Multimedia (ACM MM’12). ACM, 239–248. Google Scholar
Digital Library
- Hosnieh Sattar, Gerard Pons-Moll, and Mario Fritz. 2019.Fashion is taking shape: Understanding clothing preference based on body shape from online sources. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV’19). IEEE, 968–977.Google Scholar
Cross Ref
- Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, and Raquel Urtasun. 2015.Neuroaesthetics in fashion: Modeling the perception of fashionability. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). IEEE, 869–877.Google Scholar
Cross Ref
- Karen Simonyan and Andrew Zisserman. 2014.Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Guang-Lu Sun, Zhi-Qi Cheng, Xiao Wu, and Qiang Peng. 2018.Personalized clothing recommendation combining user social circle and fashion style consistency. Multim. Tools Applic. 77, 14 (2018), 17731–17754. Google Scholar
Digital Library
- Moeko Takagi, Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa. 2017.What makes a style: Experimental analysis of fashion prediction. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW’17). IEEE, 2247–2253.Google Scholar
Cross Ref
- Pongsate Tangseng and Takayuki Okatani. 2020.Toward explainable fashion recommendation. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. IEEE, 2153–2162.Google Scholar
Cross Ref
- Kristen Vaccaro, Sunaya Shivakumar, Ziqiao Ding, Karrie Karahalios, and Ranjitha Kumar. 2016.The elements of fashion style. In Proceedings of the 29th Symposium on User Interface Software and Technology. 777–785. Google Scholar
Digital Library
- Dario Riccardo Valenzano, Andrea Mennucci, Giandonato Tartarelli, and Alessandro Cellerino. 2006.Shape analysis of female facial attractiveness. Vis. Res. 46, 8–9 (2006), 1282–1291.Google Scholar
Cross Ref
- Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015.Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’15). IEEE, 4642–4650. Google Scholar
Digital Library
- Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. 2017.Adversarial cross-modal retrieval. In Proceedings of the 25th ACM International Conference on Multimedia (ACM MM’17). ACM, 154–162. Google Scholar
Digital Library
- Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. 2015.On deep multi-view representation learning. In Proceedings of the 32nd International Conference on Machine Learning (ICML’15), Vol. 37. ACM, 1083–1092. Google Scholar
Digital Library
- Zhonghua Wu, Guosheng Lin, Qingyi Tao, and Jianfei Cai. 2019.M2E-Try On Net: Fashion from model to everyone. In Proceedings of the 27th ACM International Conference on Multimedia (ACM MM’19). ACM, 293–301. Google Scholar
Digital Library
- Duorui Xie, Lingyu Liang, Lianwen Jin, Jie Xu, and Mengru Li. 2015.SCUT-FBP: A benchmark dataset for facial beauty perception. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 1821–1826.Google Scholar
Digital Library
- Kota Yamaguchi, Tamara L. Berg, and Luis E. Ortiz. 2014.Chic or social: Visual popularity analysis in online fashion networks. In Proceedings of the 22nd ACM International Conference on Multimedia (ACM MM’14). ACM, 773–776. Google Scholar
Digital Library
- Wei Yang, Masahiro Toyoura, and Xiaoyang Mao. 2012.Hairstyle suggestion using statistical learning. In Proceedings of the International Conference on Multimedia Modeling (MMM’12). Springer, 277–287. Google Scholar
Digital Library
- Xun Yang, Xiangnan He, Xiang Wang, Yunshan Ma, Fuli Feng, Meng Wang, and Tat-Seng Chua. 2019.Interpretable fashion matching with rich attributes. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 775–784. Google Scholar
Digital Library
- Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, and Tat-Seng Chua. 2019.TransNFCM: Translation-based neural fashion compatibility modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. AAAI, 403–410.Google Scholar
Digital Library
- Lap-Fai Yu, Sai Kit Yeung, Demetri Terzopoulos, and Tony F. Chan. 2012.DressUp!: Outfit synthesis through automatic optimization.ACM Trans. Graph. 31, 6 (2012). Google Scholar
Digital Library
- Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018.Aesthetic-based clothing recommendation. In Proceedings of the World Wide Web Conference (WWW’18). ACM, 649–658. Google Scholar
Digital Library
- Yi Yu, Suhua Tang, Francisco Raposo, and Lei Chen. 2019.Deep cross-modal correlation learning for audio and lyrics in music retrieval. ACM Trans. Multim. Comput. Commun. Applic. 15, 1 (2019), 1–16. Google Scholar
Digital Library
- Liangli Zhen, Peng Hu, Xu Wang, and Dezhong Peng. 2019.Deep supervised cross-modal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19). IEEE, 10386–10395.Google Scholar
Cross Ref
- Zhengzhong Zhou, Xiu Di, Wei Zhou, and Liqing Zhang. 2018.Fashion sensitive clothing recommendation using hierarchical collocation model. In Proceedings of the 26th ACM International Conference on Multimedia (ACM MM’18). ACM, 1119–1127. Google Scholar
Digital Library
Index Terms
Deep-based Self-refined Face-top Coordination
Recommendations
Multimodal correlation deep belief networks for multi-view classification
The Restricted Boltzmann machine (RBM) has been proven to be a powerful tool in many specific applications, such as representational learning, document modeling, and many other learning tasks. However, the extensions of the RBM are rarely used in the ...
Analysis of correlation based dimension reduction methods
Analysis of correlation based dimension reduction methodsDimension reduction is an important topic in data mining and machine learning. Especially dimension reduction combined with feature fusion is an effective preprocessing step when the data are ...
Sparse Representation Based Discriminative Canonical Correlation Analysis for Face Recognition
ICMLA '12: Proceedings of the 2012 11th International Conference on Machine Learning and Applications - Volume 01Canonical correlation analysis (CCA) has been widely used in pattern recognition and machine learning. However, both CCA and its extensions sometimes cannot give satisfactory results. In this paper, we propose a new CCA-type method termed sparse ...






Comments