skip to main content
research-article

Learning Visual Elements of Images for Discovery of Brand Posts

Authors Info & Claims
Published:22 May 2020Publication History
Skip Abstract Section

Abstract

Online Social Network Sites have become a primary platform for brands and organizations to engage their audience by sharing image and video posts on their timelines. Different from traditional advertising, these posts are not restricted to the products or logo but include visual elements that express more in general the values and attributes of the brand, called brand associations. Since marketers are increasingly spending time in discovering and re-posting user generated posts that reflect the brand attributes, there is an increasing demand for such discovery systems. The goal of these systems is to assist brand experts in filtering through online collections of new user media to discover actionable posts, which match the brand value and have the potential to engage the consumers. Driven by this real-life application, we define and formulate a new task of content discovery for brands and propose a framework that learns to rank posts for brands from their historical timeline. We design a Personalized Content Discovery (PCD) framework to address the three challenges of high inter-brand similarity, sparsity of brand--post interactions, and diversification of timeline. To learn fine-grained brand representation and to generate explanations for the ranking, we automatically learn visual elements of posts from the timeline of brands and from a set of brand attributes in the domain of marketing. To test our framework we use two large-scale Instagram datasets that contain a total of more than 1.5 million image and video posts from the historical timeline of hundreds of brands from multiple verticals such as food and fashion. Extensive experiments indicate that our model can effectively learn fine-grained brand representations and outperform the closest state-of-the-art solutions.

References

  1. Jennifer L. Aaker. 1997. Dimensions of brand personality. J. Market. Res. 34, 3 (1997), 347--356.Google ScholarGoogle ScholarCross RefCross Ref
  2. Hassan Alboqami, Wafi Al-Karaghouli, Yasser Baeshen, Ismail Erkan, Chris Evans, and Ahmad Ghoneim. 2015. Electronic word of mouth in social media: The common characteristics of retweeted and favourited marketer-generated content posted on Twitter. Int. J. Internet Market. Advert. 9, 4 (2015), 338--358.Google ScholarGoogle ScholarCross RefCross Ref
  3. Christy Ashley and Tracy Tuten. 2015. Creative strategies in social media marketing: An exploratory study of branded social content and consumer engagement. Psychol. Market. 32, 1 (2015), 15--27.Google ScholarGoogle ScholarCross RefCross Ref
  4. Janelle Barlow and Paul Stewart. 2004. Branded Customer Service: The New Competitive Edge. Berrett-Koehler.Google ScholarGoogle Scholar
  5. Chia Yu Chang. 2014. Visualizing brand personality and personal branding: case analysis on Starbucks and Nike’s brand value co-creation on Instagram. MA (Master of Arts) thesis, University of Iowa. http://ir.uiowa.edu/etd/1304.Google ScholarGoogle Scholar
  6. Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’17). 335--344.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Tao Chen, Xiangnan He, and Min-Yen Kan. 2016. Context-aware image tweet modelling and recommendation. In Proceedings of the 2016 ACM on Multimedia Conference (MM’16). 1018--1027.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Peng Cui, Zhiyu Wang, and Zhou Su. 2014. What videos are similar with you?: Learning a common attributed representation for video recommendation. In Proceedings of the 22nd ACM International Conference on Multimedia. 597--606.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Lisette De Vries, Sonja Gensler, and Peter S. H. Leeflang. 2012. Popularity of brand posts on brand fan pages: An investigation of the effects of social media marketing. J. Interact. Market. 26, 2 (2012), 83--91.Google ScholarGoogle ScholarCross RefCross Ref
  10. Yi Ding, Chee Wei Phang, Xianghua Lu, Chuan-Hoo Tan, and Juliana Sutanto. 2014. The role of marketer- and user-generated content in sustaining the growth of a social media brand community. In Proceedings of the 2014 47th Hawaii International Conference on System Sciences. 1785--1792.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Aleksandr Farseev, Kirill Lepikhin, Kenny Powar, Eu Khoon Ang, and Hendrik Schwartz. 2018. SoMin.ai: Social multimedia influencer discovery marketplace. In Proceedings of the 26th ACM International Conference on Multimedia (MM’18).Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Yue Gao, Fanglin Wang, Huanbo Luan, and Tat-Seng Chua. 2014. Brand data gathering from live social media streams. In Proceedings of the International Conference on Multimedia Retrieval (ICMR’14). 169--176.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Francesco Gelli, Xiangnan He, Tao Chen, and Tat-Seng Chua. 2017. How personality affects our likes: Towards a better understanding of actionable images. In Proceedings of the 2017 ACM on Multimedia Conference (MM’17). 1828--1837.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Francesco Gelli, Tiberio Uricchio, Xiangnan He, Alberto Del Bimbo, and Tat-Seng Chua. 2018. Beyond the product: Discovering image posts for brands in social media. In Proceedings of the 2018 ACM Multimedia Conference on Multimedia Conference (MM’18). 465--473.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Francesco Gelli, Tiberio Uricchio, Xiangnan He, Alberto Del Bimbo, and Tat-Seng Chua. 2019. Learning subjective attributes of images from auxiliary sources. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19). 2263--2271. DOI:https://doi.org/10.1145/3343031.3350574Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Xue Geng, Hanwang Zhang, Jingwen Bian, and Tat-Seng Chua. 2015. Learning image and user features for recommendation in social networks. In Proceedings of the International Conference on Computer Vision (ICCV’15). 4274--4282.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Sonja Gensler, Franziska Völckner, Yuping Liu-Thompkins, and Wiertz Caroline. 2013. Managing brands in the social media environment. J. Interact. Market. 27, 4 (2013), 242--256.Google ScholarGoogle ScholarCross RefCross Ref
  18. Khim-Yong Goh, Cheng-Suang Heng, and Zhijie Lin. 2013. Social media brand community and consumer behavior: Quantifying the relative impact of user- and marketer-generated content. Inf. Syst. Res. 24, 1 (2013), 88--107.Google ScholarGoogle ScholarCross RefCross Ref
  19. Ruining He and Julian McAuley. 2016. VBPR: Visual Bayesian personalized ranking from implicit feedback. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 144--150.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Xiangnan He, Tao Chen, Min-Yen Kan, and Xiao Chen. 2015. TriRank: Review-aware explainable recommendation by modeling aspects. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM’15). 1661--1670.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’17). 355--364.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos Tikk. 2016. Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys’16). 241--248.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Sudarsan Jayasingh and Venkatesh Rajagopalan. 2015. Customer engagement factors in facebook brand pages. Asian Soc. Sci. 11, 26 (2015). DOI: 10.5539/ass.v11n26p19Google ScholarGoogle Scholar
  24. Wang-Cheng Kang, Chen Fang, Zhaowen Wang, and Julian McAuley. 2017. Visually-aware fashion recommendation and design with generative image models. In Proceedings of the International Conference on Data Mining (ICDM’17). 207--216.Google ScholarGoogle ScholarCross RefCross Ref
  25. Wang-Cheng Kang, Eric Kim, Jure Leskovec, Charles Rosenberg, and Julian McAuley. 2019. Complete the look: Scene-based complementary product recommendation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 10532--10541.Google ScholarGoogle ScholarCross RefCross Ref
  26. Gunhee Kim and Eric P. Xing. 2013. Discovering pictorial brand associations from large-scale online image data. In Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops (ICDMW’15). 404--411.Google ScholarGoogle Scholar
  27. Gunhee Kim and Eric P. Xing. 2014. Visualizing brand associations from web community photos. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM’14). 623--632.Google ScholarGoogle Scholar
  28. Jingu Kim, Renato D. C. Monteiro, and Haesun Park. 2012. Group sparsity in nonnegative matrix factorization. In Proceedings of the 2012 SIAM International Conference on Data Mining (SDM’12). 851--862.Google ScholarGoogle ScholarCross RefCross Ref
  29. Solomon Kullback. 1997. Information Theory and Statistics. Courier Corporation.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Chenyi Lei, Dong Liu, Weiping Li, Zheng-Jun Zha, and Houqiang Li. 2016. Comparative deep learning of hybrid representations for image recommendations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 2545--2553.Google ScholarGoogle ScholarCross RefCross Ref
  31. Tereza Litsa. 2016. How the rise of user-generated content changes marketing. Retrieved from https://www.clickz.com/how-the-rise-of-user-generated-content-changes-marketing.Google ScholarGoogle Scholar
  32. Mitchell Lovett, Renana Peres, and Ron Shachar. 2014. A data set of brands and their characteristics. Market. Sci. 33, 4 (2014), 609--617.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Shuang Ma and Chang Wen Chen. 2018. D-Sempre: Learning deep semantic-preserving embeddings for user interests-social contents modeling. CoRR abs/1802.06451 (2018).Google ScholarGoogle Scholar
  34. Masoud Mazloom, Robert Rietveld, Stevan Rudinac, Marcel Worring, and Willemijn van Dolen. 2016. Multimodal popularity prediction of brand-related social media posts. In Proceedings of the 2016 ACM on Multimedia Conference (MM’16). 197--201.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Wei Niu, James Caverlee, and Haokai Lu. 2018. Neural personalized ranking for image recommendation. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM’18). 423--431.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Steve Olenski. 2016. Can Your Content Sell a Lifestyle and Not Just Products? Retrieved from https://www.forbes.com/sites/steveolenski/2016/07/02/can-your-content-sell-a-lifestyle-and-not-just-products.Google ScholarGoogle Scholar
  37. Gijs Overgoor, Masoud Mazloom, Marcel Worring, Robert Rietveld, and Willemijn van Dolen. 2017. A spatio-temporal category representation for brand popularity prediction. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval (ICMR’17). 233--241.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Irena Pletikosa Cvijikj and Florian Michahelles. 2013. Online engagement factors on Facebook brand pages. Soc. Netw. Anal. Min. 3, 4 (2013), 843--861.Google ScholarGoogle ScholarCross RefCross Ref
  39. Steffen Rendle. 2010. Factorization machines. In Proceedings of the IEEE 10th International Conference on Data Mining (ICDM’10). 995--1000.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI’09). 452--461.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ruslan Salakhutdinov and Andriy Mnih. 2007. Probabilistic matrix factorization. In Proceedings of the 20th International Conference on Neural Information Processing Systems (NIPS’07). 1257--1264.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Martin Saveski and Amin Mantrach. 2014. Item cold-start recommendations: Learning local collective embeddings. In Proceedings of the 8th ACM Conference on Recommender Systems (RecSys’14). 89--96.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Shaojung Sharon Wang, Yu-Ching Lin, and Ting-Peng Liang. 2018. Posts that attract millions of fans: The effect of brand-post congruence. Electr. Commerce Res. Appl. 28 (2018), 73--85. DOI: 10.1016/j.elerap.2017.12.010Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for Top-N recommender systems. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining (WSDM’16). 153--162.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018. Aesthetic-based clothing recommendation. In Proceedings of the 2018 World Wide Web Conference (WWW’18). 649--658.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. CoRR abs/1212.5701 (2012).Google ScholarGoogle Scholar
  47. Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16). 353--362.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, and Tat-Seng Chua. 2013. Attribute-augmented semantic hierarchy: Towards bridging semantic gap and intention gap in image retrieval. In Proceedings of the 21st ACM International Conference on Multimedia (MM’13). 33--42.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Yin Zhang and James Caverlee. 2019. Instagrammers, fashionistas, and me: Recurrent fashion recommendation with implicit visual influence. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1583--1592.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Learning Visual Elements of Images for Discovery of Brand Posts

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 16, Issue 2
          May 2020
          390 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3401894
          Issue’s Table of Contents

          Copyright © 2020 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 22 May 2020
          • Online AM: 7 May 2020
          • Revised: 1 February 2020
          • Accepted: 1 February 2020
          • Received: 1 January 2020
          Published in tomm Volume 16, Issue 2

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!