skip to main content
research-article

A2CMHNE: Attention-Aware Collaborative Multimodal Heterogeneous Network Embedding

Authors Info & Claims
Published:05 June 2019Publication History
Skip Abstract Section

Abstract

Network representation learning is playing an important role in network analysis due to its effectiveness in a variety of applications. However, most existing network embedding models focus on homogeneous networks and neglect the diverse properties such as different types of network structures and associated multimedia content information. In this article, we learn node representations for multimodal heterogeneous networks, which contain multiple types of nodes and/or links as well as multimodal content such as texts and images. We propose a novel attention-aware collaborative multimodal heterogeneous network embedding method (A2CMHNE), where an attention-based collaborative representation learning approach is proposed to promote the collaboration of structure-based embedding and content-based embedding, and generate the robust node representation by introducing an attention mechanism that enables informative embedding integration. In experiments, we compare our model with existing network embedding models on two real-world datasets. Our method leads to dramatic improvements in performance by 5%, and 9% compared with five state-of-the-art embedding methods on one benchmark (M10 Dataset), and on a multi-modal heterogeneous network dataset (WeChat dataset) for node classification, respectively. Experimental results demonstrate the effectiveness of our proposed method on both node classification and link prediction tasks.

References

  1. Smriti Bhagat, Graham Cormode, and S. Muthukrishnan. 2011. Node classification in social networks. In Social Network Data Analytics. 115--148.Google ScholarGoogle Scholar
  2. Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. GraRep: Learning graph representations with global structural information. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM 2015, Melbourne, VIC, Australia, October 19--23, 2015. 891--900. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2016. Deep neural networks for learning graph representations. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, February 12--17, 2016, Phoenix, Arizona. 1145--1152. http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12423. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C. Aggarwal, and Thomas S. Huang. 2015. Heterogeneous network embedding via deep architectures. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, August 10--13, 2015. 119--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ting Chen and Yizhou Sun. 2017. Task-guided and path-augmented heterogeneous network embedding for author identification. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6--10, 2017. 295--304. http://dl.acm.org/citation.cfm?id=3018735. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13--17, 2017. 135--144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, August 13--17, 2016. 855--864. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, June 27--30, 2016. 770--778.Google ScholarGoogle ScholarCross RefCross Ref
  9. Peter D. Hoff, Adrian E. Raftery, and Mark S. Handcock. 2002. Latent space approaches to social network analysis. Publ. Am. Stat. Assoc. 97, 460 (2002), 1090--1098.Google ScholarGoogle ScholarCross RefCross Ref
  10. Jun Hu, Shengsheng Qian, Quan Fang, and Changsheng Xu. 2018. Attentive interactive convolutional matching for community question answering in social multimedia. In 2018 ACM Multimedia Conference on Multimedia Conference, MM 2018, Seoul, Republic of Korea, October 22--26, 2018. 456--464. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Feiran Huang, Xiaoming Zhang, Chaozhuo Li, Zhoujun Li, Yueying He, and Zhonghua Zhao. 2018. Multimodal network embedding via attention based multi-view variational autoencoder. In Proceedings of the 2018 ACM International Conference on Multimedia Retrieval (ICMR’18). ACM, New York, NY, 108--116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR'15) San Diego, CA. http://arxiv.org/abs/1412.6980Google ScholarGoogle Scholar
  13. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR'17), Toulon, France. https://openreview.net/forum?id=SJU4ayYglGoogle ScholarGoogle Scholar
  14. Thomas N. Kipf and Max Welling. 2016. Variational graph auto-encoders. CoRR abs/1611.07308. arxiv:1611.07308http://arxiv.org/abs/1611.07308.Google ScholarGoogle Scholar
  15. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning (ICML'14), Beijing, China. 1188--1196. http://jmlr.org/proceedings/papers/v32/le14.html. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. David Liben-Nowell and Jon M. Kleinberg. 2007. The link-prediction problem for social networks. JASIST 58, 7 (2007), 1019--1031. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. 2016. Tri-party deep network representation. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, 9--15 July 2016. 1895--1901. http://www.ijcai.org/Abstract/16/271. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’14, New York, NY. August 24--27, 2014. 701--710. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Shengsheng Qian, Tianzhu Zhang, Changsheng Xu, and Jie Shao. 2016. Multi-modal event topic model for social event analysis. IEEE Trans. Multimedia 18, 2 (2016), 233--246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Meng Qu, Jian Tang, Jingbo Shang, Xiang Ren, Ming Zhang, and Jiawei Han. 2017. An attention-based collaboration framework for multi-view network representation learning. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM'17) Singapore. 1767--1776. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jitao Sang. 2014. User-centric Social Multimedia Computing. Springer Publishing Company, Incorporated. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 1929--1958. http://dl.acm.org/citation.cfm?id=2670313 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yizhou Sun and Jiawei Han. 2012. Mining heterogeneous information networks: Principles and methodologies. Synthesis Lectures on Data Mining and Knowledge Discovery 3, 2 (2012), 1--159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jian Tang, Meng Qu, and Qiaozhu Mei. 2015a. PTE: Predictive text embedding through large-scale heterogeneous text networks. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, August 10--13, 2015. 1165--1174. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015b. LINE: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, WWW 2015, Florence, Italy, May 18--22, 2015. 1067--1077. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Cunchao Tu, Han Liu, Zhiyuan Liu, and Maosong Sun. 2017. CANE: Context-aware network embedding for relation modeling. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30-- August 4, Volume 1: Long Papers. 1722--1731.Google ScholarGoogle Scholar
  27. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, Nov (2008), 2579--2605.Google ScholarGoogle Scholar
  28. Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, August 13--17, 2016. 1225--1234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Fei Wu, Xinyan Lu, Jun Song, Shuicheng Yan, Zhongfei (Mark) Zhang, Yong Rui, and Yueting Zhuang. 2016. Learning of multimodal representations with random walks on the click graph. IEEE Trans. Image Processing 25, 2 (2016), 630--642.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Linchuan Xu, Xiaokai Wei, Jiannong Cao, and Philip S. Yu. 2017a. Embedding of embedding (EOE): Joint embedding for coupled heterogeneous networks. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6--10, 2017. 741--749. http://dl.acm.org/citation.cfm?id=3018723. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Linchuan Xu, Xiaokai Wei, Jiannong Cao, and Philip S. Yu. 2017b. Embedding of embedding (EOE): Joint embedding for coupled heterogeneous networks. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6--10, 2017. 741--749. http://dl.acm.org/citation.cfm?id=3018723. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. 2015. Network representation learning with rich text information. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25--31, 2015. 2111--2117. http://ijcai.org/Abstract/15/299. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Hanwang Zhang, Xindi Shang, Huan-Bo Luan, Meng Wang, and Tat-Seng Chua. 2016. Learning from collective intelligence: Feature learning using social images and tags. TOMCCAP 13, 1 (2016), 1:1--1:23. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A2CMHNE: Attention-Aware Collaborative Multimodal Heterogeneous Network Embedding

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 15, Issue 2
      May 2019
      375 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3339884
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 5 June 2019
      • Accepted: 1 February 2019
      • Revised: 1 January 2019
      • Received: 1 June 2018
      Published in tomm Volume 15, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!