skip to main content
research-article
Open Access

Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks for Fake News Detection

Authors Info & Claims
Published:22 July 2021Publication History
Skip Abstract Section

Abstract

In this article, we focus on fake news detection task and aim to automatically identify the fake news from vast amount of social media posts. To date, many approaches have been proposed to detect fake news, which includes traditional learning methods and deep learning-based models. However, there are three existing challenges: (i) How to represent social media posts effectively, since the post content is various and highly complicated; (ii) how to propose a data-driven method to increase the flexibility of the model to deal with the samples in different contexts and news backgrounds; and (iii) how to fully utilize the additional auxiliary information (the background knowledge and multi-modal information) of posts for better representation learning. To tackle the above challenges, we propose a novel Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks (KMAGCN) to capture the semantic representations by jointly modeling the textual information, knowledge concepts, and visual information into a unified framework for fake news detection. We model posts as graphs and use a knowledge-aware multi-modal adaptive graph learning principal for the effective feature learning. Compared with existing methods, the proposed KMAGCN addresses challenges from three aspects: (1) It models posts as graphs to capture the non-consecutive and long-range semantic relations; (2) it proposes a novel adaptive graph convolutional network to handle the variability of graph data; and (3) it leverages textual information, knowledge concepts and visual information jointly for model learning. We have conducted extensive experiments on three public real-world datasets and superior results demonstrate the effectiveness of KMAGCN compared with other state-of-the-art algorithms.

References

  1. Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. J. Econ. Perspect. 31, 2 (2017), 211–236.Google ScholarGoogle ScholarCross RefCross Ref
  2. Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Sima’an. 2017. Graph convolutional encoders for syntax-aware neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17). 1957–1967.Google ScholarGoogle ScholarCross RefCross Ref
  3. Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceedings of the 20th International Conference on World Wide Web. ACM, 675–684. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Lihan Chen, Jiaqing Liang, Chenhao Xie, Yanghua Xiao. 2018. Short text entity linking with fine-grained Topics. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 457–466. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 1724–1734.Google ScholarGoogle Scholar
  6. Limeng Cui, Suhang Wang, and Dongwon Lee. 2019. SAME: sentiment-aware multi-modal embedding for detecting fake news. In Proceedings of the IEEE ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’19). 41–48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Jia Deng, Wei Dong, Richard Socher, Li Jia Li, and Fei Fei Li. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition. 248–255.Google ScholarGoogle ScholarCross RefCross Ref
  8. Octavian-Eugen Ganea, Marina Ganea, Aurelien Lucchi, Carsten Eickhoff, and Thomas Hofmann. 2016. Probabilistic bag-of-hyperlinks model for entity linking. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 927–938. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Amir Globerson, Nevena Lazic, Soumen Chakrabarti, Amarnag Subramanya, Michael Ringaard, and Fernando Pereira. 2016. Collective entity resolution with multi-focal attention. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 621–631.Google ScholarGoogle ScholarCross RefCross Ref
  10. Zhaochen Guo and Denilson Barbosa. 2018. Robust named entity disambiguation with random walks. Semant. Web 9, 4 (2018), 459–479.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Aditi Gupta, Ponnurangam Kumaraguru, Carlos Castillo, and Patrick Meier. 2014. Tweetcred: Real-time credibility assessment of content on twitter. In Proceedings of the International Conference on Social Informatics. Springer, 228–243.Google ScholarGoogle ScholarCross RefCross Ref
  12. Manish Gupta, Peixiang Zhao, and Jiawei Han. 2012. Evaluating event credibility on twitter. In Proceedings of the 2012 SIAM International Conference on Data Mining. SIAM, 153–164.Google ScholarGoogle ScholarCross RefCross Ref
  13. Guoyong Hu, Ye Ding, Shuhan Qi, Xuan Wang, and Qing Liao. 2019. Multi-depth graph convolutional networks for fake news detection. In Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing. 698–710.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jun Hu, Shengsheng Qian, Quan Fang, Youze Wang, Quan Zhao, Huaiwen Zhang, and Changsheng Xu. 2021. Efficient graph deep learning in tensorflow with tf_geometric. CoRR abs/2101.11552 (2021). Retrieved from https://arxiv.org/abs/2101.11552.Google ScholarGoogle Scholar
  15. Rongrong Ji, Fuhai Chen, Liujuan Cao, and Yue Gao. 2019. Cross-modality microblog sentiment prediction via bi-layer multimodal hypergraph learning. IEEE Trans. Multimedia 21, 4 (2019), 1062–1075.Google ScholarGoogle ScholarCross RefCross Ref
  16. Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, and Jiebo Luo. 2017. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 795–816. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Zhiwei Jin, Juan Cao, Yongdong Zhang, Jianshe Zhou, and Qi Tian. 2016. Novel visual and statistical image features for microblogs news verification. IEEE Trans. Multimedia 19, 3 (2016), 598–608. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Ilias Kalamaras, Anastasios Drosou, and Dimitrios Tzovaras. 2014. Multi-objective optimization for multimodal visualization. IEEE Trans. Multimedia 16, 5 (2014), 1460–1472.Google ScholarGoogle ScholarCross RefCross Ref
  19. Dhruv Khattar, Jaipal Singh Goud, Manish Gupta, and Vasudeva Varma. 2019. Mvae: Multimodal variational autoencoder for fake news detection. In Proceedings of the World Wide Web Conference. 2915–2921. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https://arxiv.org/abs/1412.6980.Google ScholarGoogle Scholar
  21. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR’17). 1–14.Google ScholarGoogle Scholar
  22. Nikolaos Kolitsas, Octavian-Eugen Ganea, and Thomas Hofmann. 2018. End-to-end neural entity linking. arXiv:1808.07699. Retrieved from https://arxiv.org/abs/1808.07699.Google ScholarGoogle Scholar
  23. Sejeong Kwon, Meeyoung Cha, Kyomin Jung, Wei Chen, and Yajun Wang. 2013. Prominent features of rumor propagation in online social media. In Proceedings of the 2013 IEEE 13th International Conference on Data Mining. IEEE, 1103–1108.Google ScholarGoogle ScholarCross RefCross Ref
  24. Nevena Lazic, Amarnag Subramanya, Michael Ringgaard, and Fernando Pereira. 2015. Plato: A selective context model for entity resolution. Trans. Assoc. Comput. Ling. 3 (2015), 503–515.Google ScholarGoogle ScholarCross RefCross Ref
  25. Phong Le and Ivan Titov. 2018. Improving entity linking by modeling latent relations between mentions. arXiv:1804.10637. Retrieved from https://arxiv.org/abs/1804.10637.Google ScholarGoogle Scholar
  26. Song Liu, Shengsheng Qian, Yang Guan, Jiawei Zhan, and Long Ying. 2020. Joint-modal distribution-based similarity hashing for large-scale unsupervised deep cross-modal retrieval. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’20). 1379–1388. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’16). 3818–3824. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jing Ma, Wei Gao, Zhongyu Wei, Yueming Lu, and Kam-Fai Wong. 2015. Detect rumors using time series of social context information on microblogging websites. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 1751–1754. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Diego Marcheggiani, Joost Bastings, and Ivan Titov. 2018. Exploiting semantics in neural machine translation with graph convolutional networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18). 486–492.Google ScholarGoogle ScholarCross RefCross Ref
  30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111–3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. David Milne and Ian H Witten. 2008. Learning to link with wikipedia. In Proceedings of the 17th ACM Conference on Information and Knowledge Management. ACM, 509–518. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. In Proceedings of the 30th Conference on Neural Information Processing Systems NeurIPS Workshop. 1–4.Google ScholarGoogle Scholar
  33. Dong ping Tian et al. 2013. A review on image feature extraction and representation techniques. Int. J. Multimedia Ubiq. Eng. 8, 4 (2013), 385–396.Google ScholarGoogle Scholar
  34. Martin Potthast, Johannes Kiesel, K. Reinartz, Janek Bevendorff, and Benno Stein. 2018. A stylometric inquiry into hyperpartisan and fake news. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 231–240.Google ScholarGoogle ScholarCross RefCross Ref
  35. Peng Qi, Juan Cao, Tianyun Yang, Junbo Guo, and Jintao Li. 2019. Exploiting multi-domain visual information for fake news detection. arXiv:1908.04472. Retrieved from https://arxiv.org/abs/1908.04472.Google ScholarGoogle Scholar
  36. François Rousseau, Emmanouil Kiagias, and Michalis Vazirgiannis. 2015. Text categorization as a graph classification problem. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’15). 1702–1712.Google ScholarGoogle ScholarCross RefCross Ref
  37. Wei Shen, Jianyong Wang, and Jiawei Han. 2015. Entity linking with a knowledge base: Issues, techniques, and solutions. IEEE Trans. Knowl. Data Eng. 27, 2 (2015), 443–460.Google ScholarGoogle ScholarCross RefCross Ref
  38. Baoxu Shi and Tim Weninger. 2016. Discriminative predicate path mining for fact checking in knowledge graphs. Know.-Based Syst. 104, C (Jul. 2016), 123–133. DOI:https://doi.org/10.1016/j.knosys.2016.04.015 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. ACM SIGKDD Explor. Newslett. 19, 1 (2017), 22–36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https://arxiv.org/abs/1409.1556.Google ScholarGoogle Scholar
  41. Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3th International Conference on Learning Representations. 1–14.Google ScholarGoogle Scholar
  42. Shivangi Singhal, Anubha Kabra, Mohit Sharma, Rajiv Ratn Shah, and Ponnurangam Kumaraguru. 2020. SpotFake+: A multimodal framework for fake news detection via transfer learning (student abstract). In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 13915–13916.Google ScholarGoogle ScholarCross RefCross Ref
  43. Shivangi Singhal, Rajiv Ratn Shah, Tanmoy Chakraborty, Ponnurangam Kumaraguru, Shin’ichi Satoh. 2019. SpotFake: A multi-modal framework for fake news detection. In Proceedings of the 2019 IEEE 5th International Conference on Multimedia Big Data. 39–47.Google ScholarGoogle ScholarCross RefCross Ref
  44. Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2008. Yago: A large ontology from wikipedia and wordnet. Web Semant. 6, 3 (2008), 203–217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Simon Tong and Daphne Koller. 2001. Support vector machine active learning with applications to text classification. J. Mach. Learn. Res. 2(Nov.2001), 45–66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Vishaal Udandarao, Abhishek Maiti, Deepak Srivatsav, Suryatej Reddy Vyalla, Yifang Yin, and Rajiv Ratn Shah. 2020. COBRA: Contrastive Bi-Modal Representation Algorithm. arXiv:cs.LG/2005.03687. Retrieved from https://arxiv/org/abs/2005.03687.Google ScholarGoogle Scholar
  47. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv:1710.10903. Retrieved from https://arxiv.org/abs/1710.10903.Google ScholarGoogle Scholar
  48. Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. 2018. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 849–857. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Youze Wang, Shengsheng Qian, Jun Hu, Quan Fang, and Changsheng Xu. 2020. Fake news detection via knowledge-driven multimodal graph convolutional networks. In Proceedings of the 2020 on International Conference on Multimedia Retrieval (ICMR’20). 540–547. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Man Wu, Shirui Pan, Chuan Zhou, Xiaojun Chang, and Xingquan Zhu. 2020. Unsupervised domain adaptive graph convolutional networks. In Proceedings of the World Wide Web Conference (WWW’20). 1457–1467. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Man Wu, Shirui Pan, and Xingquan Zhu. 2020. OpenWGL: Open-world graph learning. In Proceedings of the 20th IEEE International Conference on Data Mining (ICDM’20). 681–690.Google ScholarGoogle ScholarCross RefCross Ref
  52. Wentao Wu, Hongsong Li, Haixun Wang, and Kenny Q Zhu. 2012. Probase: A probabilistic taxonomy for text understanding. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. ACM, 481–492. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Xiao Wu, Chong-Wah Ngo, and Alexander G. Hauptmann. 2008. Multimodal news story clustering with pairwise visual near-duplicate constraint.IEEE Trans. Multimedia 10, 2 (2008), 188–199. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. 2019. A comprehensive survey on graph neural networks. arXiv:1901.00596. Retrieved from https://arxiv.org/abs/1901.00596.Google ScholarGoogle Scholar
  55. Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2017. Learning distributed representations of texts and entities from knowledge base. Trans. Assoc. Comput. Ling. 5 (2017), 397–411.Google ScholarGoogle ScholarCross RefCross Ref
  56. Xiaoshan Yang, Tianzhu Zhang, and Changsheng Xu. 2015. Cross-domain feature learning in multimedia. IEEE Trans. Multimedia 17, 1 (2015), 64–78.Google ScholarGoogle ScholarCross RefCross Ref
  57. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Proceedings of the 32th Conference on Neural Information Processing Systems. 5754–5764. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Liang Yao, Chengsheng Mao, and Yuan Luo. 2018. Graph convolutional networks for text classification. arXiv:1809.05679. Retrieved from https://arxiv.org/abs/1809.05679.Google ScholarGoogle Scholar
  59. Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Graph convolutional networks for text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7370–7377.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2017. A convolutional approach for misinformation identification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. 3901–3907. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Huaiwen Zhang, Quan Fang, Shengsheng Qian, and Changsheng Xu. 2019. Multi-modal knowledge-aware event memory network for social media rumor detection. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19). Association for Computing Machinery, New York, NY, 1942–1951. DOI:https://doi.org/10.1145/3343031.3350850 Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Yingying Zhang, Shengsheng Qian, Quan Fang, and Changsheng Xu. 2019. Multi-modal knowledge-aware hierarchical attention network for explainable medical question answering. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19). 1089–1097. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Lei Zhao, Qinghua Hu, and Wenwu Wang. 2015. Heterogeneous feature selection with multi-modal deep neural networks and sparse group LASSO. IEEE Trans. Multimedia 17, 11 (2015), 1936–1948.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Xinyi Zhou, Jindi Wu, Reza Zafarani. 2020. SAFE: Similarity-Aware multi-modal fake news detection. In Proceedings of the 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD’20). 354–367.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Xinyi Zhou, Jindi Wu, and R. Zafarani. 2020. SAFE: Similarity-aware multi-modal fake news detection. arXiv: abs/2003.04981. Retrieved from https://arxiv.org/abs/2003.04981.Google ScholarGoogle Scholar
  66. Arkaitz Zubiaga, Maria Liakata, and Rob Procter. 2017. Exploiting context for rumour detection in social media. In Proceedings of the International Conference on Social Informatics. Springer, 109–123.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Knowledge-aware Multi-modal Adaptive Graph Convolutional Networks for Fake News Detection

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3
      August 2021
      443 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3476118
      Issue’s Table of Contents

      Copyright © 2021 Association for Computing Machinery.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 22 July 2021
      • Accepted: 1 February 2021
      • Revised: 1 January 2021
      • Received: 1 August 2020
      Published in tomm Volume 17, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!