skip to main content
research-article

BayesKGR: Bayesian Few-Shot Learning for Knowledge Graph Reasoning

Authors Info & Claims
Published:17 June 2023Publication History
Skip Abstract Section

Abstract

Reasoning over knowledge graphs (KGs) has received increasing attention recently due to its promising applications in many areas, such as semantic search and recommendation systems. Subsequently, most reasoning models are inherently transductive and ignore uncertainties of KGs, making it difficult to generalize to unseen entities. Moreover, existing approaches usually require each entity in the KG to have sufficient training samples, which leads to the overfitting of the entity having few instances. In fact, long-tail distributions are quite widespread in KGs, and newly emerging entities will tend to have only a few related triples. In this work, we aim at studying knowledge graph reasoning under a challenging setting where only limited training samples are available. Specifically, we propose a Bayesian inductive reasoning method and incorporate meta-learning techniques in few-shot learning to solve data deficiency and uncertainties. We design a Bayesian graph neural network as a meta-learner to achieve Bayesian inference, which can extrapolate meta-knowledge from observed KG to emerging entities. We conduct extensive experiments on two large-scale benchmark datasets, and the results demonstrate considerable performance improvement with the proposed approach over other baselines.

REFERENCES

  1. [1] Baek Jinheon, Lee Dong Bok, and Hwang Sung Ju. 2020. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems. 546560.Google ScholarGoogle Scholar
  2. [2] Balazevic Ivana, Allen Carl, and Hospedales Timothy M.. 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 51845193.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Blundell Charles, Cornebise Julien, Kavukcuoglu Koray, and Wierstra Daan. 2015. Weight uncertainty in neural network. In Proceedings of the 32nd International Conference on Machine Learning. 16131622.Google ScholarGoogle Scholar
  4. [4] Bollacker Kurt D., Evans Colin, Paritosh Praveen K., Sturge Tim, and Taylor Jamie. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 12471250.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Bordes Antoine, Usunier Nicolas, García-Durán Alberto, Weston Jason, and Yakhnenko Oksana. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems. 27872795.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Cai Jiarui, Wang Yizhou, and Hwang Jenq-Neng. 2021. ACE: Ally complementary experts for solving long-tailed recognition in one-shot. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 112121.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Cao Zongsheng, Xu Qianqian, Yang Zhiyong, Cao Xiaochun, and Huang Qingming. 2021. Dual quaternion knowledge graph embeddings. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 68946902.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Chekol Melisachew Wudage, Pirrò Giuseppe, Schoenfisch Joerg, and Stuckenschmidt Heiner. 2017. Marrying uncertainty and time in knowledge graphs. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 8894.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Chen Mingyang, Zhang Wen, Zhang Wei, Chen Qiang, and Chen Huajun. 2019. Meta relational learning for few-shot link prediction in knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 42164225.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Chen Zhu-Mu, Yeh Mi-Yen, and Kuo Tei-Wei. 2021. PASSLEAF: A Pool-bAsed Semi-Supervised LEArning Framework for uncertain knowledge graph embedding. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 40194026.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Dadaneh Siamak Zamani, Boluki Shahin, Zhou Mingyuan, and Qian Xiaoning. 2020. ARSM gradient estimator for supervised learning to rank. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 31573161.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Defferrard Michaël, Bresson Xavier, and Vandergheynst Pierre. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems. 38373845.Google ScholarGoogle Scholar
  13. [13] Dettmers Tim, Minervini Pasquale, Stenetorp Pontus, and Riedel Sebastian. 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 18111818.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Finn Chelsea, Abbeel Pieter, and Levine Sergey. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning. 11261135.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Garnelo Marta, Rosenbaum Dan, Maddison Christopher, Ramalho Tiago, Saxton David, Shanahan Murray, Teh Yee Whye, Rezende Danilo Jimenez, and Eslami S. M. Ali. 2018. Conditional neural processes. In Proceedings of the 35th International Conference on Machine Learning. 16901699.Google ScholarGoogle Scholar
  16. [16] Hamaguchi Takuo, Oiwa Hidekazu, Shimbo Masashi, and Matsumoto Yuji. 2017. Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 18021808.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] He Shizhu, Liu Kang, Ji Guoliang, and Zhao Jun. 2015. Learning to represent knowledge graphs with Gaussian embedding. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management. 623632.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Ji Guoliang, He Shizhu, Xu Liheng, Liu Kang, and Zhao Jun. 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. 687696.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Jin Taisong, Dai Huaqiang, Cao Liujuan, Zhang Baochang, Huang Feiyue, Gao Yue, and Ji Rongrong. 2022. Deepwalk-aware graph convolutional networks. Science China Information Sciences 65, 5 (2022), 115.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Kipf Thomas N. and Welling Max. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations. 114.Google ScholarGoogle Scholar
  21. [21] Koch Gregory, Zemel Richard, and Salakhutdinov Ruslan. 2015. Siamese neural networks for one-shot image recognition. In Proceedings of the CML Deep Learning Workshop. 18.Google ScholarGoogle Scholar
  22. [22] Lin Xi Victoria, Socher Richard, and Xiong Caiming. 2018. Multi-hop knowledge graph reasoning with reward shaping. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 32433253.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Lin Yankai, Liu Zhiyuan, Sun Maosong, Liu Yang, and Zhu Xuan. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. 21812187.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Ma Jiaqi, Tang Weijing, Zhu Ji, and Mei Qiaozhu. 2019. A flexible generative framework for graph-based semi-supervised learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems. 32763285.Google ScholarGoogle Scholar
  25. [25] Mishra Nikhil, Rohaninejad Mostafa, Chen Xi, and Abbeel Pieter. 2018. A simple neural attentive meta-learner. In Proceedings of the 6th International Conference on Learning Representations. 117.Google ScholarGoogle Scholar
  26. [26] Nathani Deepak, Chauhan Jatin, Sharma Charu, and Kaul Manohar. 2019. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 47104723.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Nguyen Dai Quoc, Nguyen Tu Dinh, Nguyen Dat Quoc, and Phung Dinh Q.. 2018. A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. 327333.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Nickel Maximilian, Tresp Volker, and Kriegel Hans-Peter. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning. 809816.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Parisot Sarah, Esperança Pedro M., McDonagh Steven, Madarasz Tamas J., Yang Yongxin, and Li Zhenguo. 2022. Long-tail recognition via compositional knowledge transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 69396948.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Rusu Andrei A., Rao Dushyant, Sygnowski Jakub, Vinyals Oriol, Pascanu Razvan, Osindero Simon, and Hadsell Raia. 2019. Meta-learning with latent embedding optimization. In Proceedings of the 7th International Conference on Learning Representations. 117.Google ScholarGoogle Scholar
  31. [31] Schlichtkrull Michael Sejr, Kipf Thomas N., Bloem Peter, Berg Rianne van den, Titov Ivan, and Welling Max. 2018. Modeling relational data with graph convolutional networks. In The Semantic Web. Lecture Notes in Computer Science, Vol. 10843. Springer, 593607.Google ScholarGoogle Scholar
  32. [32] Shi Baoxu and Weninger Tim. 2018. Open-world knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 19571964.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Speer Robyn, Chin Joshua, and Havasi Catherine. 2017. ConceptNet 5.5: An open multilingual graph of general knowledge. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 44444451.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Sun Jian, Zhou Yu, and Zong Chengqing. 2022. One-shot relation learning for knowledge graphs via neighborhood aggregation and paths encoding. ACM Transactions on Asian and Low-Resource Language Information Processing 21, 3 (2022), 119.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Sun Zhiqing, Deng Zhi-Hong, Nie Jian-Yun, and Tang Jian. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations. 118.Google ScholarGoogle Scholar
  36. [36] Tuan Yi-Lin, Beygi Sajjad, Fazel-Zarandi Maryam, Gao Qiaozi, Cervone Alessandra, and Wang William Yang. 2022. Towards large-scale interpretable knowledge graph reasoning for dialogue systems. In Findings of the Association for Computational Linguistics: ACL 2022. Association for Computational Linguistics, Dublin, Ireland, 383395.Google ScholarGoogle Scholar
  37. [37] Vashishth Shikhar, Sanyal Soumya, Nitin Vikram, and Talukdar Partha P.. 2020. Composition-based multi-relational graph convolutional networks. In Proceedings of the 8th International Conference on Learning Representations. 115.Google ScholarGoogle Scholar
  38. [38] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems. 59986008.Google ScholarGoogle Scholar
  39. [39] Velickovic Petar, Cucurull Guillem, Casanova Arantxa, Romero Adriana, Liò Pietro, and Bengio Yoshua. 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations. 112.Google ScholarGoogle Scholar
  40. [40] Wang Peifeng, Han Jialong, Li Chenliang, and Pan Rong. 2019. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In Proceedings of the AAAI Conference on Artificial Intelligence. 71527159.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Wang Xiting, Liu Kunpeng, Wang Dongjie, Wu Le, Fu Yanjie, and Xie Xing. 2022. Multi-level recommendation reasoning over knowledge graphs with reinforcement learning. In Proceedings of the ACM Web Conference. 20982108.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Wang Zhen, Zhang Jianwen, Feng Jianlin, and Chen Zheng. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 11121119.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Xiong Wenhan, Hoang Thien, and Wang William Yang. 2017. DeepPath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 564573.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Xiong Wenhan, Yu Mo, Chang Shiyu, Guo Xiaoxiao, and Wang William Yang. 2018. One-shot relational learning for knowledge graphs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 19801990.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Yang Bishan, Yih Wen-Tau, He Xiaodong, Gao Jianfeng, and Deng Li. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representations. 112.Google ScholarGoogle Scholar
  46. [46] Zhang Chuxu, Yao Huaxiu, Huang Chao, Jiang Meng, Li Zhenhui, and Chawla Nitesh V.. 2020. Few-shot knowledge graph completion. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 30413048.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Zhang Ningyu, Deng Shumin, Sun Zhanlin, Chen Jiaoyan, Zhang Wei, and Chen Huajun. 2020. Relation adversarial network for low resource knowledge graph completion. In Proceedings of the Web Conference. 112.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Zhang Yingxue, Pal Soumyasundar, Coates Mark, and Üstebay Deniz. 2019. Bayesian graph convolutional neural networks for semi-supervised classification. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 58295836.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Zhang Zhaoli, Li Zhifei, Liu Hai, and Xiong Neal N.. 2022. Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering 34, 5 (2022), 23352347.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Zhao Yu, Zhang Anxiang, Xie Ruobing, Liu Kang, and Wang Xiaojie. 2020. Connecting embeddings for knowledge graph entity typing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 64196428.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. BayesKGR: Bayesian Few-Shot Learning for Knowledge Graph Reasoning

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Asian and Low-Resource Language Information Processing
          ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 6
          June 2023
          635 pages
          ISSN:2375-4699
          EISSN:2375-4702
          DOI:10.1145/3604597
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 17 June 2023
          • Online AM: 27 March 2023
          • Accepted: 13 March 2023
          • Revised: 29 December 2022
          • Received: 23 June 2022
          Published in tallip Volume 22, Issue 6

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
        • Article Metrics

          • Downloads (Last 12 months)167
          • Downloads (Last 6 weeks)46

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!