Abstract
Reasoning over knowledge graphs (KGs) has received increasing attention recently due to its promising applications in many areas, such as semantic search and recommendation systems. Subsequently, most reasoning models are inherently transductive and ignore uncertainties of KGs, making it difficult to generalize to unseen entities. Moreover, existing approaches usually require each entity in the KG to have sufficient training samples, which leads to the overfitting of the entity having few instances. In fact, long-tail distributions are quite widespread in KGs, and newly emerging entities will tend to have only a few related triples. In this work, we aim at studying knowledge graph reasoning under a challenging setting where only limited training samples are available. Specifically, we propose a Bayesian inductive reasoning method and incorporate meta-learning techniques in few-shot learning to solve data deficiency and uncertainties. We design a Bayesian graph neural network as a meta-learner to achieve Bayesian inference, which can extrapolate meta-knowledge from observed KG to emerging entities. We conduct extensive experiments on two large-scale benchmark datasets, and the results demonstrate considerable performance improvement with the proposed approach over other baselines.
- [1] . 2020. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems. 546–560.Google Scholar
- [2] . 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 5184–5193.Google Scholar
Cross Ref
- [3] . 2015. Weight uncertainty in neural network. In Proceedings of the 32nd International Conference on Machine Learning. 1613–1622.Google Scholar
- [4] . 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 1247–1250.Google Scholar
Digital Library
- [5] . 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems. 2787–2795.Google Scholar
Digital Library
- [6] . 2021. ACE: Ally complementary experts for solving long-tailed recognition in one-shot. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 112–121.Google Scholar
Cross Ref
- [7] . 2021. Dual quaternion knowledge graph embeddings. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 6894–6902.Google Scholar
Cross Ref
- [8] . 2017. Marrying uncertainty and time in knowledge graphs. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 88–94.Google Scholar
Cross Ref
- [9] . 2019. Meta relational learning for few-shot link prediction in knowledge graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 4216–4225.Google Scholar
Cross Ref
- [10] . 2021. PASSLEAF: A Pool-bAsed Semi-Supervised LEArning Framework for uncertain knowledge graph embedding. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 4019–4026.Google Scholar
Cross Ref
- [11] . 2020. ARSM gradient estimator for supervised learning to rank. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 3157–3161.Google Scholar
Cross Ref
- [12] . 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems. 3837–3845.Google Scholar
- [13] . 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 1811–1818.Google Scholar
Cross Ref
- [14] . 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning. 1126–1135.Google Scholar
Digital Library
- [15] . 2018. Conditional neural processes. In Proceedings of the 35th International Conference on Machine Learning. 1690–1699.Google Scholar
- [16] . 2017. Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1802–1808.Google Scholar
Cross Ref
- [17] . 2015. Learning to represent knowledge graphs with Gaussian embedding. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management. 623–632.Google Scholar
Digital Library
- [18] . 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. 687–696.Google Scholar
Cross Ref
- [19] . 2022. Deepwalk-aware graph convolutional networks. Science China Information Sciences 65, 5 (2022), 1–15.Google Scholar
Cross Ref
- [20] . 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations. 1–14.Google Scholar
- [21] . 2015. Siamese neural networks for one-shot image recognition. In Proceedings of the CML Deep Learning Workshop. 1–8.Google Scholar
- [22] . 2018. Multi-hop knowledge graph reasoning with reward shaping. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 3243–3253.Google Scholar
Cross Ref
- [23] . 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2181–2187.Google Scholar
Cross Ref
- [24] . 2019. A flexible generative framework for graph-based semi-supervised learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems. 3276–3285.Google Scholar
- [25] . 2018. A simple neural attentive meta-learner. In Proceedings of the 6th International Conference on Learning Representations. 1–17.Google Scholar
- [26] . 2019. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 4710–4723.Google Scholar
Cross Ref
- [27] . 2018. A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. 327–333.Google Scholar
Cross Ref
- [28] . 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning. 809–816.Google Scholar
Digital Library
- [29] . 2022. Long-tail recognition via compositional knowledge transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6939–6948.Google Scholar
Cross Ref
- [30] . 2019. Meta-learning with latent embedding optimization. In Proceedings of the 7th International Conference on Learning Representations. 1–17.Google Scholar
- [31] . 2018. Modeling relational data with graph convolutional networks. In The Semantic Web. Lecture Notes in Computer Science, Vol. 10843. Springer, 593–607.Google Scholar
- [32] . 2018. Open-world knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence. 1957–1964.Google Scholar
Cross Ref
- [33] . 2017. ConceptNet 5.5: An open multilingual graph of general knowledge. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 4444–4451.Google Scholar
Cross Ref
- [34] . 2022. One-shot relation learning for knowledge graphs via neighborhood aggregation and paths encoding. ACM Transactions on Asian and Low-Resource Language Information Processing 21, 3 (2022), 1–19.Google Scholar
Digital Library
- [35] . 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations. 1–18.Google Scholar
- [36] . 2022. Towards large-scale interpretable knowledge graph reasoning for dialogue systems. In Findings of the Association for Computational Linguistics: ACL 2022. Association for Computational Linguistics, Dublin, Ireland, 383–395.Google Scholar
- [37] . 2020. Composition-based multi-relational graph convolutional networks. In Proceedings of the 8th International Conference on Learning Representations. 1–15.Google Scholar
- [38] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems. 5998–6008.Google Scholar
- [39] . 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations. 1–12.Google Scholar
- [40] . 2019. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In Proceedings of the AAAI Conference on Artificial Intelligence. 7152–7159.Google Scholar
Digital Library
- [41] . 2022. Multi-level recommendation reasoning over knowledge graphs with reinforcement learning. In Proceedings of the ACM Web Conference. 2098–2108.Google Scholar
Digital Library
- [42] . 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112–1119.Google Scholar
Cross Ref
- [43] . 2017. DeepPath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 564–573.Google Scholar
Cross Ref
- [44] . 2018. One-shot relational learning for knowledge graphs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 1980–1990.Google Scholar
Cross Ref
- [45] . 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representations. 1–12.Google Scholar
- [46] . 2020. Few-shot knowledge graph completion. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 3041–3048.Google Scholar
Cross Ref
- [47] . 2020. Relation adversarial network for low resource knowledge graph completion. In Proceedings of the Web Conference. 1–12.Google Scholar
Digital Library
- [48] . 2019. Bayesian graph convolutional neural networks for semi-supervised classification. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 5829–5836.Google Scholar
Digital Library
- [49] . 2022. Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering 34, 5 (2022), 2335–2347.Google Scholar
Cross Ref
- [50] . 2020. Connecting embeddings for knowledge graph entity typing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6419–6428.Google Scholar
Cross Ref
Index Terms
BayesKGR: Bayesian Few-Shot Learning for Knowledge Graph Reasoning
Recommendations
Learning to teach and learn for semi-supervised few-shot image classification
AbstractThis paper presents a novel semi-supervised few-shot image classification method named Learning to Teach and Learn (LTTL) to effectively leverage unlabeled samples in small-data regimes. Our method is based on self-training, which ...
Highlights- We propose a novel self-training strategy for semi-supervised few-shot image classification.
Few-Shot Knowledge Graph Entity Typing
Advances in Knowledge Discovery and Data MiningAbstractKnowledge graph entity typing, which is an important way to complete knowledge graphs (KGs), aims at predicting the associating type of certain given entities without any external knowledge. However, previous methods suppose that many (entity, ...
Unsupervised meta-learning for few-shot learning
Highlights- Unsupervised meta-learning that auto-constructs tasks from unlabeled data.
- ...
AbstractMeta-learning is an effective tool to address the few-shot learning problem, which requires new data to be classified considering only a few training examples. However, when used for classification, it requires large labeled datasets, ...






Comments