Abstract
A knowledge graph is a graph with entities of different types as nodes and various relations among them as edges. The construction of knowledge graphs in the past decades facilitates many applications, such as link prediction, web search analysis, question answering, and so on. Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, for example, TransE, TransH, and TransR, learn the embedding representation by defining a global margin-based loss function over the data. However, the loss function is determined during experiments whose parameters are examined among a closed set of candidates. Moreover, embeddings over two knowledge graphs with different entities and relations share the same set of candidates, ignoring the locality of both graphs. This leads to the limited performance of embedding related applications. In this article, a locally adaptive translation method for knowledge graph embedding, called TransA, is proposed to find the loss function by adaptively determining its margin over different knowledge graphs. Then the convergence of TransA is verified from the aspect of its uniform stability. To make the embedding methods up-to-date when new vertices and edges are added into the knowledge graph, the incremental algorithm for TransA, called iTransA, is proposed by adaptively adjusting the optimal margin over time. Experiments on four benchmark data sets demonstrate the superiority of the proposed method, as compared to the state-of-the-art ones.
- Yoshua Bengio, Aaron Courville, and Pierre Vincent. 2013. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 8 (2013), 1798--1828. Google Scholar
Digital Library
- James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, Feb (2012), 281--305. Google Scholar
Digital Library
- Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 1247--1250. Google Scholar
Digital Library
- Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 127--135.Google Scholar
- Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2014. A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94, 2 (2014), 233--259. Google Scholar
Digital Library
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems. 2787--2795. Google Scholar
Digital Library
- Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Proceedings of the Conference on Artificial Intelligence. Google Scholar
Digital Library
- Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. 1992. A training algorithm for optimal margin classifiers. In Proceedings of the 5th Annual Workshop on Computational Learning Theory. ACM, 144--152. Google Scholar
Digital Library
- Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. J. Mach. Learn. Res. 2 (2002), 499--526. Google Scholar
Digital Library
- Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Toward an architecture for never-ending language learning. In Proceedings of AAAI Conference on Artificial Intelligence. Google Scholar
Digital Library
- Kai-Wei Chang, Wen-tau Yih, and Christopher Meek. 2013. Multi-relational latent semantic analysis. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP’13). 1602--1612.Google Scholar
- Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. 2014. Typed tensor decomposition of knowledge bases for relation extraction. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1568--1579.Google Scholar
Cross Ref
- Olivier Chapelle, Vladimir Vapnik, Olivier Bousquet, and Sayan Mukherjee. 2002. Choosing multiple parameters for support vector machines. Mach. Learn. 46, 1 (2002), 131--159. Google Scholar
Digital Library
- Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. 2000. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 4 (2000), 1253--1278. Google Scholar
Digital Library
- Foo Chuan-sheng Do, B. Chuong, and Andrew Y. Ng. 2007. Efficient multiple hyperparameter learning for log-linear models. In Advances in Neural Information Processing Systems. 377--384. Google Scholar
Digital Library
- Huyen Do, Alexandros Kalousis, and Melanie Hilario. 2009. Feature Weighting Using Margin and Radius Based Error Bound Optimization in SVMs. Springer.Google Scholar
- Huyen Do, Alexandros Kalousis, Jun Wang, and Adam Woznica. 2012. A metric learning perspective of SVM: On the relation of LMNN and SVM. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 308--317.Google Scholar
- Sergio Duarte Torres, Ingmar Weber, and Djoerd Hiemstra. 2014. Analysis of search and browsing behavior of young users on the web. ACM Trans. Web (TWEB) 8, 2 (2014), 7. Google Scholar
Digital Library
- Daniel M. Dunlavy, Tamara G. Kolda, and Evrim Acar. 2011. Temporal link prediction using matrix and tensor factorizations. ACM Trans. Knowl. Discov. Data (TKDD) 5, 2 (2011), 10. Google Scholar
Digital Library
- Miao Fan, Qiang Zhou, Emily Chang, and Thomas Fang Zheng. 2014. Transition-based knowledge graph embedding with relational mapping properties. In Proceedings of the 28th Pacific Asia Conference on Language, Information, and Computation. 328--337.Google Scholar
- Miao Fan, Qiang Zhou, Thomas Fang Zheng, and Ralph Grishman. 2015. Large margin nearest neighbor embedding for knowledge representation. arXiv:1504.01684 (2015).Google Scholar
- Thomas Franz, Antje Schultz, Sergej Sizov, and Steffen Staab. 2009. Triplerank: Ranking Semantic Web Data by Tensor Decomposition. Springer.Google Scholar
- Kun Gai, Guangyun Chen, and Chang-shui Zhang. 2010. Learning kernels with radiuses of minimum enclosing balls. In Advances in Neural Information Processing Systems. 649--657. Google Scholar
Digital Library
- Hong Huang, Jie Tang, Lu Liu, JarDer Luo, and Xiaoming Fu. 2015. Triadic closure pattern analysis and prediction in social networks. IEEE Trans. Knowl. Data Eng. 27, 12 (2015), 3374--3389. Google Scholar
Digital Library
- Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, and Guillaume R. Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems. 3167--3175. Google Scholar
Digital Library
- Yantao Jia, Yuanzhuo Wang, Xueqi Cheng, Xiaolong Jin, and Jiafeng Guo. 2014. OpenKN: An open knowledge computational engine for network big data. In Proceedings of the 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’14). IEEE, 657--664.Google Scholar
Cross Ref
- Yantao Jia, Yuanzhuo Wang, Xiaolong Jin, Zeya Zhao, and Xueqi Cheng. 2017. Link inference in dynamic heterogeneous information network: A knapsack-based approach. IEEE Trans. Comput. Soc. Syst. (2017).Google Scholar
- Yantao Jia, Yuanzhuo Wang, Hailun Lin, Xiaolong Jin, and Xueqi Cheng. Locally adaptive translation for knowledge graph embedding. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16), 992–998. Google Scholar
Digital Library
- Yan-Tao Jia, Yuan-Zhuo Wang, and Xue-Qi Cheng. 2015. Learning to predict links by integrating structure and interaction information in microblogs. J. Comput. Sci. Technol. 30, 4 (2015), 829--842.Google Scholar
Cross Ref
- Henk A. L. Kiers. 2000. Towards a standardized notation and terminology in multiway analysis. J. Chemometr. 14, 3 (2000), 105--122.Google Scholar
Cross Ref
- Ravi Kumar, Jasmine Novak, and Andrew Tomkins. 2010. Structure and evolution of online social networks. In Link Mining: Models, Algorithms, and Applications. Springer, 337--357.Google Scholar
- Jure Leskovec, Lars Backstrom, Ravi Kumar, and Andrew Tomkins. 2008. Microscopic evolution of social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 462--470. Google Scholar
Digital Library
- Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2015. Modeling relation paths for representation learning of knowledge bases. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 705--714.Google Scholar
Cross Ref
- Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2181--2187. Google Scholar
Digital Library
- Dawei Liu, Yuanzhuo Wang, Yantao Jia, Jingyuan Li, and Zhihua Yu. 2014. LSDH: A hashing approach for large-scale link prediction in microblogs. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Google Scholar
Digital Library
- Dougal Maclaurin, David Duvenaud, and Ryan P. Adams. 2015. Gradient-based hyperparameter optimization through reversible learning. In Proceedings of the 32nd International Conference on Machine Learning. Google Scholar
Digital Library
- George A. Miller. 1995. WordNet: A lexical database for English. Commun. ACM 38, 11 (1995), 39--41. Google Scholar
Digital Library
- T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohamed, N. Nakashole, E. Platanios, A. Ritter, M. Samadi, B. Settles, R. Wang, D. Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, and J. Welling. 2015. Never-ending learning. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15). 1859--1865. Google Scholar
Digital Library
- Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. 2016. Neighborhood mixture model for knowledge base completion. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. 40--50.Google Scholar
Cross Ref
- Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2012. Factorizing yago: Scalable machine learning for linked data. In Proceedings of the 21st International Conference on World Wide Web. ACM, 271--280. Google Scholar
Digital Library
- Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro, and Karthik Sridharan. 2010. Learnability, stability and uniform convergence. J. Mach. Learn. Res. 11 (2010), 2635--2670. Google Scholar
Digital Library
- Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems. 926--934. Google Scholar
Digital Library
- Ao-Jan Su, Y. Charlie Hu, Aleksandar Kuzmanovic, and Cheng-Kok Koh. 2014. How to improve your search engine ranking: Myths and reality. ACM Trans. Web (TWEB) 8, 2 (2014), 8. Google Scholar
Digital Library
- Ilya Sutskever, Joshua B. Tenenbaum, and Ruslan R. Salakhutdinov. 2009. Modelling relational data using Bayesian clustered tensor factorization. In Advances in Neural Information Processing Systems. 1821--1828. Google Scholar
Digital Library
- Jie Tang, Tiancheng Lou, Jon Kleinberg, and Sen Wu. 2016. Transfer learning to infer social ties across heterogeneous networks. ACM Trans. Info. Syst. (TOIS) 34, 2 (2016), 7. Google Scholar
Digital Library
- Vladimir Vapnik. 2013. The Nature of Statistical Learning Theory. Springer Science 8 Business Media.Google Scholar
Digital Library
- Chenguang Wang, Yangqiu Song, Ahmed El-Kishky, Dan Roth, Ming Zhang, and Jiawei Han. 2015. Incorporating world knowledge to document clustering via heterogeneous information networks. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1215--1224. Google Scholar
Digital Library
- Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han. 2016. Text classification with heterogeneous information network kernels. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Google Scholar
Digital Library
- Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Citeseer, 1112--1119. Google Scholar
Digital Library
- Kilian Q. Weinberger and Lawrence K. Saul. 2008. Fast solvers and efficient implementations for distance metric learning. In Proceedings of the 25th International Conference on Machine Learning. ACM, 1160--1167. Google Scholar
Digital Library
- Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10 (2009), 207--244. Google Scholar
Digital Library
- Wentao Wu, Hongsong Li, Haixun Wang, and Kenny Q Zhu. 2012. Probase: A probabilistic taxonomy for text understanding. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. ACM, 481--492. Google Scholar
Digital Library
- Denghui Zhang, Manling Li, Yantao Jia, Yuanzhuo Wang, and Xueqi Cheng. 2017. Efficient parallel translating embedding for knowledge graphs. In Proceedings of the 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI’17). IEEE, 460--468. Google Scholar
Digital Library
- Jing Zhang, Zhanpeng Fang, Wei Chen, and Jie Tang. 2015. Diffusion of following links in microblogging networks. IEEE Trans. Knowl. Data Eng. 27, 8 (2015), 2093--2106.Google Scholar
Digital Library
- Zeya Zhao, Yantao Jia, and Yuanzhuo Wang. 2014. Content-structural relation inference in knowledge base. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Google Scholar
Digital Library
Index Terms
Knowledge Graph Embedding: A Locally and Temporally Adaptive Translation-Based Approach
Recommendations
Locally adaptive translation for knowledge graph embedding
AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial IntelligenceKnowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, e.g., TransE and TransH, learn embedding representation by defining a global margin-based ...
FedE: Embedding Knowledge Graphs in Federated Setting
IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge GraphsKnowledge graphs (KGs) become widespread and many organizations construct as well as maintain their own knowledge graphs. Same as the data isolation which has been a long-standing problem, knowledge graph isolation is common in real knowledge graph ...
Knowledge Graph Embedding with Diversity of Structures
WWW '17 Companion: Proceedings of the 26th International Conference on World Wide Web CompanionIn recent years, different web knowledge graphs, both free and commercial, have been created. Knowledge graphs use relations between entities to describe facts in the world. We engage in embedding a large scale knowledge graph into a continuous vector ...






Comments