skip to main content
research-article

Knowledge Graph Embedding: A Locally and Temporally Adaptive Translation-Based Approach

Authors Info & Claims
Published:22 December 2017Publication History
Skip Abstract Section

Abstract

A knowledge graph is a graph with entities of different types as nodes and various relations among them as edges. The construction of knowledge graphs in the past decades facilitates many applications, such as link prediction, web search analysis, question answering, and so on. Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, for example, TransE, TransH, and TransR, learn the embedding representation by defining a global margin-based loss function over the data. However, the loss function is determined during experiments whose parameters are examined among a closed set of candidates. Moreover, embeddings over two knowledge graphs with different entities and relations share the same set of candidates, ignoring the locality of both graphs. This leads to the limited performance of embedding related applications. In this article, a locally adaptive translation method for knowledge graph embedding, called TransA, is proposed to find the loss function by adaptively determining its margin over different knowledge graphs. Then the convergence of TransA is verified from the aspect of its uniform stability. To make the embedding methods up-to-date when new vertices and edges are added into the knowledge graph, the incremental algorithm for TransA, called iTransA, is proposed by adaptively adjusting the optimal margin over time. Experiments on four benchmark data sets demonstrate the superiority of the proposed method, as compared to the state-of-the-art ones.

References

  1. Yoshua Bengio, Aaron Courville, and Pierre Vincent. 2013. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 8 (2013), 1798--1828. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, Feb (2012), 281--305. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 1247--1250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 127--135.Google ScholarGoogle Scholar
  5. Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2014. A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94, 2 (2014), 233--259. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems. 2787--2795. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Proceedings of the Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. 1992. A training algorithm for optimal margin classifiers. In Proceedings of the 5th Annual Workshop on Computational Learning Theory. ACM, 144--152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. J. Mach. Learn. Res. 2 (2002), 499--526. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Toward an architecture for never-ending language learning. In Proceedings of AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Kai-Wei Chang, Wen-tau Yih, and Christopher Meek. 2013. Multi-relational latent semantic analysis. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP’13). 1602--1612.Google ScholarGoogle Scholar
  12. Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. 2014. Typed tensor decomposition of knowledge bases for relation extraction. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1568--1579.Google ScholarGoogle ScholarCross RefCross Ref
  13. Olivier Chapelle, Vladimir Vapnik, Olivier Bousquet, and Sayan Mukherjee. 2002. Choosing multiple parameters for support vector machines. Mach. Learn. 46, 1 (2002), 131--159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. 2000. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 4 (2000), 1253--1278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Foo Chuan-sheng Do, B. Chuong, and Andrew Y. Ng. 2007. Efficient multiple hyperparameter learning for log-linear models. In Advances in Neural Information Processing Systems. 377--384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Huyen Do, Alexandros Kalousis, and Melanie Hilario. 2009. Feature Weighting Using Margin and Radius Based Error Bound Optimization in SVMs. Springer.Google ScholarGoogle Scholar
  17. Huyen Do, Alexandros Kalousis, Jun Wang, and Adam Woznica. 2012. A metric learning perspective of SVM: On the relation of LMNN and SVM. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 308--317.Google ScholarGoogle Scholar
  18. Sergio Duarte Torres, Ingmar Weber, and Djoerd Hiemstra. 2014. Analysis of search and browsing behavior of young users on the web. ACM Trans. Web (TWEB) 8, 2 (2014), 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Daniel M. Dunlavy, Tamara G. Kolda, and Evrim Acar. 2011. Temporal link prediction using matrix and tensor factorizations. ACM Trans. Knowl. Discov. Data (TKDD) 5, 2 (2011), 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Miao Fan, Qiang Zhou, Emily Chang, and Thomas Fang Zheng. 2014. Transition-based knowledge graph embedding with relational mapping properties. In Proceedings of the 28th Pacific Asia Conference on Language, Information, and Computation. 328--337.Google ScholarGoogle Scholar
  21. Miao Fan, Qiang Zhou, Thomas Fang Zheng, and Ralph Grishman. 2015. Large margin nearest neighbor embedding for knowledge representation. arXiv:1504.01684 (2015).Google ScholarGoogle Scholar
  22. Thomas Franz, Antje Schultz, Sergej Sizov, and Steffen Staab. 2009. Triplerank: Ranking Semantic Web Data by Tensor Decomposition. Springer.Google ScholarGoogle Scholar
  23. Kun Gai, Guangyun Chen, and Chang-shui Zhang. 2010. Learning kernels with radiuses of minimum enclosing balls. In Advances in Neural Information Processing Systems. 649--657. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Hong Huang, Jie Tang, Lu Liu, JarDer Luo, and Xiaoming Fu. 2015. Triadic closure pattern analysis and prediction in social networks. IEEE Trans. Knowl. Data Eng. 27, 12 (2015), 3374--3389. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, and Guillaume R. Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems. 3167--3175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Yantao Jia, Yuanzhuo Wang, Xueqi Cheng, Xiaolong Jin, and Jiafeng Guo. 2014. OpenKN: An open knowledge computational engine for network big data. In Proceedings of the 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’14). IEEE, 657--664.Google ScholarGoogle ScholarCross RefCross Ref
  27. Yantao Jia, Yuanzhuo Wang, Xiaolong Jin, Zeya Zhao, and Xueqi Cheng. 2017. Link inference in dynamic heterogeneous information network: A knapsack-based approach. IEEE Trans. Comput. Soc. Syst. (2017).Google ScholarGoogle Scholar
  28. Yantao Jia, Yuanzhuo Wang, Hailun Lin, Xiaolong Jin, and Xueqi Cheng. Locally adaptive translation for knowledge graph embedding. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16), 992–998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Yan-Tao Jia, Yuan-Zhuo Wang, and Xue-Qi Cheng. 2015. Learning to predict links by integrating structure and interaction information in microblogs. J. Comput. Sci. Technol. 30, 4 (2015), 829--842.Google ScholarGoogle ScholarCross RefCross Ref
  30. Henk A. L. Kiers. 2000. Towards a standardized notation and terminology in multiway analysis. J. Chemometr. 14, 3 (2000), 105--122.Google ScholarGoogle ScholarCross RefCross Ref
  31. Ravi Kumar, Jasmine Novak, and Andrew Tomkins. 2010. Structure and evolution of online social networks. In Link Mining: Models, Algorithms, and Applications. Springer, 337--357.Google ScholarGoogle Scholar
  32. Jure Leskovec, Lars Backstrom, Ravi Kumar, and Andrew Tomkins. 2008. Microscopic evolution of social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 462--470. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2015. Modeling relation paths for representation learning of knowledge bases. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 705--714.Google ScholarGoogle ScholarCross RefCross Ref
  34. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2181--2187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Dawei Liu, Yuanzhuo Wang, Yantao Jia, Jingyuan Li, and Zhihua Yu. 2014. LSDH: A hashing approach for large-scale link prediction in microblogs. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Dougal Maclaurin, David Duvenaud, and Ryan P. Adams. 2015. Gradient-based hyperparameter optimization through reversible learning. In Proceedings of the 32nd International Conference on Machine Learning. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. George A. Miller. 1995. WordNet: A lexical database for English. Commun. ACM 38, 11 (1995), 39--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohamed, N. Nakashole, E. Platanios, A. Ritter, M. Samadi, B. Settles, R. Wang, D. Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, and J. Welling. 2015. Never-ending learning. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15). 1859--1865. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. 2016. Neighborhood mixture model for knowledge base completion. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. 40--50.Google ScholarGoogle ScholarCross RefCross Ref
  40. Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2012. Factorizing yago: Scalable machine learning for linked data. In Proceedings of the 21st International Conference on World Wide Web. ACM, 271--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro, and Karthik Sridharan. 2010. Learnability, stability and uniform convergence. J. Mach. Learn. Res. 11 (2010), 2635--2670. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems. 926--934. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Ao-Jan Su, Y. Charlie Hu, Aleksandar Kuzmanovic, and Cheng-Kok Koh. 2014. How to improve your search engine ranking: Myths and reality. ACM Trans. Web (TWEB) 8, 2 (2014), 8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Ilya Sutskever, Joshua B. Tenenbaum, and Ruslan R. Salakhutdinov. 2009. Modelling relational data using Bayesian clustered tensor factorization. In Advances in Neural Information Processing Systems. 1821--1828. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jie Tang, Tiancheng Lou, Jon Kleinberg, and Sen Wu. 2016. Transfer learning to infer social ties across heterogeneous networks. ACM Trans. Info. Syst. (TOIS) 34, 2 (2016), 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Vladimir Vapnik. 2013. The Nature of Statistical Learning Theory. Springer Science 8 Business Media.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Chenguang Wang, Yangqiu Song, Ahmed El-Kishky, Dan Roth, Ming Zhang, and Jiawei Han. 2015. Incorporating world knowledge to document clustering via heterogeneous information networks. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1215--1224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han. 2016. Text classification with heterogeneous information network kernels. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Citeseer, 1112--1119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Kilian Q. Weinberger and Lawrence K. Saul. 2008. Fast solvers and efficient implementations for distance metric learning. In Proceedings of the 25th International Conference on Machine Learning. ACM, 1160--1167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10 (2009), 207--244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Wentao Wu, Hongsong Li, Haixun Wang, and Kenny Q Zhu. 2012. Probase: A probabilistic taxonomy for text understanding. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. ACM, 481--492. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Denghui Zhang, Manling Li, Yantao Jia, Yuanzhuo Wang, and Xueqi Cheng. 2017. Efficient parallel translating embedding for knowledge graphs. In Proceedings of the 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI’17). IEEE, 460--468. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Jing Zhang, Zhanpeng Fang, Wei Chen, and Jie Tang. 2015. Diffusion of following links in microblogging networks. IEEE Trans. Knowl. Data Eng. 27, 8 (2015), 2093--2106.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Zeya Zhao, Yantao Jia, and Yuanzhuo Wang. 2014. Content-structural relation inference in knowledge base. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Knowledge Graph Embedding: A Locally and Temporally Adaptive Translation-Based Approach

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!