Factorizing YAGO: scalable machine learning for linked data

ABSTRACT

Vast amounts of structured information have been published in the Semantic Web's Linked Open Data (LOD) cloud and their size is still growing rapidly. Yet, access to this information via reasoning and querying is sometimes difficult, due to LOD's size, partial data inconsistencies and inherent noisiness. Machine Learning offers an alternative approach to exploiting LOD's data with the advantages that Machine Learning algorithms are typically robust to both noise and data inconsistencies and are able to efficiently utilize non-deterministic dependencies in the data. From a Machine Learning point of view, LOD is challenging due to its relational nature and its scale. Here, we present an efficient approach to relational learning on LOD data, based on the factorization of a sparse tensor that scales to data consisting of millions of entities, hundreds of relations and billions of known facts. Furthermore, we show how ontological knowledge can be incorporated in the factorization to improve learning results and how computation can be distributed across multiple nodes. We demonstrate that our approach is able to factorize the YAGO~2 core ontology and globally predict statements for this large knowledge base using a single dual-core desktop computer. Furthermore, we show experimentally that our approach achieves good results in several relational learning tasks that are relevant to Linked Data. Once a factorization has been computed, our model is able to predict efficiently, and without any additional training, the likelihood of any of the 4.3 ⋅ 10¹⁴ possible triples in the YAGO~2 core ontology.

References

1. M. Ankerst, M. Breunig, H. Kriegel, and J. Sander. OPTICS: ordering points to identify the clustering structure. In ACM SIGMOD Record, volume 28, page 49--60, 1999.
2. S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives. Dbpedia: A nucleus for a web of open data. The Semantic Web, page 722--735, 2008.
3. S. Auer and J. Lehmann. Creating knowledge out of interlinked data. Semantic Web, 1(1):97--104, Jan. 2010.
4. B. W. Bader, R. A. Harshman, and T. G. Kolda. Temporal analysis of semantic graphs using ASALSAN. In Seventh IEEE International Conference on Data Mining (ICDM 2007), pages 33--42, Omaha, NE, USA, Oct. 2007.
5. V. Bicer, T. Tran, and A. Gossen. Relational kernel machines for learning from Graph-Structured RDF data. The Semantic Web: Research and Applications, page 47--62, 2011.
6. C. Bizer, T. Heath, and T. Berners-Lee. Linked data-the story so far. International Journal on Semantic Web and Information Systems, 5(3):1--22, 2009.
7. S. Bloehdorn and Y. Sure. Kernel methods for mining instance data in ontologies. In Proceedings of the 6th international The semantic web and 2nd Asian conference on Asian semantic web conference, page 58--71, 2007.
8. R. Bro. PARAFAC. tutorial and applications. Chemometrics and Intelligent Laboratory Systems, 38(2):149--171, 1997.
9. C. d'Amato, N. Fanizzi, and F. Esposito. Non-parametric statistical learning methods for inductive classifiers in semantic knowledge bases. In Proceedings of the 2008 IEEE International Conference on Semantic Computing, page 291--298, Washington, DC, USA, 2008. IEEE Computer Society.
10. J. Davis and M. Goadrich. The relationship between Precision-Recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning, page 233--240, 2006.
11. N. Fanizzi, C. D'Amato, and F. Esposito. DL-FOIL concept learning in description logics. In Proceedings of the 18th international conference on Inductive Logic Programming, ILP '08, page 107--121, Berlin, Heidelberg, 2008. Springer-Verlag.
12. T. Franz, A. Schultz, S. Sizov, and S. Staab. Triplerank: Ranking semantic web data by tensor decomposition. The Semantic Web-ISWC 2009, page 213--228, 2009.
13. H. Halpin, P. Hayes, J. McCusker, D. Mcguinness, and H. Thompson. When owl: same As isn't the same: An analysis of identity in linked data. The Semantic Web--ISWC 2010, page 305--320, 2010.
14. S. Hellmann, J. Lehmann, and S. Auer. Learning of OWL class descriptions on very large knowledge bases. Int. J. Semantic Web Inf. Syst, 5(2):25--48, 2009.
15. P. Hitzler and F. van Harmelen. A reasonable semantic web. Semantic Web, 1(1):39--44, 2010.
16. A. Hogan, A. Harth, A. Passant, S. Decker, and A. Polleres. Weaving the pedantic web. Linked Data on the Web (LDOW 2010), 2010.
17. Y. Huang, V. Tresp, M. Bundschus, and A. Rettinger. Multivariate structured prediction for learning on semantic web. 2010.
18. C. Kiefer, A. Bernstein, and A. Locher. Adding data mining support to SPARQL via statistical relational learning methods. In Proceedings of the 5th European semantic web conference, pages 478--492, 2008.
19. S. Kok and P. Domingos. Statistical predicate invention. In Proceedings of the 24th international conference on Machine learning, page 433--440, 2007.
20. T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455, 2009.
21. H. T. Lin, N. Koul, and V. Honavar. Learning relational bayesian classifiers from RDF data. In Proceedings of the International Semantic Web Conference (ISWC 2011), 2011. In press.
22. M. Nickel, V. Tresp, and H. Kriegel. A Three-Way model for collective learning on Multi-Relational data. In Proceedings of the 28th International Conference on Machine Learning, ICML '11, pages 809--816, Bellevue, WA, USA, 2011. ACM.
23. S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web, page 811--820, 2010.
24. M. Richardson and P. Domingos. Markov logic networks. Machine Learning, 62(1):107--136, 2006.
25. D. Roy, C. Kemp, V. Mansinghka, and J. Tenenbaum. Learning annotated hierarchies from relational data. Advances in neural information processing systems, 19:1185, 2007.
26. P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad. Collective classification in network data. AI Magazine, 29(3):93, 2008.
27. F. Suchanek, G. Kasneci, and G. Weikum. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, page 697--706, 2007.
28. J. Sun, D. Tao, and C. Faloutsos. Beyond streams and graphs: dynamic tensor analysis. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, page 374--383, 2006.
29. I. Sutskever, R. Salakhutdinov, and J. B. Tenenbaum. Modelling relational data using bayesian clustered tensor factorization. Advances in Neural Information Processing Systems, 22, 2009.
30. P. Tan, M. Steinbach, V. Kumar, et al. Introduction to data mining. Pearson Addison Wesley Boston, 2006.
31. J. Völker and M. Niepert. Statistical schema induction. The Semantic Web: Research and Applications, page 124--138, 2011.
32. K. Weinberger, A. Dasgupta, J. Langford, A. Smola, and J. Attenberg. Feature hashing for large scale multitask learning. In Proceedings of the 26th Annual International Conference on Machine Learning, page 1113--1120, 2009.
33. M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning, Bellevue, WA, USA, 2011.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

    • Machine learning approaches

      • Markov decision processes

• Theory of computation

  • Theory and algorithms for application domains

    • Machine learning theory

      • Markov decision processes