Grale: Designing Networks for Graph Learning

ABSTRACT

How can we find the right graph for semi-supervised learning? In real world applications, the choice of which edges to use for computation is the first step in any graph learning process. Interestingly, there are often many types of similarity available to choose as the edges between nodes, and the choice of edges can drastically affect the performance of downstream semi-supervised learning systems. However, despite the importance of graph design, most of the literature assumes that the graph is static.

In this work, we present Grale, a scalable method we have developed to address the problem of graph design for graphs with billions of nodes. Grale operates by fusing together different measures of (potentially weak) similarity to create a graph which exhibits high task-specific homophily between its nodes. Grale is designed for running on large datasets. We have deployed Grale in more than 20 different industrial settings at Google, including datasets which have tens of billions of nodes, and hundreds of trillions of potential edges to score. By employing locality sensitive hashing techniques, we greatly reduce the number of pairs that need to be scored, allowing us to learn a task specific model and build the associated nearest neighbor graph for such datasets in hours, rather than the days or even weeks that might be required otherwise.

We illustrate this through a case study where we examine the application of Grale to an abuse classification problem on YouTube with hundreds of million of items. In this application, we find that Grale detects a large number of malicious actors on top of hard-coded rules and content classifiers, increasing the total recall by 89% over those approaches alone.

References

1. S. Abu-El-Haija et al. 2017. Learning edge representations via low-rank asymmetric projections. CIKM.
2. M. Al Hasan et al. 2006. Link prediction using supervised learning. SDM Workshops.
3. A. Blum et al. 2001. Learning from labeled and unlabeled data using graph mincuts. ICML.
4. J. Bromley et al. 1994. Signature verification using a "siamese" time delay neural network. NIPS.
5. I. Chami et al. 2020. Machine learning on graphs: a model and comprehensive taxonomy. arXiv preprint arXiv:2005.03675.
6. H. Chen et al. 2018. A tutorial on network embeddings. arXiv preprint arXiv:1808.02590.
7. H. Chen et al. 2018. Enhanced network embeddings via exploiting edge labels. CIKM.
8. H. Chen et al. 2005. Link prediction approach to collaborative filtering. JCDL.
9. P. Cui et al. 2018. A survey on network embedding. TKDE.
10. C. A. R. de Sousa et al. 2013. Influence of graph construction on semi-supervised learning. ECML/PKDD.
11. C. Kanich et al. 2011. Show me the money: characterizing spam-advertised revenue. SEC.
12. M. Karasuyama et al. 2017. Adaptive edge weighting for graph-based learning algorithms. Mach. Learn.
13. D. P. Kingma et al. 2014. Semi-supervised learning with deep generative models. NIPS.
14. G. Koch. 2015. Siamese neural networks for one-shot image recognition. ICML Workshops.
15. K. Levchenko et al. 2011. Click trajectories: end-to-end analysis of the spam value chain. S&P.
16. L. v. d. Maaten et al. 2008. Visualizing data using t-sne. JMLR.
17. D. McCoy et al. 2012. Pharmaleaks: understanding the business of online pharmaceutical affiliate programs. SEC.
18. E. Müller et al. 2009. Evaluating clustering in subspace projections of high dimensional data. VLDB.
19. A. Murua et al. 2008. On potts model clustering, kernel k-means and density estimation. Journal of Computational and Graphical Statistics.
20. B. Perozzi et al. 2014. Deepwalk: online learning of social representations. KDD.
21. B. Perozzi et al. 2014. Focused clustering and outlier detection in large attributed graphs. KDD.
22. B. Perozzi et al. 2016. When recommendation goes wrong: anomalous link discovery in recommendation networks. KDD.
23. N. Ponomareva et al. 2017. Compact multi-class boosted trees. Big Data.
24. N. Ponomareva et al. 2017. Tf boosted trees: a scalable tensorflow based framework for gradient boosting. ECML/PKDD. Y. Altun et al., editors.
25. S. Ravi et al. 2016. Large scale distributed semi-supervised learning using streaming approximation. Artificial Intelligence and Statistics, 519--528.
26. G. T. Report. [n. d.] https://transparencyreport.google.com/youtube-policy/ removals. ().
27. T. Salimans et al. 2016. Improved techniques for training gans. NIPS.
28. D. Samosseiko. 2009. The partnerka-what is it, and why should you care. Virus Bulletin Conference.
29. X. Wu et al. 2018. A quest for structure: jointly learning the graph structure and semi-supervised classification. CIKM.
30. Z. Yang et al. 2016. Revisiting semi-supervised learning with graph embeddings. ICML.
31. YouTube. [n. d.] https://www.youtube.com/intl/en-GB/about/press/. ().
32. Y.-M. Zhang et al. 2013. Fast knn graph construction with locality sensitive hashing. ECML/PKKD.
33. D. Zhou et al. 2003. Learning with local and global consistency. NIPS.
34. X. Zhu et al. 2003. Semi-supervised learning using gaussian fields and harmonic functions. ICML.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

    • Learning settings

      • Semi-supervised learning settings

    • Machine learning approaches

      • Instance-based learning