skip to main content
research-article

Decoupled Variational Embedding for Signed Directed Networks

Authors Info & Claims
Published:28 October 2020Publication History
Skip Abstract Section

Abstract

Node representation learning for signed directed networks has received considerable attention in many real-world applications such as link sign prediction, node classification, and node recommendation. The challenge lies in how to adequately encode the complex topological information of the networks. Recent studies mainly focus on preserving the first-order network topology that indicates the closeness relationships of nodes. However, these methods generally fail to capture the high-order topology that indicates the local structures of nodes and serves as an essential characteristic of the network topology. In addition, for the first-order topology, the additional value of non-existent links is largely ignored. In this article, we propose to learn more representative node embeddings by simultaneously capturing the first-order and high-order topology in signed directed networks. In particular, we reformulate the representation learning problem on signed directed networks from a variational auto-encoding perspective and further develop a decoupled variational embedding (DVE) method. DVE leverages a specially designed auto-encoder structure to capture both the first-order and high-order topology of signed directed networks, and thus learns more representative node embeddings. Extensive experiments are conducted on three widely used real-world datasets. Comprehensive results on both link sign prediction and node recommendation task demonstrate the effectiveness of DVE. Qualitative results and analysis are also given to provide a better understanding of DVE.

References

  1. Luca Maria Aiello, Alain Barrat, Rossano Schifanella, Ciro Cattuto, Benjamin Markines, and Filippo Menczer. 2012. Friendship prediction and homophily in social media. ACM Trans. Web 6, 2 (June 2012). DOI:https://doi.org/10.1145/2180861.2180866Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. 2018. Relational inductive biases, deep learning, and graph networks. Arxiv Preprint Arxiv:1806.01261 (2018).Google ScholarGoogle Scholar
  3. Mikhail Belkin and Partha Niyogi. 2002. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 585--591.Google ScholarGoogle Scholar
  4. Smriti Bhagat, Graham Cormode, and S. Muthukrishnan. 2011. Node classification in social networks. In Social Network Data Analytics. Springer, 115--148.Google ScholarGoogle Scholar
  5. Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. 2015. Importance weighted autoencoders. Arxiv Preprint Arxiv:1509.00519 (2015).Google ScholarGoogle Scholar
  6. Fidel Cacheda, Roi Blanco, and Nicola Barbieri. 2018. Characterizing and predicting users’ behavior on local search queries. ACM Trans. Web 12, 2 (May 2018). DOI:https://doi.org/10.1145/3157059Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Dorwin Cartwright and Frank Harary. 1956. Structural balance: A generalization of Heider’s theory. Psychol. Rev. 63, 5 (1956), 277.Google ScholarGoogle ScholarCross RefCross Ref
  8. Jie Chen, Tengfei Ma, and Cao Xiao. 2018. FastGCN: Fast learning with graph convolutional networks via importance sampling. Arxiv Preprint Arxiv:1801.10247 (2018).Google ScholarGoogle Scholar
  9. Jianfei Chen, Jun Zhu, and Le Song. 2017. Stochastic training of graph convolutional networks with variance reduction. Arxiv Preprint Arxiv:1710.10568 (2017).Google ScholarGoogle Scholar
  10. Wenhu Chen, Wenhan Xiong, Xifeng Yan, and William Wang. 2018. Variational knowledge graph reasoning. Arxiv Preprint Arxiv:1803.06581 (2018).Google ScholarGoogle Scholar
  11. Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 3844--3852.Google ScholarGoogle Scholar
  12. Tyler Derr, Yao Ma, and Jiliang Tang. 2018. Signed graph convolutional network. Arxiv Preprint Arxiv:1808.06354 (2018).Google ScholarGoogle Scholar
  13. Carl Doersch. 2016. Tutorial on variational autoencoders. Arxiv Preprint Arxiv:1606.05908 (2016).Google ScholarGoogle Scholar
  14. Yuxiao Dong, Jing Zhang, Jie Tang, Nitesh V. Chawla, and Bai Wang. 2015. CoupledLP: Link prediction in coupled networks. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 199--208.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Rossano Gaeta. 2018. A model of information diffusion in interconnected online social networks. ACM Trans. Web 12, 2 (June 2018). DOI:https://doi.org/10.1145/3160000Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jean Gallier. 2016. Spectral theory of unsigned and signed graphs. applications to graph clustering: A survey. Arxiv Preprint Arxiv:1601.04692 (2016).Google ScholarGoogle Scholar
  17. Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra. 2015. Draw: A recurrent neural network for image generation. In Proceedings of the 32nd International Conference on Machine Learning.Google ScholarGoogle Scholar
  18. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 855--864.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 1024--1034.Google ScholarGoogle Scholar
  20. William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation learning on graphs: Methods and applications. Arxiv Preprint Arxiv:1709.05584 (2017).Google ScholarGoogle Scholar
  21. Ruining He and Julian McAuley. 2016. VBPR: Visual Bayesian personalized ranking from implicit feedback. In Proceedings of the 30th AAAI Conference on Artificial Intelligence.Google ScholarGoogle Scholar
  22. Cho-Jui Hsieh, Kai-Yang Chiang, and Inderjit S. Dhillon. 2012. Low rank modeling of signed networks. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 507--515.Google ScholarGoogle Scholar
  23. Ya Zhang Huangjie Zheng, Jiangchao Yao and Ivor W. Tsang. 2018. Degeneration in VAE: in the light of Fisher information loss. ArXiv abs/1802.06677 (2018).Google ScholarGoogle Scholar
  24. Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. Arxiv Preprint Arxiv:1505.05770 (2015).Google ScholarGoogle Scholar
  25. Andrej Karpathy, et al. 2016. Cs231n convolutional neural networks for visual recognition. Neural Netw. 1 (2016).Google ScholarGoogle Scholar
  26. Junghwan Kim, Haekyu Park, Ji-Eun Lee, and U. Kang. 2018. SIDE: Representation learning in signed directed networks. In Proceedings of the World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 509--518.Google ScholarGoogle Scholar
  27. Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational Bayes. Arxiv Preprint Arxiv:1312.6114 (2013).Google ScholarGoogle Scholar
  28. Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. 2018. Neural relational inference for interacting systems. Arxiv Preprint Arxiv:1802.04687 (2018).Google ScholarGoogle Scholar
  29. Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. Arxiv Preprint Arxiv:1609.02907 (2016).Google ScholarGoogle Scholar
  30. Thomas N. Kipf and Max Welling. 2016. Variational graph auto-encoders. Arxiv Preprint Arxiv:1611.07308 (2016).Google ScholarGoogle Scholar
  31. Jérôme Kunegis, Julia Preusse, and Felix Schwagereit. 2013. What is the added value of negative links in online social networks? In Proceedings of the 22nd International Conference on World Wide Web. ACM, 727--736.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Jérôme Kunegis, Stephan Schmidt, Andreas Lommatzsch, Jürgen Lerner, Ernesto W. De Luca, and Sahin Albayrak. 2010. Spectral analysis of signed graphs for clustering, prediction and visualization. In Proceedings of the SIAM International Conference on Data Mining. SIAM, 559--570.Google ScholarGoogle ScholarCross RefCross Ref
  33. Matt J. Kusner, Brooks Paige, and José Miguel Hernández-Lobato. 2017. Grammar variational autoencoder. In Proceedings of the 34th International Conference on Machine Learning, Vol. 70. JMLR.org, 1945--1954.Google ScholarGoogle Scholar
  34. Jure Leskovec, Daniel Huttenlocher, and Jon Kleinberg. 2010. Predicting positive and negative links in online social networks. In Proceedings of the 19th International Conference on World Wide Web. ACM, 641--650.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. David Liben-Nowell and Jon Kleinberg. 2007. The link-prediction problem for social networks. J. Amer. Society Inf. Sci. Technol. 58, 7 (2007), 1019--1031.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Qiang Liu, Shu Wu, and Liang Wang. 2017. DeepStyle: Learning user preferences for visual recommendation. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 841--844.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, Nov. (2008), 2579--2605.Google ScholarGoogle Scholar
  38. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 3111--3119.Google ScholarGoogle Scholar
  39. Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric transitivity preserving graph embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1105--1114.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Symeon Papadopoulos, Yiannis Kompatsiaris, Athena Vakali, and Ploutarchos Spyridonos. 2012. Community detection in social media. Data Mining Knowl. Disc. 24, 3 (2012), 515--554.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 701--710.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. 2016. Variational autoencoder for deep learning of images, labels and captions. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 2352--2360.Google ScholarGoogle Scholar
  43. Yi Qian and Sibel Adali. 2013. Extended structural balance theory for modeling trust in social networks. In Proceedings of the 11th International Conference on Privacy, Security and Trust (PST’13). IEEE, 283--290.Google ScholarGoogle ScholarCross RefCross Ref
  44. Yi Qian and Sibel Adali. 2014. Foundations of trust and distrust in networks: Extended structural balance theory. ACM Trans. Web 8, 3 (July 2014). DOI:https://doi.org/10.1145/2628438Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. 2018. Network embedding as matrix factorization: Unifying DeepWalk, LINE, PTE, and node2vec. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 459--467.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 452--461.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Steffen Rendle and Lars Schmidt-Thieme. 2010. Pairwise interaction tensor factorization for personalized tag recommendation. In Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. ACM, 81--90.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. Arxiv Preprint Arxiv:1505.05770 (2015).Google ScholarGoogle Scholar
  49. Tim Salimans, Diederik Kingma, and Max Welling. 2015. Markov chain Monte Carlo and variational inference: Bridging the gap. In Proceedings of the International Conference on Machine Learning. 1218--1226.Google ScholarGoogle Scholar
  50. Xiaobo Shen, Shirui Pan, Weiwei Liu, Yew-Soon Ong, and Quan-Sen Sun. 2018. Discrete network embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. AAAI Press, 3549--3555.Google ScholarGoogle ScholarCross RefCross Ref
  51. Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 3483--3491.Google ScholarGoogle Scholar
  52. Jiliang Tang, Shiyu Chang, Charu Aggarwal, and Huan Liu. 2015. Negative link prediction in social media. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. ACM, 87--96.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1067--1077.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Tijmen Tieleman and Geoffery Hinton. 2014. RMSprop gradient optimization. Retrieved from http://www. cs. toronto. edu/tijmen/csc321/slides/lecture_slides_lec6. pdf.Google ScholarGoogle Scholar
  55. Jakub M. Tomczak and Max Welling. 2017. VAE with a VampPrior. Arxiv Preprint Arxiv:1705.07120 (2017).Google ScholarGoogle Scholar
  56. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. Arxiv Preprint Arxiv:1710.10903 (2017).Google ScholarGoogle Scholar
  57. Patricia Victor, Nele Verbiest, Chris Cornelis, and Martine De Cock. 2013. Enhancing the trust-based recommendation process with explicit distrust. ACM Trans. Web 7, 2 (May 2013). DOI:https://doi.org/10.1145/2460383.2460385Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1225--1234.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Hongwei Wang, Jia Wang, Miao Zhao, Jiannong Cao, and Minyi Guo. 2017. Joint topic-semantic-aware social recommendation for online voting. In Proceedings of the ACM Conference on Information and Knowledge Management. ACM, 347--356.Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Hongwei Wang, Fuzheng Zhang, Min Hou, Xing Xie, Minyi Guo, and Qi Liu. 2018. SHINE: Signed heterogeneous information network embedding for sentiment link prediction. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 592--600.Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Suhang Wang, Charu Aggarwal, Jiliang Tang, and Huan Liu. 2017. Attributed signed network embedding. In Proceedings of the ACM Conference on Information and Knowledge Management. ACM, 137--146.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Suhang Wang, Jiliang Tang, Charu Aggarwal, Yi Chang, and Huan Liu. 2017. Signed network embedding in social media. In Proceedings of the SIAM International Conference on Data Mining. SIAM, 327--335.Google ScholarGoogle ScholarCross RefCross Ref
  63. Suhang Wang, Jiliang Tang, Charu Aggarwal, and Huan Liu. 2016. Linked document embedding for classification. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 115--124.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2019. A comprehensive survey on graph neural networks. Arxiv Preprint Arxiv:1901.00596 (2019).Google ScholarGoogle Scholar
  65. Bingbing Xu, Huawei Shen, Qi Cao, Keting Cen, and Xueqi Cheng. 2019. Graph convolutional networks using heat kernel for semi-supervised learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 1928--1934.Google ScholarGoogle ScholarCross RefCross Ref
  66. Mingzhang Yin and Mingyuan Zhou. 2018. Semi-implicit variational inference. In Proceedings of the International Conference on Machine Learning.Google ScholarGoogle Scholar
  67. Shuhan Yuan, Xintao Wu, and Yang Xiang. 2017. SNE: Signed network embedding. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 183--195.Google ScholarGoogle ScholarCross RefCross Ref
  68. Xianchao Zhang, Zhaoxing Li, Shaoping Zhu, and Wenxin Liang. 2016. Detecting spam and promoting campaigns in Twitter. ACM Trans. Web 10 (Feb. 2016). DOI:https://doi.org/10.1145/2846102Google ScholarGoogle Scholar
  69. Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2018. Deep learning on graphs: A survey. Arxiv Preprint Arxiv:1812.04202 (2018).Google ScholarGoogle Scholar
  70. Shengjia Zhao, Jiaming Song, and Stefano Ermon. 2017. InfoVAE: Information maximizing variational autoencoders. Arxiv Preprint Arxiv:1706.02262 (2017).Google ScholarGoogle Scholar
  71. Shengjia Zhao, Jiaming Song, and Stefano Ermon. 2017. Towards deeper understanding of variational autoencoding models. Arxiv Preprint Arxiv:1702.08658 (2017).Google ScholarGoogle Scholar
  72. Huangjie Zheng, Jiangchao Yao, Ya Zhang, and Ivor W. Tsang. 2018. Degeneration in VAE: In the light of Fisher information loss. Arxiv Preprint Arxiv:1802.06677 (2018).Google ScholarGoogle Scholar
  73. Huangjie Zheng, Jiangchao Yao, Ya Zhang, Ivor W. Tsang, and Jia Wang. 2019. Understanding VAEs in Fisher-Shannon plane. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5917--5924.Google ScholarGoogle ScholarCross RefCross Ref
  74. Huangjie Zheng, Jiangchao Yao, Ya Zhang, and Ivor Wai-Hung Tsang. 2019. Understanding VAEs in Fisher-Shannon plane. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  75. Chang Zhou, Yuqiong Liu, Xiaofei Liu, Zhongyi Liu, and Jun Gao. 2017. Scalable graph embedding for asymmetric proximity. In Proceedings of the AAAI Conference on Artificial Intelligence. 2942--2948.Google ScholarGoogle Scholar

Index Terms

  1. Decoupled Variational Embedding for Signed Directed Networks

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!