skip to main content
research-article

Script Event Prediction via Multilingual Event Graph Networks

Authors Info & Claims
Published:27 December 2022Publication History
Skip Abstract Section

Abstract

Predicting what happens next in text plays a critical role in building NLP applications. Many methods including count-based and neural-network-based have been proposed to tackle the task called script event prediction: predicting the most suitable subsequent event from a candidate list given a chain of narrative events (context). However, two problems including event ambiguity and evidence bias hinder the performance of these monolingual approaches. The former means that some events in the event chain are ambiguous. The latter means that both the wrong and correct candidate events can obtain sufficient support from the event context. In this article, we propose a novel multilingual approach to address two issues simultaneously. Specifically, to alleviate the event ambiguity problem, we project the monolingual event chains to parallel cross-lingual event chains, which can provide complementary information for monolingual event disambiguation. To deal with the evidence bias problem, we construct two monolingual event graphs and a cross-lingual event aligned graph to fully explore connections between events. What’s more, we design a graph attention mechanism to model the confidence of the complement clues, which controls the information integration from various languages. By modeling the events with graphs instead of pairs or chains, the model can compare the candidate subsequent events simultaneously and choose the more suitable subsequent event as the final answer. Extensive experiments were conducted on the widely used New York Times corpus for script event prediction task and experimental results show that our approach outperforms previous models.

REFERENCES

  1. [1] Abu-El-Haija Sami, Perozzi Bryan, Kapoor Amol, Alipourfard Nazanin, Lerman Kristina, Harutyunyan Hrayr, Steeg Greg Ver, and Galstyan Aram. 2019. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In Proceedings of the International Conference on Machine Learning. PMLR, 2129.Google ScholarGoogle Scholar
  2. [2] Bahdanau Dzmitry, Cho Kyung Hyun, and Bengio Yoshua. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations.Google ScholarGoogle Scholar
  3. [3] Barba Edoardo, Procopio Luigi, Campolungo Niccolo, Pasini Tommaso, and Navigli Roberto. 2021. Mulan: Multilingual label propagation for word sense disambiguation. In Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence. 38373844.Google ScholarGoogle Scholar
  4. [4] Bruna Joan, Zaremba Wojciech, Szlam Arthur, and LeCun Yann. 2014. Spectral networks and deep locally connected networks on graphs. In Proceedings of the 2nd International Conference on Learning Representations.Google ScholarGoogle Scholar
  5. [5] Chambers Nathanael and Jurafsky Dan. 2008. Unsupervised learning of narrative event chains. In Proceedings of the ACL-08: HLT. 789797.Google ScholarGoogle Scholar
  6. [6] Curran James R., Clark Stephen, and Bos Johan. 2007. Linguistically motivated large-scale NLP with C&C and Boxer. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions. 3336.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Defferrard Michaël, Bresson Xavier, and Vandergheynst Pierre. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the NIPS.Google ScholarGoogle Scholar
  8. [8] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 41714186.Google ScholarGoogle Scholar
  9. [9] Ding Xiao, Zhang Yue, Liu Ting, and Duan Junwen. 2016. Knowledge-driven event embedding for stock prediction. In Proceedings of the Coling 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 21332142.Google ScholarGoogle Scholar
  10. [10] Gao Jun, Wang Wei, Yu Changlong, Zhao Huan, Ng Wilfred, and Xu Ruifeng. 2022. Improving event representation via simultaneous weakly supervised contrastive learning and clustering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 30363049.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Glavaš Goran and Vulić Ivan. 2020. Non-linear instance-based cross-lingual mapping for non-isomorphic embedding spaces. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 75487555.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Granroth-Wilding Mark and Clark Stephen. 2016. What happens next? Event prediction using a compositional neural network model. In Proceedings of the AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Hamilton William L., Ying Rex, and Leskovec Jure. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 10251035.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Huang Luyao, Sun Chi, Qiu Xipeng, and Huang Xuan-Jing. 2019. GlossBERT: BERT for word sense disambiguation with gloss knowledge. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 35093514.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Jans Bram, Bethard Steven, Vulić Ivan, and Moens Marie Francine. 2012. Skip n-grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. 336344.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Kingma Diederik P. and Ba Jimmy. 2015. Adam: A method for stochastic optimization. In Proceedings of the ICLR.Google ScholarGoogle Scholar
  17. [17] Kipf Thomas N. and Welling Max. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR’17).Google ScholarGoogle Scholar
  18. [18] Li Zhongyang, Ding Xiao, and Liu Ting. 2018. Constructing narrative event evolutionary graph for script event prediction. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 42014207.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Lin Yankai, Liu Zhiyuan, and Sun Maosong. 2017. Neural relation extraction with multi-lingual attention. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 3443.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Lv Shangwen, Qian Wanhui, Huang Longtao, Han Jizhong, and Hu Songlin. 2019. SAM-Net: Integrating event-level and chain-level attentions to predict what happens next. In Proceedings of the AAAI Conference on Artificial Intelligence. 68026809.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Lv Shangwen, Zhu Fuqing, and Hu Songlin. 2020. Integrating external event knowledge for script learning. In Proceedings of the 28th International Conference on Computational Linguistics. 306315.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Mao Q., Li X., Peng H., Li J., and Wang L.. 2021. Event prediction based on evolutionary event ontology knowledge. Future Generation Computer Systems 115 (2021), 76–89.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Mikolov Tomas, Sutskever Ilya, Chen Kai, Corrado Greg, and Dean Jeffrey. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems. 31113119.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Nie Allen, Bennett Erin, and Goodman Noah. 2019. DisSent: Learning sentence representations from explicit discourse relations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 44974510.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Perozzi Bryan, Al-Rfou Rami, and Skiena Steven. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701710.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Pichotta Karl and Mooney Raymond. 2014. Statistical script learning with multi-argument events. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. 220229.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Pichotta Karl and Mooney Raymond. 2016. Learning statistical scripts with LSTM recurrent neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Veličković Petar, Cucurull Guillem, Casanova Arantxa, Romero Adriana, Lio Pietro, and Bengio Yoshua. 2017. Graph attention networks[J]. stat 1050 (2017), 20.Google ScholarGoogle Scholar
  29. [29] Vial Loïc, Lecouteux Benjamin, and Schwab Didier. 2019. Sense vocabulary compression through the semantic knowledge of WordNet for neural word sense disambiguation. In Proceedings of the 10th Global Wordnet Conference. 108117.Google ScholarGoogle Scholar
  30. [30] Voita Elena, Talbot David, Moiseev Fedor, Sennrich Rico, and Titov Ivan. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. ACL Anthology, 57975808.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Wang Ziqi, Wang Xiaozhi, Han Xu, Lin Yankai, Hou Lei, Liu Zhiyuan, Li Peng, Li Juanzi, and Zhou Jie. 2021. CLEVE: Contrastive pre-training for event extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 62836297.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Wang Zhongqing, Zhang Yue, and Chang Ching Yun. 2017. Integrating order information and event relation for script event prediction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 5767.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Wu Jun, He Jingrui, and Xu Jiejun. 2019. DEMO-Net: Degree-specific graph neural networks for node and graph classification. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2019. Association for Computing Machinery, 406415.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Wu Lijun, Tian Fei, Zhao Li, Lai Jianhuang, and Liu Tie-Yan. 2018. Word attention for sequence to sequence text understanding. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Wu Xianchao, Martinez Ander, and Klyen Momo. 2018. Dialog generation using multi-turn reasoning neural networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 20492059.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Yu Changlong, Zhang Hongming, Song Yangqiu, and Ng Wilfred. 2022. CoCoLM: Complex commonsense enhanced language model with discourse relations. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2022. 11751187.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Zhang Qi, Fu Jinlan, Liu Xiaoyu, and Huang Xuanjing. 2018. Adaptive co-attention network for named entity recognition in tweets. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Zhang Rui, Santos Cicero dos, Yasunaga Michihiro, Xiang Bing, and Radev Dragomir. 2018. Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 102107.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Zhou Yucheng, Geng Xiubo, Shen Tao, Long Guodong, and Jiang Daxin. 2022. Eventbert: A pre-trained model for event correlation reasoning. In Proceedings of the ACM Web Conference 2022. 850859.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Zhou Yucheng, Geng Xiubo, Shen Tao, Pei Jian, Zhang Wenqiang, and Jiang Daxin. 2021. Modeling event-pair relations in external knowledge graphs for script reasoning. In Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 45864596.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Zhou Yucheng, Shen Tao, Geng Xiubo, Long Guodong, and Jiang Daxin. 2022. ClarET: Pre-training a correlation-aware context-to-event transformer for event-centric generation and classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 25592575.Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Zhu Haichao, Wei Furu, Qin Bing, and Liu Ting. 2018. Hierarchical attention flow for multiple-choice reading comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Script Event Prediction via Multilingual Event Graph Networks

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 2
      February 2023
      624 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3572719
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 December 2022
      • Online AM: 18 August 2022
      • Accepted: 9 August 2022
      • Revised: 8 June 2022
      • Received: 8 October 2021
      Published in tallip Volume 22, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)207
      • Downloads (Last 6 weeks)20

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!