research-article

Extracting Knowledge from Web Text with Monte Carlo Tree Search

Authors:

Ping LiAuthors Info & Claims

WWW '20: Proceedings of The Web Conference 2020

Pages 2585 - 2591

https://doi.org/10.1145/3366423.3380010

Published: 20 April 2020 Publication History

Abstract

To extract knowledge from general web text, it requires to build a domain-independent extractor that scales to the entire web corpus. This task is known as Open Information Extraction (OIE). This paper proposes to apply Monte-Carlo Tree Search (MCTS) to accomplish OIE. To achieve this goal, we define a Markov Decision Process for OIE and build a simulator to learn the reward signals, which provides a complete reinforcement learning framework for MCTS. Using this framework, MCTS explores candidate words (and symbols) under the guidance of a pre-trained Sequence-to-Sequence (Seq2Seq) predictor and generates abundant exploration samples during training. We apply the exploration samples to update the reward simulator and the predictor, based on which we implement another MCTS to search the optimal predictions during inference. Empirical evaluation demonstrates that the MCTS inference substantially improves the accuracy of prediction (more than 10%) and achieves a leading performance over other state-of-the-art comparison models.

References

[1]

Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2017. An Actor-Critic Algorithm for Sequence Prediction. In Proceedings of the 5th International Conference on Learning Representations (ICLR). Toulon, France.

[2]

Michele Banko, Michael J. Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. 2007. Open Information Extraction from the Web. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI). Hyderabad, India, 2670–2676.

Digital Library

[3]

Guillaume Chaslot, Mark H. M. Winands, and H. Jaap van den Herik. [n.d.]. Parallel Monte-Carlo Tree Search. In Proceedings of 6th International Conference on Computers and Games (CG).

[4]

Janara Christensen, Mausam, Stephen Soderland, and Oren Etzioni. 2011. An Analysis of Open Information Extraction Based on Semantic Role Labeling. In Proceedings of the 6th International Conference on Knowledge Capture (K-CAP). Banff, Alberta, Canada, 113–120.

Digital Library

[5]

Rémi Coulom. 2006. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In Proceedings of the 5th International Conference on Computers and Games (CG). Turin, Italy, 72–83.

Digital Library

[6]

Filipe de Sá Mesquita, Jordan Schmidek, and Denilson Barbosa. 2013. Effectiveness and Efficiency of Open Relation Extraction. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). Seattle, WA, 447–457.

[7]

Oren Etzioni, Anthony Fader, Janara Christensen, Stephen Soderland, and Mausam. 2011. Open Information Extraction: The Second Generation. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI). Barcelona, Spain, 3–10.

[8]

Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2014. Open Question Answering over Curated and Extracted Knowledge Bases. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). New York, NY, 1156–1165.

Digital Library

[9]

Kayvon Fatahalian, Jeremy Sugerman, and Pat Hanrahan. 2004. Understanding the efficiency of GPU algorithms for matrix-matrix multiplication. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Symposium on Graphics Hardware 2004. Grenoble, France, 133–137.

Digital Library

[10]

Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. Incorporating Copying Mechanism in Sequence-to-Sequence Learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany.

[11]

Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. 2019. Knowledge Graph Embedding Based Question Answering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (WSDM). Melbourne, Australia, 105–113.

Digital Library

[12]

Shengbin Jia, Shijia E, Maozhen Li, and Yang Xiang. 2018. Chinese Open Relation Extraction and Knowledge Base Establishment. ACM Trans. Asian & Low-Resource Lang. Inf. Process. 17, 3(2018), 15:1–15:22.

[13]

Dingcheng Li, Siamak Zamani, Jingyuan Zhang, and Ping Li. 2019. Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NNAACL-HLT). Minneapolis, MN, 940–950.

[14]

Xu Li, Mingming Sun, and Ping Li. 2019. Multi-Agent Discussion Mechanism for Natural Language Generation. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI). Honolulu, Hawaii, 6096–6103.

[15]

Guiliang Liu, Xu Li, Miningming Sun, and Ping Li. 2020. An Advantage Actor-Critic Algorithm with Confidence Exploration for Open Information Extraction. In Proceedings of the 2020 SIAM International Conference on Data Mining (SDM). Cincinnati, Ohio.

[16]

Mausam. 2016. Open Information Extraction Systems and Downstream Applications. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI). New York, NY, 4074–4077.

[17]

Makoto Miwa and Mohit Bansal. 2016. End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany.

[18]

Christina Niklaus, Matthias Cetto, André Freitas, and Siegfried Handschuh. 2018. A Survey on Open Information Extraction. In Proceedings of the 27th International Conference on Computational Linguistics (COLING). Santa Fe, New Mexico, 3866–3878.

[19]

Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence Level Training with Recurrent Neural Networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR). San Juan, Puerto Rico.

[20]

John W Ratcliff and David E Metzener. 1988. Pattern Matching the Gestalt Approach. Dr Dobbs Journal 13, 7 (1988), 46.

[21]

Christopher D. Rosin. 2010. Multi-armed bandits with episode context. In International Symposium on Artificial Intelligence and Mathematics (ISAIM). Fort Lauderdale, FL.

[22]

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016), 484–489.

[23]

David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the game of Go without human knowledge. Nature 550, 7676 (2017), 354.

[24]

Gabriel Stanovsky and Ido Dagan. 2016. Creating a Large Benchmark for Open Information Extraction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). Austin, TX, 2300–2305.

[25]

Mingming Sun, Xu Li, and Ping Li. 2018. Logician and Orator: Learning from the Duality between Language and Knowledge in Open Domain. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). Brussels, Belgium, 2119–2130.

[26]

Mingming Sun, Xu Li, Xin Wang, Miao Fan, Yue Feng, and Ping Li. 2018. Logician: A Unified End-to-End Neural Approach for Open-Domain Information Extraction. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM). Marina Del Rey, CA, 556–564.

Digital Library

[27]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems (NIPS). Montreal, Canada, 3104–3112.

[28]

Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.

Digital Library

[29]

Yuen-Hsien Tseng, Lung-Hao Lee, Shu-Yen Lin, Bo-Shun Liao, Mei-Jun Liu, Hsin-Hsi Chen, Oren Etzioni, and Anthony Fader. 2014. Chinese Open Relation Extraction for Knowledge Acquisition. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Gothenburg, Sweden, 12–16.

[30]

Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling Coverage for Neural Machine Translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany.

[31]

Mohamed Yahya, Steven Whang, Rahul Gupta, and Alon Y. Halevy. 2014. ReNoun: Fact Extraction for Nominal Attributes. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, 325–335.

[32]

Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2002. Kernel Methods for Relation Extraction. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, (EMNLP). Philadelphia, Pennsylvania.

Digital Library

Cited By

Liu JChen LJiang SWang CZhang SLiang JXiao YSong R(2024)A Crossword Solving System Based on Monte Carlo Tree SearchArtificial Intelligence10.1016/j.artint.2024.104192(104192)Online publication date: Jul-2024
https://doi.org/10.1016/j.artint.2024.104192
Xie HCheng LDeng JCheng CWang T(2023)Distant supervision relation extraction of combination bag with hierarchical attentionProceedings of the 2023 15th International Conference on Machine Learning and Computing10.1145/3587716.3587726(61-67)Online publication date: 17-Feb-2023
https://dl.acm.org/doi/10.1145/3587716.3587726
Maddigan PSusnjak T(2023)Chat2VIS: Generating Data Visualizations via Natural Language Using ChatGPT, Codex and GPT-3 Large Language ModelsIEEE Access10.1109/ACCESS.2023.327419911(45181-45193)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3274199
Show More Cited By

Index Terms

Extracting Knowledge from Web Text with Monte Carlo Tree Search

Index terms have been assigned to the content through auto-classification.

Recommendations

LinUCB applied to Monte Carlo tree search

UCT is a standard method of Monte Carlo tree search (MCTS) algorithms, which have been applied to various domains and have achieved remarkable success. This study proposes a family of LinUCT algorithms that incorporate LinUCB into MCTS algorithms. ...
Monte-Carlo tree search for Bayesian reinforcement learning

Bayesian model-based reinforcement learning can be formulated as a partially observable Markov decision process (POMDP) to provide a principled framework for optimally balancing exploitation and exploration. Then, a POMDP solver can be used to solve the ...
Monte-carlo tree search for constrained POMDPs
NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems

Monte-Carlo Tree Search (MCTS) has been successfully applied to very large POMDPs, a standard model for stochastic sequential decision-making problems. However, many real-world problems inherently have multiple goals, where multi-objective formulations ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Proceedings of The Web Conference 2020

April 2020

3143 pages

ISBN:9781450370233

DOI:10.1145/3366423

Editors:
Yennun Huang
Acadmica sinica, Taiwan
,
Irwin King
The Chinese University of Hong Kong, Hong Kong
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
522
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)6

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu JChen LJiang SWang CZhang SLiang JXiao YSong R(2024)A Crossword Solving System Based on Monte Carlo Tree SearchArtificial Intelligence10.1016/j.artint.2024.104192(104192)Online publication date: Jul-2024
https://doi.org/10.1016/j.artint.2024.104192
Xie HCheng LDeng JCheng CWang T(2023)Distant supervision relation extraction of combination bag with hierarchical attentionProceedings of the 2023 15th International Conference on Machine Learning and Computing10.1145/3587716.3587726(61-67)Online publication date: 17-Feb-2023
https://dl.acm.org/doi/10.1145/3587716.3587726
Maddigan PSusnjak T(2023)Chat2VIS: Generating Data Visualizations via Natural Language Using ChatGPT, Codex and GPT-3 Large Language ModelsIEEE Access10.1109/ACCESS.2023.327419911(45181-45193)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3274199
Yu JCai YSun MLi PBellogín ABoratto LCena F(2022)SpaceE: Knowledge Graph Embedding by Relational Linear Transformation in the Entity SpaceProceedings of the 33rd ACM Conference on Hypertext and Social Media10.1145/3511095.3531284(64-72)Online publication date: 28-Jun-2022
https://dl.acm.org/doi/10.1145/3511095.3531284
Zhang YFei HLi PAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)End-to-end Distantly Supervised Information Extraction with Retrieval AugmentationProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531876(2449-2455)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531876
Smith EPapadopoulos DBraschler MStockinger K(2022)LILLIEInformation Systems10.1016/j.is.2021.101938105:COnline publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.is.2021.101938
Hao XZhou YWu DZhang WLi BWang WCheng WKankanhalli MWang MChu WLiu JWorring M(2021)Multi-Feature Graph Attention Network for Cross-Modal Video-Text RetrievalProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463608(135-143)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463608
Yu BZhang ZSheng JLiu TWang YWang YWang B(2021)Semi-Open Information ExtractionProceedings of the Web Conference 202110.1145/3442381.3450029(1661-1672)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3450029
Heist NPaulheim H(2021)Information Extraction From Co-Occurring Similar EntitiesProceedings of the Web Conference 202110.1145/3442381.3449836(3999-4009)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449836
Zhang YFei HLi PDiaz FShah CSuel TCastells PJones RSakai T(2021)ReadsRE: Retrieval-Augmented Distantly Supervised Relation ExtractionProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3463103(2257-2262)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3463103

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents