skip to main content
short-paper

Domain-specific Named Entity Recognition with Document-Level Optimization

Authors Info & Claims
Published:27 July 2018Publication History
Skip Abstract Section

Abstract

Previous studies normally formulate named entity recognition (NER) as a sequence labeling task and optimize the solution in the sentence level. In this article, we propose a document-level optimization approach to NER and apply it in a domain-specific document-level NER task. As a baseline, we apply a state-of-the-art approach, i.e., long-short-term memory (LSTM), to perform word classification. On this basis, we define a global objective function with the obtained word classification results and achieve global optimization via Integer Linear Programming (ILP). Specifically, in the ILP-based approach, we propose four kinds of constraints, i.e., label transition, entity length, label consistency, and domain-specific regulation constraints, to incorporate various entity recognition knowledge in the document level. Empirical studies demonstrate the effectiveness of the proposed approach to domain-specific document-level NER.

References

  1. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvist. Invest. 30, 1 (2007), 3--26.Google ScholarGoogle ScholarCross RefCross Ref
  2. Razvan C. Bunescu and Raymond J. Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’05). 724--731. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Mark Dredze, Paul McNamee, Delip Rao, Adam Gerber, and Tim Finin. 2010. Entity disambiguation for knowledge base population. In Proceedings of the International Conference on Computational Linguistics (COLING’10). 277--285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bogdan Babych and Anthony Hartley. 2003. Improving machine translation quality with automatic named entity recognition. In Proceedings of the 7th International European Association for Machine Translation Workshop (EAMT’03). 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. David Miller, Richard Schwartz, Ralph Weischedel, and Rebecca Stone. 1999. Named entity extraction from broadcast news. In Proceedings of the Defense Advanced Research Projects Agency Broadcast News Workshop (DARPA’99). 37--40.Google ScholarGoogle Scholar
  6. Guodong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tagger. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’02). 473--480. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’05). 363--370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ayah Zirikly and Masato Hagiwara. 2015. Cross-lingual transfer of named entity recognizers without parallel corpora. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’15). 390--396.Google ScholarGoogle ScholarCross RefCross Ref
  9. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’04). 1--8.Google ScholarGoogle Scholar
  10. Ming-Wei Chang, Dan Goldwasser, Dan Roth, and Vivek Srikumar. 2010. Discriminative learning over constrained latent representations. In Proceedings of the 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’10). 429--437. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2010. Global learning of focused entailment graphs. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’10). 1220--1229. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 1064--1074.Google ScholarGoogle ScholarCross RefCross Ref
  13. Kuru Onur, Ozan Arkan Can, and Deniz Yuret. 2016. Charner: Character-level named entity recognition. In Proceedings of the International Conference on Computational Linguistics (COLING’16). 911--921.Google ScholarGoogle Scholar
  14. Dan Klein, Joseph Smarr, Huy Nguyen, and Christopher D. Manning. 2003. Named entity recognition with character-level models. In Proceedings of the 7th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’03). 180--183. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2016. Multilingual language processing from bytes. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 1296--1306.Google ScholarGoogle ScholarCross RefCross Ref
  16. Xinnian Mao, Yuan Dong, Saike He, Sencheng Bao, and Haila Wang. 2008. Chinese word segmentation and named entity recognition based on conditional random fields. In Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing. 2008.Google ScholarGoogle Scholar
  17. Nanyun Peng and Mark Dredze. 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 548--554.Google ScholarGoogle ScholarCross RefCross Ref
  18. Nanyun Peng and Mark Dredze. 2016. Improving named entity recognition for chinese social media with word segmentation representation learning. 2016, Arxiv preprint arxiv:1603.00786.Google ScholarGoogle Scholar
  19. Nanyun Peng and Mark Dredze. 2017. Multi-task domain adaptation for sequence tagging. In Proceedings of the 2nd Workshop on Representation Learning for Natural Language Processing (NLP’17). 2017, 91--100.Google ScholarGoogle ScholarCross RefCross Ref
  20. Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’02). 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Kazuhiro Yoshida and Jun'ichi Tsujii. 2007. Reranking for biomedical named-entity recognition. In Proceedings of the Conference on Biomedical Natural Language Processing (BioNLP’07). 209--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Yefeng Wang. 2009. Annotating and recognizing named entities in clinical notes. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 18--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Xiaohua Liu, Shaodian Zhang, Furu Wei, and Ming Zhou. 2011. Recognizing named entities in tweets. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’11). 359--367. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Julian Brooke, Timothy Baldwin, and Adan Hammond. 2016. Bootstrapped text-level named entity recognition for literature. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’16). 344--350.Google ScholarGoogle ScholarCross RefCross Ref
  25. Honglei Guo, Huijia Zhu, Zhili Guo, Xiaoxun Zhang, Xian Wu, and Zhong Su. 2009. Domain adaptation with latent semantic association for named entity recognition. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’09). 281--289. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. David Burkett, Slav Petrov, John Blitzer, and Dan Klein. 2010. Learning better monolingual models with unanno-tated bilingual text. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’10). 46--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Wangxiang Che, Mengqiu Wang, Christopher D. Manning, and Ting Liu. 2013. Named entity recognition with bilingual constraints. In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’13). 52--62.Google ScholarGoogle Scholar
  28. James Hammerton. 2003. Named entity recognition with long short-term memory. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’03). 172--175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493--2537. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR, abs/1508.01991.Google ScholarGoogle Scholar
  31. Jason P. C. Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics (TACL’16) 4 (2016), 357--370.Google ScholarGoogle Scholar
  32. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260--270.Google ScholarGoogle ScholarCross RefCross Ref
  33. Vijay Krishnan and Christopher D. Manning. 2006. An effective two-stage model for exploiting non-local dependencies in named entity recognition. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’06). 1121--1128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Will Radford, Xavier Carreras, and James Henderson. 2015. Named entity recognition with document-specific KB tag gazetteers. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’15). 512--517.Google ScholarGoogle ScholarCross RefCross Ref
  35. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontonotes. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’13). 143--152.Google ScholarGoogle Scholar
  36. Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. Neur. Comput. 9, 8 (1997), 1735--1780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Alex Graves. 2013. Generating sequences with recurrent neural networks. CoRR, abs/1308.0850, 2013.Google ScholarGoogle Scholar
  38. Regina Barzilay and Mirella Lapata. 2006. Aggregation via set partitioning for natural language generation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (ACL’06). 359--366. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Quang Do, Wei Lu, and Dan Roth. 2012. Joint inference for event timeline construction. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (EMNLP-CoNLL’12). 677--687. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Peifeng Li, Qiaoming Zhu, Hongjun Diao, and Guodong Zhou. 2012. Joint modeling of trigger identification and event type determination in Chinese event extraction. In Proceedings of the International Conference on Computational Linguistics (COLING’12). 1635--1652.Google ScholarGoogle Scholar

Index Terms

  1. Domain-specific Named Entity Recognition with Document-Level Optimization

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Asian and Low-Resource Language Information Processing
        ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 17, Issue 4
        December 2018
        193 pages
        ISSN:2375-4699
        EISSN:2375-4702
        DOI:10.1145/3229525
        Issue’s Table of Contents

        Copyright © 2018 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 27 July 2018
        • Accepted: 1 May 2018
        • Revised: 1 December 2017
        • Received: 1 April 2017
        Published in tallip Volume 17, Issue 4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • short-paper
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!