Abstract
Previous studies normally formulate named entity recognition (NER) as a sequence labeling task and optimize the solution in the sentence level. In this article, we propose a document-level optimization approach to NER and apply it in a domain-specific document-level NER task. As a baseline, we apply a state-of-the-art approach, i.e., long-short-term memory (LSTM), to perform word classification. On this basis, we define a global objective function with the obtained word classification results and achieve global optimization via Integer Linear Programming (ILP). Specifically, in the ILP-based approach, we propose four kinds of constraints, i.e., label transition, entity length, label consistency, and domain-specific regulation constraints, to incorporate various entity recognition knowledge in the document level. Empirical studies demonstrate the effectiveness of the proposed approach to domain-specific document-level NER.
- David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvist. Invest. 30, 1 (2007), 3--26.Google Scholar
Cross Ref
- Razvan C. Bunescu and Raymond J. Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’05). 724--731. Google Scholar
Digital Library
- Mark Dredze, Paul McNamee, Delip Rao, Adam Gerber, and Tim Finin. 2010. Entity disambiguation for knowledge base population. In Proceedings of the International Conference on Computational Linguistics (COLING’10). 277--285. Google Scholar
Digital Library
- Bogdan Babych and Anthony Hartley. 2003. Improving machine translation quality with automatic named entity recognition. In Proceedings of the 7th International European Association for Machine Translation Workshop (EAMT’03). 1--8. Google Scholar
Digital Library
- David Miller, Richard Schwartz, Ralph Weischedel, and Rebecca Stone. 1999. Named entity extraction from broadcast news. In Proceedings of the Defense Advanced Research Projects Agency Broadcast News Workshop (DARPA’99). 37--40.Google Scholar
- Guodong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tagger. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’02). 473--480. Google Scholar
Digital Library
- Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’05). 363--370. Google Scholar
Digital Library
- Ayah Zirikly and Masato Hagiwara. 2015. Cross-lingual transfer of named entity recognizers without parallel corpora. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’15). 390--396.Google Scholar
Cross Ref
- Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’04). 1--8.Google Scholar
- Ming-Wei Chang, Dan Goldwasser, Dan Roth, and Vivek Srikumar. 2010. Discriminative learning over constrained latent representations. In Proceedings of the 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’10). 429--437. Google Scholar
Digital Library
- Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2010. Global learning of focused entailment graphs. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’10). 1220--1229. Google Scholar
Digital Library
- Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 1064--1074.Google Scholar
Cross Ref
- Kuru Onur, Ozan Arkan Can, and Deniz Yuret. 2016. Charner: Character-level named entity recognition. In Proceedings of the International Conference on Computational Linguistics (COLING’16). 911--921.Google Scholar
- Dan Klein, Joseph Smarr, Huy Nguyen, and Christopher D. Manning. 2003. Named entity recognition with character-level models. In Proceedings of the 7th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’03). 180--183. Google Scholar
Digital Library
- Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2016. Multilingual language processing from bytes. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 1296--1306.Google Scholar
Cross Ref
- Xinnian Mao, Yuan Dong, Saike He, Sencheng Bao, and Haila Wang. 2008. Chinese word segmentation and named entity recognition based on conditional random fields. In Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing. 2008.Google Scholar
- Nanyun Peng and Mark Dredze. 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 548--554.Google Scholar
Cross Ref
- Nanyun Peng and Mark Dredze. 2016. Improving named entity recognition for chinese social media with word segmentation representation learning. 2016, Arxiv preprint arxiv:1603.00786.Google Scholar
- Nanyun Peng and Mark Dredze. 2017. Multi-task domain adaptation for sequence tagging. In Proceedings of the 2nd Workshop on Representation Learning for Natural Language Processing (NLP’17). 2017, 91--100.Google Scholar
Cross Ref
- Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’02). 1--8. Google Scholar
Digital Library
- Kazuhiro Yoshida and Jun'ichi Tsujii. 2007. Reranking for biomedical named-entity recognition. In Proceedings of the Conference on Biomedical Natural Language Processing (BioNLP’07). 209--216. Google Scholar
Digital Library
- Yefeng Wang. 2009. Annotating and recognizing named entities in clinical notes. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 18--26. Google Scholar
Digital Library
- Xiaohua Liu, Shaodian Zhang, Furu Wei, and Ming Zhou. 2011. Recognizing named entities in tweets. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’11). 359--367. Google Scholar
Digital Library
- Julian Brooke, Timothy Baldwin, and Adan Hammond. 2016. Bootstrapped text-level named entity recognition for literature. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’16). 344--350.Google Scholar
Cross Ref
- Honglei Guo, Huijia Zhu, Zhili Guo, Xiaoxun Zhang, Xian Wu, and Zhong Su. 2009. Domain adaptation with latent semantic association for named entity recognition. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’09). 281--289. Google Scholar
Digital Library
- David Burkett, Slav Petrov, John Blitzer, and Dan Klein. 2010. Learning better monolingual models with unanno-tated bilingual text. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’10). 46--54. Google Scholar
Digital Library
- Wangxiang Che, Mengqiu Wang, Christopher D. Manning, and Ting Liu. 2013. Named entity recognition with bilingual constraints. In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’13). 52--62.Google Scholar
- James Hammerton. 2003. Named entity recognition with long short-term memory. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’03). 172--175. Google Scholar
Digital Library
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493--2537. Google Scholar
Digital Library
- Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR, abs/1508.01991.Google Scholar
- Jason P. C. Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics (TACL’16) 4 (2016), 357--370.Google Scholar
- Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260--270.Google Scholar
Cross Ref
- Vijay Krishnan and Christopher D. Manning. 2006. An effective two-stage model for exploiting non-local dependencies in named entity recognition. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’06). 1121--1128. Google Scholar
Digital Library
- Will Radford, Xavier Carreras, and James Henderson. 2015. Named entity recognition with document-specific KB tag gazetteers. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’15). 512--517.Google Scholar
Cross Ref
- Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontonotes. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL’13). 143--152.Google Scholar
- Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. Neur. Comput. 9, 8 (1997), 1735--1780. Google Scholar
Digital Library
- Alex Graves. 2013. Generating sequences with recurrent neural networks. CoRR, abs/1308.0850, 2013.Google Scholar
- Regina Barzilay and Mirella Lapata. 2006. Aggregation via set partitioning for natural language generation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (ACL’06). 359--366. Google Scholar
Digital Library
- Quang Do, Wei Lu, and Dan Roth. 2012. Joint inference for event timeline construction. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (EMNLP-CoNLL’12). 677--687. Google Scholar
Digital Library
- Peifeng Li, Qiaoming Zhu, Hongjun Diao, and Guodong Zhou. 2012. Joint modeling of trigger identification and event type determination in Chinese event extraction. In Proceedings of the International Conference on Computational Linguistics (COLING’12). 1635--1652.Google Scholar
Index Terms
Domain-specific Named Entity Recognition with Document-Level Optimization
Recommendations
Learning multilingual named entity recognition from Wikipedia
We automatically create enormous, free and multilingual silver-standard training annotations for named entity recognition (ner) by exploiting the text and structure of Wikipedia. Most ner systems rely on statistical models of annotated data to identify ...
Two-stage approach to named entity recognition using Wikipedia and DBpedia
IMCOM '17: Proceedings of the 11th International Conference on Ubiquitous Information Management and CommunicationIn natural language understanding, extraction of named entity (NE) mentions in given text and classification of the mentions into pre-defined NE types are important processes. Most NE recognition (NER) relies on resources such as a training corpus or NE ...
Re-ranking for joint named-entity recognition and linking
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge ManagementRecognizing names and linking them to structured data is a fundamental task in text analysis. Existing approaches typically perform these two steps using a pipeline architecture: they use a Named-Entity Recognition (NER) system to find the boundaries of ...






Comments