Abstract
End-to-end neural network-based approaches have recently demonstrated significant improvements in natural language processing (NLP). However, in the NLP application such as assistant systems, NLP components are still processed to extract results using a pipeline paradigm. The pipeline-based concept has issues with error propagation. In Korean, morphological analysis and part-of-speech (POS) tagging step, incorrectly analyzing POS tags for a sentence containing spacing errors negatively affects other modules behind the POS module. Hence, we present a multi-task learning-based POS tagging neural model for Korean with word spacing challenges. When we apply this model to the Korean morphological analysis and POS tagging, we get findings that are robust to word spacing errors. We adopt syllable-level input and output formats, as well as a simple structure for ELECTRA and RNN-CRF models for multi-task learning, and we achieve a good performance 98.30 of F1, better than previous studies on the Sejong corpus test set.
- [1] . 2017. Deep Voice: Real-time neural text-to-speech. In Proceedings of the International Conference on Machine Learning. PMLR, 195–204.Google Scholar
- [2] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations (ICLR’15). May 7-9, 2015. Conference Track Proceedings, San Diego, CA.Google Scholar
- [3] Michael Braun, Anja Mainz, Ronee Chadowitz, Bastian Pfleging, and Florian Alt. 2019. At your service: Designing voice assistant personalities to improve automotive user interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI’19). Association for Computing Machinery, 1–11.Google Scholar
- [4] . 1997. Multitask learning. Mach. Learn. 28, 1 (1997), 41–75.Google Scholar
Digital Library
- [5] . 2018. Multi-task learning for sequence tagging: An empirical study. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 2965–2977.Google Scholar
- [6] . 2016. A grapheme-level approach for constructing a Korean morphological analyzer without linguistic knowledge. In Proceedings of the IEEE International Conference on Big Data (Big Data). 3872–3879.Google Scholar
Cross Ref
- [7] . 2009. Word segmentation standard in Chinese, Japanese and Korean. In Proceedings of the 7th Workshop on Asian Language Resources (ALR’09). 179–186.Google Scholar
Digital Library
- [8] . 2017. A syllable-based technique for word embeddings of Korean words. In Proceedings of the 1st Workshop on Subword and Character Level Models in NLP. Association for Computational Linguistics, 36–40.Google Scholar
Cross Ref
- [9] . 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. (2014).
arxiv:cs.NE/1412.3555 .Google Scholar - [10] . 2020. ELECTRA: Pre-training text encoders as discriminators rather than generators. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [11] . 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning (ICML’08). Association for Computing Machinery, New York, NY, 160–167.Google Scholar
Digital Library
- [12] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4171–4186.Google Scholar
- [13] . 2020. Syllable-based Korean named entity recognition and slot filling with ELECTRA. In Proceedings of the 32nd Annual Conference on Human and Cognitive Language Technology.Google Scholar
- [14] . 2016. Recurrent neural network grammars. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 199–209.Google Scholar
Cross Ref
- [15] . 2004. A morphological tagger for Korean: Statistical tagging combined with corpus-based morphological rule application. Mach. Translat. 18, 4 (2004), 275–297.Google Scholar
Digital Library
- [16] . 2019. Streaming end-to-end speech recognition for mobile devices. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 6381–6385.Google Scholar
Cross Ref
- [17] . 1997. Long short-term memory. Neural Computat. 9, 8 (1997), 1735–1780.Google Scholar
Digital Library
- [18] . 2016. Korean morphological analysis using sequence-to-sequence learning with copying mechanism. In Proceedings of the Conference on Korea Software Congress. 443–445.Google Scholar
- [19] . 2009. Cutting-plane training of structural SVMs. Mach. Learn. 77, 1 (
Oct. 2009), 27–59.Google ScholarDigital Library
- [20] . 2018. Character-level supervision for low-resource POS tagging. In Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP. 1–11.Google Scholar
Cross Ref
- [21] . 2013. Romanization-based approach to morphological analysis in Korean SMS text processing. In Proceedings of the 6th International Joint Conference on Natural Language Processing. 145–152.Google Scholar
- [22] . 2017. Adam: A Method for Stochastic Optimization. (2017).
arxiv:cs.LG/1412.6980 .Google Scholar - [23] . 2016. CharNER: Character-level named entity recognition. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, 911–921.Google Scholar
- [24] . 2013. Joint models for korean word spacing and POS tagging using structural SVM. In J. KISS: Softw. Applic. 40, 12 (2013), 826–832.Google Scholar
- [25] . 2009. Probabilistic modeling of Korean morphology. IEEE Trans. Aud., Speech, Lang. Process. 17, 5 (2009), 945–955.Google Scholar
Digital Library
- [26] . 2011. Three-step probabilistic model for Korean morphological analysis. In J. KISS: Softw. Applic. 257–268.Google Scholar
- [27] . 2021. When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute. (2021).
arxiv:cs.CL/2102.12459 .Google Scholar - [28] . 2018. Simple recurrent units for highly parallelizable recurrence. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 4470–4481.Google Scholar
Cross Ref
- [29] . 2006. Word spacing error correction for the postprocessing of speech recognition. In Proceedings of the Korean Information Science Society Conference. 25–27.Google Scholar
- [30] . 2016. An empirical study of automatic Chinese word segmentation for spoken language understanding and named entity recognition. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 238–248.Google Scholar
Cross Ref
- [31] . 2016. Multi-task sequence to sequence learning. In Proceedings of the 4th International Conference on Learning Representations.Google Scholar
- [32] . 2018. Stack-pointer networks for dependency parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1403–1414.Google Scholar
Cross Ref
- [33] . 2018. Rich character-level information for Korean morphological analysis and part-of-speech tagging. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 2482–2492.Google Scholar
- [34] . 2020. Stack pointer network for Korean morphological analysis. In Proceedings of theConference on Korea Software Congress. 371–373.Google Scholar
- [35] . 2015. Conditional random fields for Korean morpheme segmentation and POS tagging. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 14, 3 (
June 2015).Google ScholarDigital Library
- [36] . 2018. Jam-packing Korean sentence classification method robust for spacing errors. In Proceedings of the Annual Conference on Human and Language Technology. 600–604.Google Scholar
- [37] . 2020. An empirical study of tokenization strategies for various Korean NLP tasks. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, 133–142.Google Scholar
- [38] . 1995. Text chunking using transformation-based learning. In Proceedings of the 3rd Workshop on Very Large Corpora.Google Scholar
- [39] . 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017).Google Scholar
- [40] . 2019. Better morphology prediction for better speech systems. In Proceedings of the 20th Annual Conference of the International Speech Communication Association. ISCA, 3535–3539.Google Scholar
Cross Ref
- [41] . 2011. Syllable-based POS tagging without korean morphological analysis. Korean J. Cognit. Sci. 22, 3 (2011), 327–345.Google Scholar
Cross Ref
- [42] . 2019. Korean morphological analysis with tied sequence-to-sequence multi-task model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, 1436–1441.Google Scholar
Cross Ref
- [43] . 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, Vol. 27. Curran Associates, Inc.Google Scholar
Digital Library
- [44] . 2018. Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 4784–4788.Google Scholar
Digital Library
- [45] . 2020. Voice assistants and smart speakers in everyday life and in education. Inform. Educ. 19, 3 (2020), 473–490.Google Scholar
Cross Ref
- [46] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc.Google Scholar
- [47] . 2022. Edge computing driven low-light image dynamic enhancement for object detection. IEEE Trans. Netw. Sci. Eng. (2022).
DOI: Google ScholarCross Ref
- [48] . 2021. Multi-scale relation reasoning for multi-modal Visual Question Answering. Sign. Process.: Image Commun. 96 (2021), 116319.
DOI: Google ScholarCross Ref
- [49] . 2022. CE-text: A context-Aware and embedded text detector in natural scene images. Pattern Recog. Lett. 159 (2022), 77–83.
DOI: Google ScholarDigital Library
- [50] . 2015. Conditional random fields as recurrent neural networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 1529–1537.Google Scholar
Digital Library
Index Terms
Robust Multi-task Learning-based Korean POS Tagging to Overcome Word Spacing Errors
Recommendations
Korean Part-of-speech Tagging Based on Morpheme Generation
Two major problems of Korean part-of-speech (POS) tagging are that the word-spacing unit is not mapped one-to-one to a POS tag and that morphemes should be recovered during POS tagging. Therefore, this article proposes a novel two-step Korean POS tagger ...
Machine Learning-based approach to automatic POS tagging of Macedonian language
BCI '17: Proceedings of the 8th Balkan Conference in InformaticsThis paper presents the research that has contributed to the creation of an automatic part-of-speech (POS) tagger of Macedonian, a Slavic language that has a rich morphology, but limited language resources and contributions towards establishing of ...
Morphological Segmentation and Part-of-Speech Tagging for the Arabic Heritage
We annotate 60,000 words of Classical Arabic (CA) with topics in philosophy, religion, literature, and law with fine-grain segment-based morphological descriptions. We use these annotations for building a morphological segmenter and part-of-speech (POS) ...






Comments