Abstract
Singlish can be interesting to the computational linguistics community both linguistically, as a major low-resource creole based on English, and computationally, for information extraction and sentiment analysis of regional social media. In our conference paper, Wang et al. (2017), we investigated part-of-speech (POS) tagging and dependency parsing for Singlish by constructing a treebank under the Universal Dependencies scheme and successfully used neural stacking models to integrate English syntactic knowledge for boosting Singlish POS tagging and dependency parsing, achieving the state-of-the-art accuracies of 89.50% and 84.47% for Singlish POS tagging and dependency, respectively. In this work, we substantially extend Wang et al. (2017) by enlarging the Singlish treebank to more than triple the size and with much more diversity in topics, as well as further exploring neural multi-task models for integrating English syntactic knowledge. Results show that the enlarged treebank has achieved significant relative error reduction of 45.8% and 15.5% on the base model, 27% and 10% on the neural multi-task model, and 21% and 15% on the neural stacking model for POS tagging and dependency parsing, respectively. Moreover, the state-of-the-art Singlish POS tagging and dependency parsing accuracies have been improved to 91.16% and 85.57%, respectively. We make our treebanks and models available for further research.
- Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah Smith. 2016. Many languages, one parser. Trans. Assoc. Comput. Ling. 4 (2016), 431--444.Google Scholar
Cross Ref
- Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. Globally normalized transition-based neural networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’16). Association for Computational Linguistics, 2442--2452.Google Scholar
Cross Ref
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint abs/1409.0473 (2014). http://arxiv.org/abs/1409.0473.Google Scholar
- Miguel Ballesteros, Chris Dyer, and A. Noah Smith. 2015. Improved transition-based parsing by modeling characters instead of words with LSTMs. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). Association for Computational Linguistics, 349--359.Google Scholar
- Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012. English Web Treebank LDC2012T13. (2012).Google Scholar
- Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). Association for Computational Linguistics, 740--750.Google Scholar
Cross Ref
- Hongshen Chen, Yue Zhang, and Qun Liu. 2016. Neural network for heterogeneous annotations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’16). Association for Computational Linguistics, 731--741.Google Scholar
Cross Ref
- Shay Cohen and A. Noah Smith. 2009. Shared logistic normal distributions for soft parameter tying in unsupervised grammar induction. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’09). Association for Computational Linguistics, 74--82. Google Scholar
Digital Library
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 (2011), 2493--2537. Google Scholar
Digital Library
- Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In International Conference on Learning Representations 2017. arXiv preprint abs/1611.01734. http://arxiv.org/abs/1611.01734.Google Scholar
- John C. Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12 (2011), 2121--2159. Google Scholar
Digital Library
- Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. 2015. A neural network model for low-resource universal dependency parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). Association for Computational Linguistics, 339--348.Google Scholar
Cross Ref
- Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and A. Noah Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). Association for Computational Linguistics, 334--343.Google Scholar
- Kuzman Ganchev, Jennifer Gillenwater, and Ben Taskar. 2009. Dependency grammar induction via bitext projection constraints. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). Association for Computational Linguistics, 369--377. Google Scholar
Digital Library
- Douwe Gelling, Trevor Cohn, Phil Blunsom, and Joao Graca. 2012. The PASCAL challenge on grammar induction. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Workshop on the Induction of Linguistic Structure (NAACL-HLT’12). Association for Computational Linguistics, 64--80. Google Scholar
Digital Library
- Jennifer Gillenwater, Kuzman Ganchev, João Graça, Fernando Pereira, and Ben Taskar. 2010. Sparsity in dependency grammar induction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’10) (Short Papers). Association for Computational Linguistics, 194--199. http://www.aclweb.org/anthology/P10-2036. Google Scholar
Digital Library
- Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neur. Netw. 18, 5 (2005), 602--610. Google Scholar
Digital Library
- Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, and Jürgen Schmidhuber. 2015. LSTM: A search space odyssey. CoRR abs/1503.04069 (2015). http://arxiv.org/abs/1503.04069.Google Scholar
- Jiang Guo, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2015. Cross-lingual dependency parsing based on distributed representations. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). Association for Computational Linguistics, 1234--1244.Google Scholar
Cross Ref
- Shinichi Harada. 2009. The roles of singapore standard english and singlish. Inf. Res. 40 (2009), 70--82.Google Scholar
- Kenneth Heafield, Ivan Pouzyrevsky, H. Jonathan Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’13). Association for Computational Linguistics, 690--696. http://aclweb.org/anthology/P13-2121.Google Scholar
- Sanjika Hewavitharana, Nguyen Bach, Qin Gao, Vamshi Ambati, and Stephan Vogel. 2011. CMU Haitian Creole-English translation system for WMT. In Proceedings of the 6th Workshop on Statistical Machine Translation. Association for Computational Linguistics, 386--392. Google Scholar
Digital Library
- Chang Hu, Philip Resnik, Yakov Kronrod, Vladimir Eidelman, Olivia Buzek, and B. Benjamin Bederson. 2011. The value of monolingual crowdsourcing in a real-world translation scenario: Simulation using Haitian Creole emergency SMS messages. In Proceedings of the 6th Workshop on Statistical Machine Translation. Association for Computational Linguistics, 399--404. Google Scholar
Digital Library
- Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint abs/1508.01991 (2015). http://arxiv.org/abs/1508.01991.Google Scholar
- Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Nat. Lang. Eng. 11, 3 (Sep. 2005), 311--325. Google Scholar
Digital Library
- Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. Trans. Assoc. Comput. Ling. 4 (2016), 313--327.Google Scholar
Cross Ref
- Karen Lahousse and Béatrice Lamiroy. 2012. Word order in French, Spanish and Italian: A grammaticalization account. Folia Ling. 46, 2 (2012), 387--415.Google Scholar
- Jakob R. E. Leimgruber. 2009. Modelling Variation in Singapore English. Ph.D. Dissertation. Oxford University.Google Scholar
- Jakob R. E. Leimgruber. 2011. Singapore English. Lang. Ling. Compass 5, 1 (2011), 47--62.Google Scholar
Cross Ref
- Jiwei Li, Thang Luong, Dan Jurafsky, and Eduard Hovy. 2015. When are tree structures necessary for deep learning of representations? In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). Association for Computational Linguistics, 2304--2314.Google Scholar
Cross Ref
- Lisa Lim. 2007. Mergers and acquisitions: On the ages and origins of Singapore English particles. World Engl. 26, 4 (2007), 446--473.Google Scholar
Cross Ref
- Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A. Smith. 2018. Parsing tweets into universal dependencies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18). 965--975. https://aclanthology.info/papers/N18-1088/n18-1088Google Scholar
- Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’14). 55--60.Google Scholar
Cross Ref
- Héctor Martínez Alonso, Željko Agić, Barbara Plank, and Anders Søgaard. 2017. Parsing universal dependencies without training. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL’17). Association for Computational Linguistics, 230--240. http://www.aclweb.org/anthology/E17-1022.Google Scholar
- Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’11). Association for Computational Linguistics, 62--72. Google Scholar
Digital Library
- Ho Mian-Lian and John T. Platt. 1993. Dynamics of a Contact Continuum: Singaporean English. Oxford University Press, New York.Google Scholar
- Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’16). Association for Computational Linguistics, 1105--1116.Google Scholar
Cross Ref
- Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. Selective sharing for multilingual dependency parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’12). Association for Computational Linguistics, 629--637. http://aclweb.org/anthology/P12-1066 Google Scholar
Digital Library
- Tahira Naseem, Harr Chen, Regina Barzilay, and Mark Johnson. 2010. Using universal linguistic knowledge to guide grammar induction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’10). Association for Computational Linguistics, Cambridge, MA, 1234--1244. http://www.aclweb.org/anthology/D10-1120. Google Scholar
Digital Library
- Paroo Nihilani. 1992. The international computerized corpus of English. Words in a Cultural Context. Singapore: UniPress (1992), 84--88.Google Scholar
- Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’16). European Language Resources Association, 23--28.Google Scholar
- Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session of Conference on Empirical Methods in Natural Language Processing (EMNLP-CoNLL’07). Association for Computational Linguistics, 915--932.Google Scholar
- Brendan T. O’Connor, Su Lin Blodgett, and Johnny Wei. 2018. Twitter universal dependency parsing for African-American and mainstream American english. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL’18). 1415--1425.Google Scholar
- Vincent B. Y. Ooi. 1997. Analysing the Singapore ICE Corpus for Lexicographic Evidence.Google Scholar
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). Association for Computational Linguistics, 1532--1543.Google Scholar
Cross Ref
- Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’16). Association for Computational Linguistics, 412--418.Google Scholar
Cross Ref
- Manuela Sanguinetti, Cristina Bosco, Alberto Lavelli, Alessandro Mazzei, Oronzo Antonelli, and Fabio Tamburini. 2018. PoSTWITA-UD: An Italian Twitter treebank in universal dependencies. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC’18).Google Scholar
- Manuela Sanguinetti, Cristina Bosco, Alessandro Mazzei, Alberto Lavelli, and Fabio Tamburini. 2017. Annotating Italian social media texts in universal dependencies. In Proceedings of the 4th International Conference on Dependency Linguistics. 229--239. https://aclanthology.info/papers/W17-6526/w17-6526Google Scholar
- Chun-Wei Seah, Hai Leong Chieu, Kian Ming Adam Chai, Loo-Nin Teow, and Lee Wei Yeong. 2015. Troll detection by domain-adapting sentiment analysis. In Proceedings of the 18th International Conference on Information Fusion (Fusion’15). IEEE, 792--799.Google Scholar
- Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, D. Christopher Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’13). Association for Computational Linguistics, 1631--1642.Google Scholar
- Anders Søgaard. 2012a. Two baselines for unsupervised dependency parsing. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Workshop on the Induction of Linguistic Structure (NAACL-HLT’12). Association for Computational Linguistics, 81--83. Google Scholar
Digital Library
- Anders Søgaard. 2012b. Unsupervised dependency parsing without training. Nat. Lang. Eng. 18, 2 (2012), 187--203. Google Scholar
Digital Library
- Oscar Täckström, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012). Association for Computational Linguistics, 477--487. Google Scholar
Digital Library
- Hongmin Wang, Yue Zhang, GuangYong Leonard Chan, Jie Yang, and Hai Leong Chieu. 2017. Universal dependencies parsing for colloquial Singaporean english. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1732--1744. http://aclweb.org/anthology/P17-1159.Google Scholar
Cross Ref
- David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured training for neural network transition-based parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). Association for Computational Linguistics, 323--333.Google Scholar
Cross Ref
- Yuan Zhang and Regina Barzilay. 2015. Hierarchical low-rank tensors for multilingual transfer parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15). Association for Computational Linguistics, 1857--1867.Google Scholar
Cross Ref
- Yuan Zhang and David Weiss. 2016. Stack-propagation: Improved representation learning for syntax. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). Association for Computational Linguistics, 1557--1566.Google Scholar
Cross Ref
- Hao Zhou, Yue Zhang, Shujian Huang, and Jiajun Chen. 2015. A neural probabilistic structured-prediction model for transition-based dependency parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). Association for Computational Linguistics, 1213--1222.Google Scholar
Cross Ref
Index Terms
From Genesis to Creole Language: Transfer Learning for Singlish Universal Dependencies Parsing and POS Tagging
Recommendations
Singlish Checker: A Tool for Understanding and Analysing an English Creole Language
From Born-Physical to Born-Virtual: Augmenting Intelligence in Digital LibrariesAbstractAs English is a widely used language in many countries of different cultures, variants of English also known as English creoles have also been created. Singlish is one such English creole used by people in Singapore. Nevertheless, unlike English, ...
A Cross-lingual Part-of-Speech Tagging for Malay Language
ICAART 2015: Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 2Cross-lingual annotation projection methods can benefit from rich-resourced languages to improve the performance
of Natural Language Processing (NLP) tasks in less-resourced languages. In this research, Malay
is experimented as the less-resourced ...
How Important Is POS to Dependency Parsing? Joint POS Tagging and Dependency Parsing Neural Networks
Chinese Computational LinguisticsAbstractIt is widely accepted that part-of-speech (POS) tagging and dependency parsing are highly related. Most state-of-the-art dependency parsing methods still rely on the results of POS tagging, though the tagger is not perfect yet. Inevitably, ...






Comments