Abstract
Punctuation prediction is critical as it can enhance the readability of machine-transcribed speeches or texts significantly by adding appropriate punctuation. Furthermore, systems like Automatic Speech Recognizer (ASR) produce texts that are unpunctuated, making the readability difficult for humans and also hampers the performance of various natural language processing (NLP) tasks. Such NLP related tasks have been investigated thoroughly for English; however, very limited work is done for punctuation prediction in the Bangla language. In this study, we train a bidirectional recurrent neural network (BRNN) along with Attention model with a plausibly large Bangla dataset. Afterwards, we apply extensive postprocessing techniques for predicting punctuation more accurately with the employed model. Initially, we perform experimentation with a relatively imbalanced dataset, and our model shows promising results F1=56.9 for Period) in punctuation prediction. Later, we also investigate the model’s performance using a balanced Bangla dataset to achieve higher performance scores (F1=62.2 for Question). Thus, the goal of this study is to propose an efficient approach that can predict punctuation in Bangla texts effectively. Our study also includes investigation on how our postprocessing techniques affect the prediction performance. Being an early attempt for the punctuation prediction in Bangla text, our work is expected to significantly contribute in the NLP field for the Bangla language, and will pave the way for future work with the Bangla language in this direction.
- [1] . 2016. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th \(\lbrace\)USENIX\(\rbrace\) Conference on Operating Systems Design and Implementation (OSDI’16). USENIX Association, 265–283.Google Scholar
- [2] . 2020. Effect of corpora on classification of fake news using naive Bayes classifier. International Journal of Automation, Artificial Intelligence and Machine Learning 1, 1 (2020), 80–92.Google Scholar
- [3] . 2020. Punctuation restoration using transformer models for resource-rich and-poor languages. In Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT’20). 132–142.Google Scholar
Cross Ref
- [4] . 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- [5] . 2016. A neural network architecture for multilingual punctuation generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing; 2016 Nov. 1–5; Austin (TX, USA).[place unknown]: ACL; 2016. p. 1048–53. ACL (Association for Computational Linguistics).Google Scholar
Cross Ref
- [6] . 1998. Cyberpunc: A lightweight punctuation annotation system for speech. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98 (Cat. No. 98CH36181), Vol. 2. IEEE, 689–692.Google Scholar
Cross Ref
- [7] . 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014).Google Scholar
- [8] . 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).Google Scholar
- [9] . 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).Google Scholar
- [10] . 2019. Using bidirectional LSTM with BERT for Chinese punctuation prediction. In 2019 IEEE International Conference on Signal, Information and Data Processing (ICSIDP’19). IEEE, 1–5.Google Scholar
Cross Ref
- [11] . 2020. Emotion recognition from Bengali speech using RNN modulation-based categorization. In 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT’20). IEEE, 1131–1136.Google Scholar
Cross Ref
- [12] . 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.Google Scholar
Digital Library
- [13] . 2021. Towards achieving a delicate blending between rule-based translator and neural machine translator. Neural Computing and Applications 33, 18 (2021), 12141–12167.Google Scholar
Digital Library
- [14] . 2016. Polygot: Going beyond database driven and syntax-based translation. In Proceedings of the 7th Annual Symposium on Computing for Development. ACM, Article
28 , 4 pages.Google ScholarDigital Library
- [15] . 2017. Polygot: An approach towards reliable translation by name identification and memory optimization using semantic analysis. In 4th International Conference on Networking, Systems and Security (NSysS’17). IEEE, 1–8.Google Scholar
Cross Ref
- [16] . 2022. Comprehensive guidelines for emotion annotation. In Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents (Faro, Portugal) (
IVA’22 ). Association for Computing Machinery, Article5 , 8 pages.DOI: Google ScholarDigital Library
- [17] . 2017. Punctuation prediction using a bidirectional recurrent neural network with part-of-speech tagging. In TENCON 2017-2017 IEEE Region 10 Conference. IEEE, 1806–1811.Google Scholar
Cross Ref
- [18] . 2019. Deep recurrent neural networks with layer-wise multi-head attentions for punctuation restoration. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’19). IEEE, 7280–7284.Google Scholar
Cross Ref
- [19] . 2020. A 43 language multilingual punctuation prediction neural network model. In INTERSPEECH. 1067–1071.Google Scholar
- [20] . 2018. Investigating for punctuation prediction in Chinese speech transcriptions. In 2018 International Conference on Asian Language Processing (IALP’18). IEEE, 74–78.Google Scholar
Cross Ref
- [21] . 2019. Transfer learning for punctuation prediction. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC’19). IEEE, 268–273.Google Scholar
Cross Ref
- [22] . 1999. Performance measures for information extraction. In Proceedings of DARPA Broadcast News Workshop. Herndon, VA, 249–252.Google Scholar
- [23] . 2010. Developing an open-source, rule-based proofreading tool. Software: Practice and Experience 40, 7 (2010), 543–566.Google Scholar
Digital Library
- [24] . 2017. A phonological phrase sequence modelling approach for resource efficient and robust real-time punctuation recovery. In INTERSPEECH. 558–562.Google Scholar
- [25] . 2021. A comprehensive guideline for Bengali sentiment annotation. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 21, 2, Article
30 (Oct. 2021), 19 pages.DOI: Google ScholarDigital Library
- [26] . 2019. Empirical evaluation and combination of punctuation prediction models applied to broadcast news. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’19). IEEE, 7275–7279.Google Scholar
Cross Ref
- [27] . 2017. Attentional parallel RNNs for generating punctuation in transcribed speech. In International Conference on Statistical Language and Speech Processing. Springer, 131–142.Google Scholar
Cross Ref
- [28] . 2019. Analysis of data pre-processing methods for sentiment analysis of reviews. Computer Science 20 (2019).Google Scholar
Cross Ref
- [29] . 2011. Modeling punctuation prediction as machine translation. In Proceedings of the 8th International Workshop on Spoken Language Translation: Papers.Google Scholar
- [30] . 2019. Punctuation prediction for Vietnamese texts using conditional random fields. In Proceedings of the Tenth International Symposium on Information and Communication Technology. 322–327.Google Scholar
Digital Library
- [31] . 2018. Beyond one-hot encoding: Lower dimensional target embedding. CoRR abs/1806.10805 (2018).
arxiv:1806.10805 . http://arxiv.org/abs/1806.10805.Google Scholar - [32] . 2017. Deep learning for punctuation restoration in medical reports. In BioNLP 2017. 159–164.Google Scholar
Cross Ref
- [33] . 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45, 11 (1997), 2673–2681.Google Scholar
Digital Library
- [34] . 2019. On extended long short-term memory and dependent bidirectional recurrent neural network. Neurocomputing 356 (2019), 151–161.Google Scholar
Digital Library
- [35] . 2019. Leveraging a character, word and prosody triplet for an ASR error robust and agglutination friendly punctuation approach. Proc. Interspeech 2019 (2019), 2988–2992.Google Scholar
Cross Ref
- [36] . 2012. Parallel data, tools and interfaces in OPUS. In Lrec, Vol. 2012. 2214–2218.Google Scholar
- [37] . 2016. Bidirectional recurrent neural network with attention mechanism for punctuation restoration. In Interspeech 2016. 3047–3051.
DOI: Google ScholarCross Ref
- [38] . 2018. Joint word-and character-level embedding CNN-RNN models for punctuation restoration. In 2018 9th IEEE International Conference on Cognitive Infocommunications (CogInfoCom’18). IEEE, 000135–000140.Google Scholar
Cross Ref
- [39] . 2018. Restoring punctuation and capitalization using transformer models. In International Conference on Statistical Language and Speech Processing. Springer, 91–102.Google Scholar
Cross Ref
- [40] . 2018. Self-attention based network for punctuation restoration. In 2018 24th International Conference on Pattern Recognition (ICPR’18). IEEE, 2803–2808.Google Scholar
Cross Ref
- [41] . 2015. Larger-context language modelling. arXiv preprint arXiv:1511.03729 (2015).Google Scholar
- [42] . 2016. Investigating LSTM for punctuation prediction. In 2016 10th International Symposium on Chinese Spoken Language Processing (ISCSLP’16). 1–5.
DOI: Google ScholarCross Ref
- [43] . 2019. Self-attention based model for punctuation prediction using word and speech embeddings. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’19). IEEE, 7270–7274.Google Scholar
Cross Ref
- [44] . 2020. Bangla Newspaper Dataset.
DOI: Google ScholarCross Ref
- [45] . 2018. Punctuation prediction model for conversational speech. arXiv preprint arXiv:1807.00543 (2018).Google Scholar
Index Terms
Punctuation Prediction in Bangla Text
Recommendations
Punctuation Prediction in Vietnamese ASRs Using Transformer-Based Models
PRICAI 2021: Trends in Artificial IntelligenceAbstractPunctuation prediction is the task of predicting and inserting punctuation like periods, commas, exclamation marks, etc. into the appropriate positions in transcribed texts in ASR systems. This helps to improve user readability and the performance ...
Stroke Segmentation and Recognition from Bangla Online Handwritten Text
ICFHR '12: Proceedings of the 2012 International Conference on Frontiers in Handwriting RecognitionThis paper deals with recognition of online handwritten Bangla (Bengali) text. Here, at first, we segment cursive words into strokes. A stroke may represent a character or a part of a character. We selected a set of Bangla words written by different ...






Comments