Abstract
We present a novel Indic handwritten word recognition scheme by fusion of spatio-temporal information extracted from handwritten images. The main challenge in Indic word recognition lies in its complexity because of modifiers, touching characters, and compound characters. Hidden Markov Models (HMMs) are being used to model such data due to their ability to learn sequential data, however, the recognition performance is not satisfactory. We propose here a Long Short-Term Memory (LSTM)-based architecture for offline Indic word recognition. Offline recognition methods usually involve spatial data, whereas it has been observed that online recognition schemes show better performance than the offline methodologies. Online information usually refers to the temporal information obtained from the strokes of the pen tip while writing, which is missing in offline word images. In this article, an effort has been made to extract the online temporal information from offline images using stroke recovery and later it is combined with spatial information in LSTM architecture. During recognition, the character models are trained using both offline and extracted pseudo-online handwritten data separately. Finally, a novel fusion scheme has been used to combine them together. From the experiment, it is noted that recognition performance of handwritten Indic words improves considerably due to the fusion scheme of spatial and temporal data.
- Mohamed Nidhal Abdi and Maher Khemakhem. 2015. A model-based approach to offline text-independent Arabic writer identification and verification. Pattern Recog. 48, 5 (2015), 1890--1903.Google Scholar
Digital Library
- Chandranath Adak, Bidyut B. Chaudhuri, and Michael Blumenstein. 2016. Offline cursive Bengali word recognition using CNNs with a recurrent model. In Proceedings of the 15th International Conference on Frontiers in Handwriting Recognition (ICFHR’16). IEEE, 429--434.Google Scholar
Cross Ref
- Irfan Ahmad and Sabri A. Mahmoud. 2012. Arabic bank check analysis and zone extraction. In Proceedings of the International Conference on Image Analysis and Recognition. Springer, 141--148.Google Scholar
- Jawad H. AlKhateeb, Jinchang Ren, Jianmin Jiang, and Husni Al-Muhtaseb. 2011. Offline handwritten Arabic cursive text recognition using hidden Markov models and re-ranking. Pattern Recog. Lett. 32, 8 (2011), 1081--1088.Google Scholar
Digital Library
- Sandhya Arora, Debotosh Bhattacharjee, Mita Nasipuri, Dipak Kumar Basu, and Mahantapas Kundu. 2010. Multiple classifier combination for off-line handwritten Devnagari character recognition. Retrieved from Arxiv Preprint Arxiv:1006.5913 (2010).Google Scholar
- Ahmad-Montaser Awal, Harold Mouchère, and Christian Viard-Gaudin. 2014. A global learning approach for an online handwritten mathematical expression recognition system. Pattern Recog. Lett. 35 (2014), 68--77.Google Scholar
Digital Library
- A. Bharath and Sriganesh Madhvanath. 2012. HMM-based lexicon-driven and lexicon-free word recognition for online handwritten Indic scripts. IEEE Trans. Pattern Anal. Mach. Intell. 34, 4 (2012), 670--682.Google Scholar
Digital Library
- Nilanjana Bhattacharya and Umapada Pal. 2012. Stroke segmentation and recognition from bangla online handwritten text. In Proceedings of the International Conference on Frontiers in Handwriting Recognition. 740--745.Google Scholar
Digital Library
- Fadi Biadsy, Jihad El-Sana, and Nizar Habash. 2006. Online Arabic handwriting recognition using hidden Markov models. In Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition. Suvisoft.Google Scholar
- Anne-Laure Bianne-Bernard, Farès Menasri, Ramy Al-Hajj Mohamad, Chafic Mokbel, Christopher Kermorvant, and Laurence Likforman-Sulem. 2011. Dynamic and contextual information in HMM modeling for handwritten word recognition. IEEE Trans. Pattern Anal. Mach. Intell. 33, 10 (2011), 2066--2080.Google Scholar
Digital Library
- Horst Bunke and Tamás Varga. 2007. Off-line Roman cursive handwriting recognition. In Digital Document Processing. Springer, 165--183.Google Scholar
- Youssouf Chherawala, Partha Pratim Roy, and Mohamed Cheriet. 2016. Feature set evaluation for offline handwriting recognition systems: Application to the recurrent neural network model. IEEE Trans. Cyber. 46, 12 (2016), 2825--2836.Google Scholar
Cross Ref
- Youssouf Chherawala, Partha Pratim Roy, and Mohamed Cheriet. 2017. Combination of context-dependent bidirectional long short-term memory classifiers for robust offline handwriting recognition. Pattern Recog. Lett. 90 (2017), 58--64.Google Scholar
Digital Library
- P. Doetsch, M. Kozielski, and H. Ney. 2014. Fast and robust training of recurrent neural networks for offline handwriting recognition. In Proceedings of the 14th International Conference on Frontiers in Handwriting Recognition. 279--284.Google Scholar
- Kartik Dutta, Praveen Krishnan, Minesh Mathew, and C. V. Jawahar. 2017. Towards accurate handwritten word recognition for Hindi and Bangla. In Proceedings of the National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics. Springer, 470--480.Google Scholar
- S. Espana-Boquera, M. J. Castro-Bleda, J. Gorbe-Moya, and F. Zamora-Martinez. 2011. Improving offline handwritten text recognition with hybrid HMM/ANN models. IEEE Trans. Pattern Anal. Mach. Intell. 33, 4 (2011), 767--779.Google Scholar
Digital Library
- Vahid Ghods, Ehsanollah Kabir, and Farbod Razzazi. 2013. Decision fusion of horizontal and vertical trajectories for recognition of online Farsi subwords. Eng. Appl. Artific. Intell. 26, 1 (2013), 544--550.Google Scholar
Digital Library
- Rajib Ghosh, Pradeep Kumar, and Partha Pratim Roy. 2019. A Dempster-Shafer theory based classifier combination for online signature recognition and verification systems. Int. J. Mach. Learn. Cyber. 10, 9 (2019), 2467--2482.Google Scholar
Cross Ref
- Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd International Conference on Machine Learning. ACM, 369--376.Google Scholar
Digital Library
- Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. 2001. Gradient flow in recurrent nets: The difficulty of learning long-term dependencies.Google Scholar
- Yoshiharu Kato and Makoto Yasuhara. 2000. Recovery of drawing order from single-stroke handwriting images. IEEE Trans. Pattern Anal. Mach. Intell. 22, 9 (2000), 938--949.Google Scholar
Digital Library
- JinHyung Kim and Bong-Kee Sin. 2014. Online handwriting recognition. In Handbook of Document Image Processing and Recognition. Springer, 887--915.Google Scholar
- Pradeep Kumar, Himaanshu Gauba, Partha Pratim Roy, and Debi Prosad Dogra. 2017a. Coupled HMM-based multi-sensor data fusion for sign language recognition. Pattern Recog. Lett. 86 (2017), 1--8.Google Scholar
Digital Library
- Pradeep Kumar, Himaanshu Gauba, Partha Pratim Roy, and Debi Prosad Dogra. 2017b. A multimodal framework for sensor based sign language recognition. Neurocomputing 259 (2017), 21--38.Google Scholar
Cross Ref
- Pradeep Kumar, Rajkumar Saini, Partha Pratim Roy, and Debi Prosad Dogra. 2016. Study of text segmentation and recognition using leap motion sensor. IEEE Sens. J. 17, 5 (2016), 1293--1301.Google Scholar
Cross Ref
- Pradeep Kumar, Rajkumar Saini, Partha Pratim Roy, and Debi Prosad Dogra. 2017c. 3D text segmentation and recognition using leap motion. Multimedia Tools Appl. 76, 15 (2017), 16491--16510.Google Scholar
Digital Library
- Pradeep Kumar, Rajkumar Saini, Partha Pratim Roy, and Debi Prosad Dogra. 2018. A position and rotation invariant framework for sign language recognition (SLR) using Kinect. Multimedia Tools Appl. 77, 7 (2018), 8823--8846.Google Scholar
Digital Library
- Longin Jan Latecki, Quan-nan Li, Xiang Bai, and Wen-yu Liu. 2007. Skeletonization using SSM of the distance transform. In Proceedings of the International Conference on Image Processing, Vol. 5. V--349.Google Scholar
Cross Ref
- Umapada Pal, Ramachandran Jayadevan, and Nabin Sharma. 2012a. Handwriting recognition in Indian regional scripts: A survey of offline techniques. ACM Trans. Asian Lang. Inform. Proc. 11, 1 (2012), 1.Google Scholar
Digital Library
- Umapada Pal, Ramachandran Jayadevan, and Nabin Sharma. 2012b. Handwriting recognition in Indian regional scripts: A survey of offline techniques. ACM Trans. Asian Lang. Inform. Proc. 11, 1 (2012), 1.Google Scholar
Digital Library
- Mohammad Tanvir Parvez and Sabri A. Mahmoud. 2013a. Arabic handwriting recognition using structural and syntactic pattern attributes. Pattern Recog. 46, 1 (2013), 141--154.Google Scholar
Digital Library
- Mohammad Tanvir Parvez and Sabri A. Mahmoud. 2013b. Offline Arabic handwritten text recognition: A survey. Comput. Surv. 45, 2 (2013), 23.Google Scholar
Digital Library
- Réjean Plamondon and Claudio M. Privitera. 1999. The segmentation of cursive handwriting: An approach based on off-line recovery of the motor-temporal information. IEEE Trans. Image Proc. 8, 1 (1999), 80--91.Google Scholar
Digital Library
- Steve Procter and John Illingworth. 1998. Combining HMM classifiers in a handwritten text recognition system. In Proceedings of the International Conference on Image Processing, Vol. 2. 934--938.Google Scholar
Cross Ref
- Yu Qiao, Jianzhuang Liu, and Xiaoou Tang. 2007. Offline signature verification using online handwriting registration. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 1--8.Google Scholar
Cross Ref
- Kaushik Roy, Szilárd Vajda, Umapada Pal, Bidyut Baran Chaudhuri, and Abdel Belaïd. 2005. A system for Indian postal automation. In Proceedings of the 8th International Conference on Document Analysis and Recognition. 1060--1064.Google Scholar
Digital Library
- Partha Pratim Roy, Ayan Kumar Bhunia, Ayan Das, Prasenjit Dey, and Umapada Pal. 2016. HMM-based Indic handwritten word recognition using zone segmentation. Pattern Recog. 60 (Dec. 2016), 1057--1075.Google Scholar
- Partha Pratim Roy, Ayan Kumar Bhunia, Ayan Das, Prithviraj Dhar, and Umapada Pal. 2017b. Keyword spotting in doctor’s handwriting on medical prescriptions. Expert Syst. Appl. 76 (2017), 113--128.Google Scholar
Digital Library
- Partha Pratim Roy, Ayan Kumar Bhunia, and Umapada Pal. 2017a. HMM-based writer identification in music score documents without staff-line removal. Expert Syst. Appl. 89 (2017), 222--240.Google Scholar
Digital Library
- Partha Pratim Roy, Umapada Pal, and Josep Lladós. 2011. Document seal detection using GHT and character proximity graphs. Pattern Recog. 44, 6 (2011), 1282--1295.Google Scholar
Digital Library
- Partha Pratim Roy, Guoqiang Zhong, and Mohamed Cheriet. 2017c. Tandem hidden Markov models using deep belief networks for offline handwriting recognition. Frontiers of IT 8 EE 18, 7 (2017), 978--988.Google Scholar
- Jaakko Sauvola and Matti Pietikäinen. 2000. Adaptive document image binarization. Pattern Recog. 33, 2 (2000), 225--236.Google Scholar
Cross Ref
- Ching Y. Suen and Louisa Lam. 2000. Multiple classifier combination methodologies for different output levels. In Proceedings of the International Workshop on Multiple Classifier Systems. Springer, 52--66.Google Scholar
- Stanford University. 1999a. Neural Networks and Regularization. Retrieved from http://cs231n.github.io/neural-networks-2/#reg.Google Scholar
- Stanford University. 1999b. Unsupervised Learning. Retrieved from http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders.Google Scholar
- Lei Xu, Adam Krzyzak, and Ching Y. Suen. 1992. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. Syst., Man, Cyber. 22, 3 (1992), 418--435.Google Scholar
Cross Ref
- Xu-Yao Zhang, Yoshua Bengio, and Cheng-Lin Liu. 2017. Online and offline handwritten Chinese character recognition: A comprehensive study and new benchmark. Pattern Recog. 61 (2017), 348--360.Google Scholar
Cross Ref
Index Terms
Fusion of Spatio-temporal Information for Indic Word Recognition Combining Online and Offline Text Data
Recommendations
Offline arabic handwritten text recognition: A Survey
Research in offline Arabic handwriting recognition has increased considerably in the past few years. This is evident from the numerous research results published recently in major journals and conferences in the area of handwriting recognition. Features ...
Character and numeral recognition for non-Indic and Indic scripts: a survey
AbstractA collection of different scripts is employed in writing languages throughout the world. Character and numeral recognition of a particular script is a key area in the field of pattern recognition. In this paper, we have presented a comprehensive ...
HMM-based Indic handwritten word recognition using zone segmentation
This paper presents a novel approach towards Indic handwritten word recognition using zone-wise information. Because of complex nature due to compound characters, modifiers, overlapping and touching, etc., character segmentation and recognition is a ...






Comments