HHMM-based Chinese lexical analyzer ICTCLAS

ABSTRACT

This document presents the results from Inst. of Computing Tech., CAS in the ACL SIGHAN-sponsored First International Chinese Word Segmentation Bake-off. The authors introduce the unified HHMM-based frame of our Chinese lexical analyzer ICTCLAS and explain the operation of the six tracks. Then provide the evaluation results and give more analysis. Evaluation on ICTCLAS shows that its performance is competitive. Compared with other system, ICTCLAS has ranked top both in CTB and PK closed track. In PK open track, it ranks second position. ICTCLAS BIG5 version was transformed from GB version only in two days; however, it achieved well in two BIG5 closed tracks. Through the first bakeoff, we could learn more about the development in Chinese word segmentation and become more confident on our HHMM-based approach. At the same time, we really find our problems during the evaluation. The bakeoff is interesting and helpful.

References

1. Lawrence. R. Rabiner. 1989. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of IEEE 77(2): pp. 257--286.
2. Shai Fine, Yoram Singer, and Naftali Tishby. 1998. The hierarchical Hidden Markov Model: Analysis and applications. Machine Learning, 32:41
3. Zhang Hua-Ping, Liu Qun. Model of Chinese Words Rough Segmentation Based on N-Shortest-Paths Method. Journal of Chinese information processing, 2002, 16(5): 1--7 (in Chinese)
4. Zhang Hua-Ping, Liu Qun, Zhang Hao and Cheng Xue-Qi. 2002. Automatic Recognition of Chinese Unknown Words Recognition. Proc. of First SigHan attached on COLING 2002
5. Zhang Hua-Ping, Liu Qun, Yu Hong-Kui, Cheng Xue-Qi, Bai Shuo. Chinese Named Entity Recognition Using Role Model. International Journal of Computational Linguistics and Chinese language processing, 2003, Vol. 8 (2)