Handwriting beautification using token means

Abstract

In this paper, we propose a general purpose approach to handwriting beautification using online input from a stylus. Given a sample of writings, drawings, or sketches from the same user, our method improves a user's strokes in real-time as they are drawn. Our approach relies on one main insight. The appearance of the average of multiple instances of the same written word or shape is better than most of the individual instances. We utilize this observation using a two-stage approach. First, we propose an efficient real-time method for finding matching sets of stroke samples called tokens in a potentially large database of writings from a user. Second, we refine the user's most recently written strokes by averaging them with the matching tokens. Our approach works without handwriting recognition, and does not require a database of predefined letters, words, or shapes. Our results show improved results for a wide range of writing styles and drawings.

References

1. Arvo, J., and Novins, K. 2000. Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes. In In Proc. of UIST, ACM.
2. Bahlmann, C., and Burkhardt, H. 2004. The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping. PAMI 26, 3.
3. Bahlmann, C., Haasdonk, B., and Burkhardt, H. 2002. Online handwriting recognition with support vector machines-a kernel approach. In Proc. of Work. on Frontiers in Handwriting Recognition, IEEE.
4. Baran, I., Lehtinen, J., and Popović, J. 2010. Sketching clothoid splines using shortest paths. In Computer Graphics Forum, vol. 29.
5. Belhumeur, P. 1996. A bayesian approach to binocular steropsis. IJCV 19, 3.
6. Buades, A., Coll, B., and Morel, J. 2005. A non-local algorithm for image denoising. In In Proc. of CVPR, vol. 2.
7. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2012. Where do people draw lines? ACM Trans. Graph. 55, 1.
8. Dixon, D., Prasad, M., and Hammond, T. 2010. icandraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In Proc. of the Int. Conf. on Human Factors in Computing Systems, ACM.
9. Dudek, G., and Tsotsos, J. 1997. Shape representation and recognition from multiscale curvature. Computer Vision and Image Understanding 68, 2.
10. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4.
11. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., and Schmidhuber, J. 2009. A novel connectionist system for unconstrained handwriting recognition. PAMI 31, 5.
12. Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., and Janet, S. 1994. Unipen project of on-line data exchange and recognizer benchmarks. In In Proc. of ICPR, vol. 2, IEEE.
13. Igarashi, T., Matsuoka, S., Kawachiya, S., and Tanaka, H. 1997. Interactive beautification: a technique for rapid geometric design. In In Proc. of UIST.
14. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: a sketching interface for 3d freeform design. In Proc. of the Conf. on Computer Graphics and Interactive Techniques.
15. Jain, A., and Namboodiri, A. 2003. Indexing and retrieval of on-line handwritten documents. In Conf. on Document Analysis and Recognition.
16. Jawahar, C., Balasubramanian, A., Meshesha, M., and Namboodiri, A. 2009. Retrieval of online handwriting by synthesis and matching. Pattern Recognition 42, 7.
17. Karpenko, O., and Hughes, J. 2006. Smoothsketch: 3d free-form shapes from complex sketches. In ACM Trans. Graph., vol. 25.
18. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proc. of the IEEE 86, 11.
19. Lee, Y., Zitnick, C., and Cohen, M. 2011. Shadowdraw: real-time user guidance for freehand drawing. In ACM Trans. Graph., vol. 30.
20. Limpaecher, A., Feltman, N., Treuille, A., and Cohen, M. 2013. Real-time drawing assistance through crowdsourcing. ACM Trans. Graph. 32, 4.
21. Lu, J., Yu, F., Finkelstein, A., and DiVerdi, S. 2012. Helpinghand: example-based stroke stylization. ACM Trans. Graph. 31, 4.
22. Mokhtarian, F., and Mackworth, A. 1992. A theory of multiscale, curvature-based shape representation for planar curves. PAMI 14, 8.
23. Olsen, L., Samavati, F., Sousa, M., and Jorge, J. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1.
24. Orbay, G., and Kara, L. 2011. Beautification of design sketches using trainable stroke clustering and curve fitting. TVCG 17, 5.
25. Pavlidis, T., and Van Wyk, C. 1985. An automatic beautifier for drawings and illustrations. Computer Graphics 85 19, 3.
26. Plamondon, R., and Srihari, S. 2000. Online and off-line handwriting recognition: a comprehensive survey. PAMI 22, 1.
27. Plötz, T., and Fink, G. 2009. Markov models for offline handwriting recognition: a survey. Int. J. on Document Analysis and Recognition 12, 4.
28. Rivers, A., Adams, A., and Durand, F. 2012. Sculpting by numbers. ACM Trans. Graph. 31, 6.
29. Schmid, J., Senn, M., Gross, M., and Sumner, R. 2011. Overcoat: an implicit canvas for 3d painting. ACM Trans. Graph. 30, 4.
30. Senior, A., and Robinson, A. 1998. An off-line cursive handwriting recognition system. PAMI 20, 3.
31. Simard, P., Steinkraus, D., and Agrawala, M. 2005. Ink normalization and beautification. In Conf. on Document Analysis and Recognition, IEEE.
32. Thiel, Y., Singh, K., and Balakrishnan, R. 2011. Elasti-curves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2d curves. In In Proc. of UIST.
33. Vinciarelli, A., and Perrone, M. 2003. Combining online and offline handwriting recognition. In Conf. on Document Analysis and Recognition.
34. Whitney, H. 1937. On regular closed curves in the plane. Compositio Mathematica 4.
35. Zanibbi, R., Novins, K., Arvo, J., and Zanibbi, K. 2001. Aiding manipulation of handwritten mathematical expressions through style-preserving morphs. Graphics Interface 2001.

Index Terms

• Applied computing

  • Document management and text processing

    • Document capture

      • Graphics recognition and interpretation

• Computing methodologies

  • Computer graphics