Real-time drawing assistance through crowdsourcing

Abstract

We propose a new method for the large-scale collection and analysis of drawings by using a mobile game specifically designed to collect such data. Analyzing this crowdsourced drawing database, we build a spatially varying model of artistic consensus at the stroke level. We then present a surprisingly simple stroke-correction method which uses our artistic consensus model to improve strokes in real-time. Importantly, our auto-corrections run interactively and appear nearly invisible to the user while seamlessly preserving artistic intent. Closing the loop, the game itself serves as a platform for large-scale evaluation of the effectiveness of our stroke correction algorithm.

References

1. Albinsson, P.-A., and Zhai, S. 2003. High precision touch screen interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI '03, 105--112.
2. Benko, H., and Wigdor, D. 2010. Imprecision, Inaccuracy, and Frustration: The Tale of Touch Input. in Tabletops - Horizontal Interactive Displays. Springer HCI Series. Springer-Verlag London Ltd.
3. Benko, H., Wilson, A. D., and Baudisch, P. 2006. Precise selection techniques for multi-touch screens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI '06, 1263--1272.
4. Botsch, M., and Sorkine, O. 2008. On linear variational surface deformation methods. IEEE Transactions on Visualization and Computer Graphics 14, 1 (Jan.), 213--230.
5. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H. S., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? ACM Transactions on Graphics 27, 3 (Aug.), 88:1--88:11.
6. Comaniciu, D., and Meer, P. 1999. Mean shift analysis and applications. Computer Vision, IEEE International Conference on 2, 1197.
7. Dixon, D., Prasad, M., and Hammond, T. 2010. icandraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI '10, 897--906.
8. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4 (July), 44:1--44:10.
9. Forlines, C., Forlines, C., Vogel, D., Vogel, D., Balakrishnan, R., and Balakrishnan, R. 2006. Hybrid-pointing: Fluid switching between absolute and relative pointing with a direct input device. In In UIST, ACM Press, 211.
10. Gingold, Y., Vouga, E., Grinspun, E., and Hirsh, H. 2012. Diamonds from the rough: Improving drawing, painting, and singing via crowdsourcing. In Proceedings of the AAAI Workshop on Human Computation (HCOMP).
11. Lee, Y. J., Zitnick, L., and Cohen, M. 2011. Shadowdraw: Real-time user guidance for freehand drawing. ACM Trans. Graph. 22, 3 (July), 879--887.
12. Mortensen, E. N., and Barrett, W. A. 1995. Intelligent scissors for image composition. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 191--198.
13. Orbay, G., and Kara, L. B. 2011. Beautification of Design Sketches Using Trainable Stroke Clustering and Curve Fitting. IEEE Transactions on Visualization and Computer Graphics 17, 694--708.
14. Scott, J., Izadi, S., Rezai, L. S., Ruszkowski, D., Bi, X., and Balakrishnan, R. 2010. Reartype: text entry using keys on the back of a device. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services, ACM, New York, NY, USA, MobileHCI '10, 171--180.
15. Thiel, Y., Singh, K., and Balakrishnan, R. 2011. Elasti-curves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2d curves. In Proceedings of the 24th annual ACM symposium on User interface software and technology, ACM, New York, NY, USA, UIST '11, 383--392.
16. Vogel, D., and Baudisch, P. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proc. CHI '07, ACM Press, 657--666.
17. von Ahn, L., and Dabbish, L. 2004. Labeling images with a computer game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI '04, 319--326.
18. von Ahn, L., Liu, R., and Blum, M. 2006. Peekaboom: a game for locating objects in images. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI '06, 55--64.
19. Wigdor, D., Leigh, D., Forlines, C., Shipman, S., Barnwell, J., Balakrishnan, R., and Shen, C. 2006. Under the table interaction. In Proceedings of the 19th annual ACM symposium on User interface software and technology, ACM, New York, NY, USA, UIST '06, 259--268.
20. Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., and Shen, C. 2007. Lucid touch: a see-through mobile device. In Proceedings of the 20th annual ACM symposium on User interface software and technology, ACM, New York, NY, USA, UIST '07, 269--278.
21. Wigdor, D., Williams, S., Cronin, M., Levy, R., White, K., Mazeev, M., and Benko, H. 2009. Ripples: utilizing per-contact visualizations to improve user interaction with touch displays. In Proceedings of the 22nd annual ACM symposium on User interface software and technology, ACM, New York, NY, USA, UIST '09, 3--12.
22. Zitnick, L. 2013. Handwriting beautification using tokens means. ACM Trans. Graph..

Index Terms

• Applied computing

  • Arts and humanities

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering