skip to main content
research-article

ShotVis: Smartphone-Based Visualization of OCR Information from Images

Published:21 October 2015Publication History
Skip Abstract Section

Abstract

While visualization has been widely used as a data presentation tool in both desktop and mobile devices, the rapid visualization of information from images is still underexplored. In this work, we present a smartphone image acquisition and visualization approach for text-based data. Our prototype, ShotVis, takes images of text captured from mobile devices and extracts information for visualization. First, scattered characters in the text are processed and interactively reformulated to be stored as structured data (i.e., tables of numbers, lists of words, sentences). From there, ShotVis allows users to interactively bind visual forms to the underlying data and produce visualizations of the selected forms through touch-based interactions. In this manner, ShotVis can quickly summarize text from images into word clouds, scatterplots, and various other visualizations all through a simple click of the camera. In this way, ShotVis facilitates the interactive exploration of text data captured via cameras in smartphone devices. To demonstrate our prototype, several case studies are presented along with one user study to demonstrate the effectiveness of our approach.

Skip Supplemental Material Section

Supplemental Material

References

  1. Stefano Burigat, Luca Chittaro, and Silvia Gabrielli. 2006. Visualizing locations of off-screen objects on mobile devices: A comparative evaluation of three approaches. In Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, 239--246. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Fabio Buttussi and Luca Chittaro. 2008. MOPET: A context-aware and user-adaptive wearable system for fitness training. Artif. Intelli. Med. 42, 2, 153--163. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Luca Chittaro. 2006. Visualizing information on mobile devices. Computer 39, 3, 40--45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bernd Girod, Vijay Chandrasekhar, David M. Chen, Ngai-Man Cheung, Radek Grzeszczuk, Yuriy Reznik, Gabriel Takacs, Sam S. Tsai, and Ramakrishna Vedantham. 2011. Mobile visual search. IEEE Signal Processing Mag. 28, 4, 61--76.Google ScholarGoogle ScholarCross RefCross Ref
  5. Jie Hao and Kang Zhang. 2007. A mobile interface for hierarchical information visualization and navigation. In Proceedings of the IEEE International Symposium on Consumer Electronics, 2007 (ISCE'07). IEEE, 1--7.Google ScholarGoogle ScholarCross RefCross Ref
  6. Mark Harrower and Cynthia A. Brewer. 2003. Colorbrewer. Org: An online tool for selecting colour schemes for maps. Cartograph. J. 40, 1, 27--37.Google ScholarGoogle ScholarCross RefCross Ref
  7. Sean Kandel, Andreas Paepcke, Joseph Hellerstein, and Jeffrey Heer. 2011. Wrangler: Interactive visual specification of data transformation scripts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 3363--3372. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Eiman Kanjo, Steve Benford, Mark Paxton, Alan Chamberlain, Danae Stanton Fraser, Dawn Woodgate, David Crellin, and Adrain Woolard. 2008. MobGeoSen: Facilitating personal geosensor data collection and visualization using mobile phones. Pers. Ubiq. Comput. 12, 8, 599--607. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. SungYe Kim, Ross Maciejewski, Karl Ostmo, Edward J. Delp, Timothy F. Collins, and David S. Ebert. 2008. Mobile analytics for emergency response and training. J. Inf. Visuali. 7, 1, 77--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Fabrizio Lamberti and Andrea Sanna. 2007. A streaming-based solution for remote visualization of 3D graphics on mobile devices. IEEE Trans. Visuali. Comput. Graph. 13, 2, 247--260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Hyunjoon Lee, Eli Shechtman, Jue Wang, and Seungyong Lee. 2014. Automatic upright adjustment of photographs with robust camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5, 833--844.Google ScholarGoogle ScholarCross RefCross Ref
  12. Richard G. Lyons. 2010. Understanding Digital Signal Processing (3rd Ed.). Prentice-Hall.Google ScholarGoogle Scholar
  13. Elsa Macías, Hanna Abdelfatah, Alvaro Suárez, and Alejandro Cánovas. 2011. Full geo-localized mobile video in Android mobile telephones. Netw. Protocols Algor. 3, 1, 64--81.Google ScholarGoogle ScholarCross RefCross Ref
  14. Elsa Macías, Jaime Lloret, Alvaro Suarez, and Miguel Garcia. 2012. Architecture and protocol of a semantic system designed for video tagging with sensor data in mobile devices. Sensors 12, 2, 2062--2087.Google ScholarGoogle ScholarCross RefCross Ref
  15. Microsoft. 2014. Office lens: A OneNote scanner for your pocket. http://blogs.office.com/2014/03/17/office-lens-a-onenote-scanner-for-your-pocket/ (March 2014).Google ScholarGoogle Scholar
  16. Gary Miner, John Elder, Andrew Fast, Thomas Hill, Robert Nisbet, and Dursun Delen. 2012. Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications. Academic Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Tamara Munzner. 2014. Visualization Analysis and Design. A K Peters/CRC Press.Google ScholarGoogle Scholar
  18. Antti Oulasvirta, Tye Rattenbury, Lingyi Ma, and Eeva Raita. 2012. Habits make smartphone use more pervasive. Pers. Ubiq. Comput. 16, 1 (Jan. 2012), 105--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. A. Razip, A. Malik, S. Afzal, S. Joshi, R. Maciejewski, Y. Jang, N. Elmqvist, and D. S. Ebert. 2014. A mobile visual analytics approach for situational awareness and risk assessment. In Proceedings of IEEE PacificVis. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Jonathan C. Roberts, Panagiotis D. Ritsos, Sriram Karthik Badam, Dominique Brodbeck, Jessie Kennedy, and Niklas Elmqvist. 2014. Visualization beyond the desktop-the next big thing. IEEE Comput. Graphi. Appl. 34, 6, 26--34.Google ScholarGoogle ScholarCross RefCross Ref
  21. Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. ACM, 393--402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Rosália G. Schneider and Tinne Tuytelaars. 2014. Sketch classification and classification-driven analysis using Fisher vectors. ACM Trans. Graph. 33, 6, Article 174. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Richard Szeliski. 2010. Computer Vision: Algorithms and Applications. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Andrey Vasilev, Ilya Paramonov, Sergey Balandin, Ekaterina Dashkova, and Yevgeni Koucheryavy. 2012. Context capturing in smart space applications. Netw. Protocols Algorith. 4, 4, 84--100.Google ScholarGoogle Scholar
  25. Yao Wang, Jeorn Ostermann, and Ya-Qin Zhang. 2001. Video Processing and Communications. Prentice Hall. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Mark Weiser. 1999. The computer for the 21st century. SIGMOBILE Mob. Comput. Commun. Rev. 3, 3 (July 1999), 3--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Wikipedia. 2014. Optical character recognition. http://en.wikipedia.org/wiki/Optical_character_recognition (Nov. 2014).Google ScholarGoogle Scholar
  28. J. O. Wobbrock, A. D. Wilson, and Y. Li. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'07). ACM, 159--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Hee Yong Yoo and Suh Hyun Cheon. 2006. Visualization by information type on mobile device. In Proceedings of the Asia-Pacific Symposium on Information Visualisation. Vol. 60, Australian Computer Society, Inc., 143--146. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Hong Zhou, Huamin Qu, Yingcai Wu, and Ming-Yuen Chan. 2006. Volume visualization on mobile devices. In Proceedings of the 14th Pacific Conference on Computer Graphics and Applications. 76--84.Google ScholarGoogle Scholar

Index Terms

  1. ShotVis: Smartphone-Based Visualization of OCR Information from Images

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 12, Issue 1s
            Special Issue on Smartphone-Based Interactive Technologies, Systems, and Applications and Special Issue on Extended Best Papers from ACM Multimedia 2014
            October 2015
            317 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/2837676
            Issue’s Table of Contents

            Copyright © 2015 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 21 October 2015
            • Accepted: 1 June 2015
            • Revised: 1 April 2015
            • Received: 1 January 2015
            Published in tomm Volume 12, Issue 1s

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!