Abstract
While visualization has been widely used as a data presentation tool in both desktop and mobile devices, the rapid visualization of information from images is still underexplored. In this work, we present a smartphone image acquisition and visualization approach for text-based data. Our prototype, ShotVis, takes images of text captured from mobile devices and extracts information for visualization. First, scattered characters in the text are processed and interactively reformulated to be stored as structured data (i.e., tables of numbers, lists of words, sentences). From there, ShotVis allows users to interactively bind visual forms to the underlying data and produce visualizations of the selected forms through touch-based interactions. In this manner, ShotVis can quickly summarize text from images into word clouds, scatterplots, and various other visualizations all through a simple click of the camera. In this way, ShotVis facilitates the interactive exploration of text data captured via cameras in smartphone devices. To demonstrate our prototype, several case studies are presented along with one user study to demonstrate the effectiveness of our approach.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, ShotVis: Smartphone-Based Visualization of OCR Information from Images
- Stefano Burigat, Luca Chittaro, and Silvia Gabrielli. 2006. Visualizing locations of off-screen objects on mobile devices: A comparative evaluation of three approaches. In Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, 239--246. Google Scholar
Digital Library
- Fabio Buttussi and Luca Chittaro. 2008. MOPET: A context-aware and user-adaptive wearable system for fitness training. Artif. Intelli. Med. 42, 2, 153--163. Google Scholar
Digital Library
- Luca Chittaro. 2006. Visualizing information on mobile devices. Computer 39, 3, 40--45. Google Scholar
Digital Library
- Bernd Girod, Vijay Chandrasekhar, David M. Chen, Ngai-Man Cheung, Radek Grzeszczuk, Yuriy Reznik, Gabriel Takacs, Sam S. Tsai, and Ramakrishna Vedantham. 2011. Mobile visual search. IEEE Signal Processing Mag. 28, 4, 61--76.Google Scholar
Cross Ref
- Jie Hao and Kang Zhang. 2007. A mobile interface for hierarchical information visualization and navigation. In Proceedings of the IEEE International Symposium on Consumer Electronics, 2007 (ISCE'07). IEEE, 1--7.Google Scholar
Cross Ref
- Mark Harrower and Cynthia A. Brewer. 2003. Colorbrewer. Org: An online tool for selecting colour schemes for maps. Cartograph. J. 40, 1, 27--37.Google Scholar
Cross Ref
- Sean Kandel, Andreas Paepcke, Joseph Hellerstein, and Jeffrey Heer. 2011. Wrangler: Interactive visual specification of data transformation scripts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 3363--3372. Google Scholar
Digital Library
- Eiman Kanjo, Steve Benford, Mark Paxton, Alan Chamberlain, Danae Stanton Fraser, Dawn Woodgate, David Crellin, and Adrain Woolard. 2008. MobGeoSen: Facilitating personal geosensor data collection and visualization using mobile phones. Pers. Ubiq. Comput. 12, 8, 599--607. Google Scholar
Digital Library
- SungYe Kim, Ross Maciejewski, Karl Ostmo, Edward J. Delp, Timothy F. Collins, and David S. Ebert. 2008. Mobile analytics for emergency response and training. J. Inf. Visuali. 7, 1, 77--88. Google Scholar
Digital Library
- Fabrizio Lamberti and Andrea Sanna. 2007. A streaming-based solution for remote visualization of 3D graphics on mobile devices. IEEE Trans. Visuali. Comput. Graph. 13, 2, 247--260. Google Scholar
Digital Library
- Hyunjoon Lee, Eli Shechtman, Jue Wang, and Seungyong Lee. 2014. Automatic upright adjustment of photographs with robust camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5, 833--844.Google Scholar
Cross Ref
- Richard G. Lyons. 2010. Understanding Digital Signal Processing (3rd Ed.). Prentice-Hall.Google Scholar
- Elsa Macías, Hanna Abdelfatah, Alvaro Suárez, and Alejandro Cánovas. 2011. Full geo-localized mobile video in Android mobile telephones. Netw. Protocols Algor. 3, 1, 64--81.Google Scholar
Cross Ref
- Elsa Macías, Jaime Lloret, Alvaro Suarez, and Miguel Garcia. 2012. Architecture and protocol of a semantic system designed for video tagging with sensor data in mobile devices. Sensors 12, 2, 2062--2087.Google Scholar
Cross Ref
- Microsoft. 2014. Office lens: A OneNote scanner for your pocket. http://blogs.office.com/2014/03/17/office-lens-a-onenote-scanner-for-your-pocket/ (March 2014).Google Scholar
- Gary Miner, John Elder, Andrew Fast, Thomas Hill, Robert Nisbet, and Dursun Delen. 2012. Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications. Academic Press. Google Scholar
Digital Library
- Tamara Munzner. 2014. Visualization Analysis and Design. A K Peters/CRC Press.Google Scholar
- Antti Oulasvirta, Tye Rattenbury, Lingyi Ma, and Eeva Raita. 2012. Habits make smartphone use more pervasive. Pers. Ubiq. Comput. 16, 1 (Jan. 2012), 105--114. Google Scholar
Digital Library
- A. Razip, A. Malik, S. Afzal, S. Joshi, R. Maciejewski, Y. Jang, N. Elmqvist, and D. S. Ebert. 2014. A mobile visual analytics approach for situational awareness and risk assessment. In Proceedings of IEEE PacificVis. Google Scholar
Digital Library
- Jonathan C. Roberts, Panagiotis D. Ritsos, Sriram Karthik Badam, Dominique Brodbeck, Jessie Kennedy, and Niklas Elmqvist. 2014. Visualization beyond the desktop-the next big thing. IEEE Comput. Graphi. Appl. 34, 6, 26--34.Google Scholar
Cross Ref
- Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. Revision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. ACM, 393--402. Google Scholar
Digital Library
- Rosália G. Schneider and Tinne Tuytelaars. 2014. Sketch classification and classification-driven analysis using Fisher vectors. ACM Trans. Graph. 33, 6, Article 174. Google Scholar
Digital Library
- Richard Szeliski. 2010. Computer Vision: Algorithms and Applications. Springer. Google Scholar
Digital Library
- Andrey Vasilev, Ilya Paramonov, Sergey Balandin, Ekaterina Dashkova, and Yevgeni Koucheryavy. 2012. Context capturing in smart space applications. Netw. Protocols Algorith. 4, 4, 84--100.Google Scholar
- Yao Wang, Jeorn Ostermann, and Ya-Qin Zhang. 2001. Video Processing and Communications. Prentice Hall. Google Scholar
Digital Library
- Mark Weiser. 1999. The computer for the 21st century. SIGMOBILE Mob. Comput. Commun. Rev. 3, 3 (July 1999), 3--11. Google Scholar
Digital Library
- Wikipedia. 2014. Optical character recognition. http://en.wikipedia.org/wiki/Optical_character_recognition (Nov. 2014).Google Scholar
- J. O. Wobbrock, A. D. Wilson, and Y. Li. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'07). ACM, 159--168. Google Scholar
Digital Library
- Hee Yong Yoo and Suh Hyun Cheon. 2006. Visualization by information type on mobile device. In Proceedings of the Asia-Pacific Symposium on Information Visualisation. Vol. 60, Australian Computer Society, Inc., 143--146. Google Scholar
Digital Library
- Hong Zhou, Huamin Qu, Yingcai Wu, and Ming-Yuen Chan. 2006. Volume visualization on mobile devices. In Proceedings of the 14th Pacific Conference on Computer Graphics and Applications. 76--84.Google Scholar
Index Terms
ShotVis: Smartphone-Based Visualization of OCR Information from Images
Recommendations
Data Visualization on Mobile Devices
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing SystemsAs mobile visualization is increasingly used and new mobile device form factors and hardware capabilities continuously emerge, it is timely to reflect on what has been discovered to date and to look into the future. This workshop will bring together ...
A Nested Hierarchy of Localized Scatterplots
SIBGRAPI '14: Proceedings of the 2014 27th SIBGRAPI Conference on Graphics, Patterns and ImagesThe simplicity and visual clarity of scatterplots makes them one of the most widely-used visualization techniques for multivariate data. In complex data sets the important information can be hidden in subsets of the data, often obscured in the typical ...
Human Genome Data Visualization Using a Wall Type Display
APCHI '08: Proceedings of the 8th Asia-Pacific conference on Computer-Human InteractionInformation visualization is an effective method to grasp a complete view of huge data. In the field of genome science, researchers are faced with a large biological data set analysis. Large biological data set are accumulated in public web sites. They ...






Comments