Multimodal Access to Georeferenced Mobile Video through Shape, Speed and Time

ABSTRACT

Video is becoming increasingly accessed, captured, and published on the Web, from different platforms and devices. Users can easily georeference the information they capture and access, allowing to enrich their contextualization. But video search has been limited to keywords, or a set of parameters, providing limited support for temporal and spatial dimensions. We propose novel ways to search and access georeferenced videos, where these dimensions are of central importance, especially by video trajectories shape and speed, and by time, using a multimodal interactive mobile interface, involving gestures and movement, with the potential for more natural interactions, increased engagement, sense of presence and immersion. The evaluation based on high-fidelity prototypes had positive results. Users found most features useful, satisfactory, sometimes fun, and easy to use. Different options and modalities were found interesting and adequate for different use scenarios that could be identified and suggested, and some concerns and challenges were identified to be taken into account in future developments, towards more flexible and effective interactive content access, through more natural interaction with mobile devices on their own or as second screens to a larger screen on TV or public displays.

References

1. Courtois, C., D'heer, E. (2012) Second screen applications and tablet users: constellation, awareness, experience, and interest. Proc. EuroiTV'12, ACM Press, 153-156.
2. Finsterwald, J. M., Grefenstette, G., Law-To, J., Bouchard, H., and Mezaour, A.D (2012) The Movie Mashup Application MoMa: Geolocalizing and Finding Movies. Proc. GeoMM'12, ACM, 15--18.
3. Hao, J., Wang, G., Seo, B., and Zimmermann, R. (2011) Keyframe Presentation for Browsing of User-generated Videos on Map Interfaces. Proc. ACM MM'11, ACM Press, 1013--1016.
4. Lei, Z., and Coulton, P. A. (2009) Mobile Geo-wand Enabling Gesture Based POI Search an User Generated Directional POI Photography. Proc. ACE'09, Int. Conf. on Advances in Computer Entertainment Technology, ACM, 392--395.
5. Lund, A.M. (2001) Measuring usability with the USE questionnaire. Usability and User Experience, 8(2).
6. Norman, D. (1993) Things that Make us Smart. Addison Wesley Publishing Company.
7. Noronha, G., Álvares, C., and Chambel, T. (2012) Sight Surfers: 360° Videos and Maps Navigation. Proc. GeoMM'12, ACM Press, 19--22.
8. Premraj, V., Schedel, M., and Berg, T.L. (2010) iWalk, A Tool for Interacting with Geo-Located Data Through Movement and Gesture. Proc. ACM MM'10, ACM Press, 1059--1062.
9. Ramalho, J. and Chambel, T. (2013a) Windy Sight Surfers: sensing and awareness of 360° immersive videos on the move. In Proc. of EuroITV 2013, Como, Italy. 107--116.
10. Ramalho, J., and Chambel, T. (2013b) Immersive 360° Mobile Video with an Emotional Perspective. Proc. ImmersiveMe'2013, ACM Press, 35--40.
11. Rego, A., Baptista, C., Silva, E., Schiel, U., and Figueirědo, H. (2007) VideoLib: a Video Digital Library with Support to Spatial and Temporal Dimensions. Proc. SAC'07, ACM, 1074--1078.
12. Scheible, J., Ojala, T., and Coulton, P. (2008) MobiToss: A novel gesture based interface for creating and sharing mobile multimedia art on large public displays. Proc. ACM MM'08, ACM Press, 957--960.
13. Serra, S., and Chambel, T. (2014). Towards Multimodal Consumption of Georeferenced Mobile Video Using Shape and Speed. In Proc. of 2nd Int. Workshop on Interactive Content Consumption at ACM TVX 2014.

Index Terms

• Applied computing

  • Computers in other domains

    • Personal computers and PC applications

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods