Finding the right shots: assessing usability and performance of a digital video library interface

ABSTRACT

The authors developed a system in which visually dense displays of thumbnail imagery in storyboard views are used for shot-based video retrieval. The views allow for effective retrieval, as evidenced by the success achieved by expert users with the system in interactive query for NIST TRECVID 2002 and 2003. This paper demonstrates that novice users also achieve comparatively high retrieval performance with these views using the TRECVID 2003 benchmarks. Through an analysis of the user interaction logs, heuristic evaluation, and think-aloud protocol, the usability of the video information retrieval system is appraised with respect to shot-based retrieval. Design implications are presented based on these TRECVID usability evaluations regarding efficient, effective information retrieval interfaces to locate visual information from video corpora.

References

1. Christel, M., Huang, C., Moraveji, N., and Papernick, N. Exploiting Multiple Modalities for Interactive Video Retrieval. In Proc. ICASSP 2004 (Montreal, Canada, May 2004), Vol. III, pp. 1032--1035.
2. Dix, A., Finlay, J., Abowd, G. and Beale, R. Human Computer Interaction, 2nd Edition. Prentice Hall, 1998.
3. Doubleday, A., Ryan, M., Springett, M. and Sutcliffe, A. A Comparison of Usability Techniques for Evaluating Design. In Proc. ACM Conf. on Designing Interactive Systems (Amsterdam, Aug. 1997), pp. 101--110.
4. Gauvain, J.L., Lamel, L. and Adda, G. The LIMSI Broadcast News Transcription System. Speech Communication, 37, 1-2 (2002), pp. 89--108, ftp://tlp.limsi.fr/public/spcH4_limsi.ps.Z.
5. Komlodi, A., and Marchionini, G. Key Frame Preview Techniques for Video Browsing. In Proc. ACM Digital Libraries (Pittsburgh, PA, June 1998), pp. 118--125.
6. Lee, H. and Smeaton, A.F. Designing the User Interface for the Fischlar Digital Video Library. J. Digital Info, 2, 4 (May 2002), http://jodi.ecs.soton.ac.uk/Articles/v02/i04/Lee/.
7. Li, G., Gupta, A., Sanocki, E., He, L. and Rui, Y. Browsing Digital Video. In Proc. ACM CHI 2000 (The Hague, The Netherlands, April 2000), pp. 169--176.
8. Nielsen, J., Clemmensen, T., and Yssing, C. Getting Access to What Goes on in People's Heads? Reflections on the Think-Aloud Technique. In Proc. ACM Nordic CHI (Aarhus, Denmark, Oct. 2002), pp. 101--110.
9. Nielsen, J., and Molich, R. Heuristic Evaluation of User Interfaces. In Proc. ACM CHI 1990 (Seattle, WA, April 1990), pp. 249--256.
10. Nielsen, J. Evaluating the Thinking Aloud Technique for Use by Computer Scientists. In Hartson, H. R. and Hix, D. (Eds.), Advances in Human-Computer Interaction Vol. 3. Ablex, Norwood, NJ, 1992, pp. 75--88.
11. Nielsen, J. Heuristic Evaluation. In Nielsen, J., and Mack, R.L. (eds.), Usability Inspection Methods. John Wiley & Sons, New York, NY, 1994.
12. NIST, Digital Video Retrieval at NIST: TREC Video Retrieval Evaluation, 2001-2004, http://www-nlpir.nist.gov/projects/trecvid/.
13. Ponceleon, D., Srinivasan, S., Amir, A., Petkovic, D., and Diklic, D. Key to Effective Video Retrieval: Effective Cataloging and Browsing. In Proc. ACM Multimedia (Bristol, UK, Sept. 1998), pp. 99--107.
14. Rowe, L.A. and Jain, R. ACM SIGMM Retreat Report on Future Directions in Multimedia Research, March 2004, www.acm.org/sigmm/main/events/sigmm_retreat/sigmm-retreat03-final.htm.
15. Smeaton, A., Kraaij, W., and Over P. The TREC Video Retrieval Evaluation (TRECVID): A Case Study and Status Report. In Proc. RIAO 2004 (Avignon, France, April 2004).
16. Smeaton, A.F., and Over, P. The TREC-2002 Video Track Report. In TREC 2002 Proc. (NIST, Gaithersburg, MD, Nov. 2002), http://trec.nist.gov/pubs/trec11/t11_proceedings.html.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition