Addressing the challenge of visual information access from digital image and video libraries

ABSTRACT

While it would seem that digital video libraries should benefit from access mechanisms directed to their visual contents, years of TREC Video Retrieval Evaluation (TRECVID) research have shown that text search against transcript narrative text provides almost all the retrieval capability, even with visually oriented generic topics. A within-subjects study involving 24 novice participants on TRECVID 2004 tasks again confirms this result. The study shows that satisfaction is greater and performance is significantly better on specific and generic information retrieval tasks from news broadcasts when transcripts are available for search. Additional runs with 7 expert users reveal different novice and expert interaction patterns with the video library interface, helping explain the novices' lack of success with image search and visual feature browsing for visual information needs. Analysis of TRECVID visual features well suited for particular tasks provides additional insights into the role of automated feature classification for digital image and video libraries.

References

1. Christel, M., and Moraveji, N. Finding the Right Shots: Assessing Usability and Performance of a Digital Video Library Interface. In Proc. ACM Multimedia (New York, October 2004), 732--739.
2. Enser, P.G.B. Pictorial information retrieval. Journal of Documentation, 51, 2 (1995), 126--170.
3. Enser, P.G.B. and Sandom, C.J. Retrieval of Archival Moving Imagery - CBIR Outside the Frame? In Image and Video Retrieval (CIVR 2002 Proceedings), 206--214.
4. French, J.C., Chapin, A.C., and Martin, W.N. An Application of Multiple Viewpoints to Content-Based Image Retrieval. In Proc. JCDL (Houston, May 2003), 128--130.
5. Graham, J. Search engine Google sets sights on video. USA Today, Jan. 25, 2005, www.usatoday.com/tech/news/2005-01-25-google-usat_x.htm http://www.usatoday.com/tech/news/2005-01-25-google-usat_x.htm.
6. Hauptmann, A., and Christel, M. Successful Approaches in the TREC Video Retrieval Evaluations. In Proc. ACM Multimedia (New York, October 2004), 668--675.
7. Jose, J.M., Furner, J., and Harper, D.J. Spatial querying for image retrieval: a user-oriented evaluation. In Proc. ACM SIGIR (1998), 232--240.
8. Kraaij, W., Smeaton, A.F., Over, P., and Arlandis, J. TRECVID 2004 - An Introduction. In TRECVID 2004 Proceedings, http://www-nlpir.nist.gov/projects/trecvid/.
9. Lee, H. and Smeaton, A.F. Designing the User Interface for the Físchlár Digital Video Library, J. Digital Info. 2(4), http://jodi.ecs.soton.ac.uk/Articles/v02/i04/Lee/, May 2002.
10. Markkula, M. and Sormunen, E. End-user searching challenges indexing practices in the digital newspaper photo archive. Information Retrieval, 1, 4 (2000), 259--285.
11. Moënne-Loccoz, N., Janvier, B., Marchand-Maillet, S., and Bruno, E. Managing Video Collections at Large. In ACM Proc. Workshop on Computer Vision meets Databases (Paris, June 2004), 59--66.
12. Nielsen, J. Heuristic Evaluation. In Nielsen, J., and Mack, R.L. (eds.), Usability Inspection Methods. John Wiley & Sons, New York, NY, 1994.
13. Rasmussen, E. Indexing images. Annual Review of Information Science and Technology, 32 (1997), 169-196.
14. Rodden, K. and Wood, K.R. How Do People Manage Their Digital Photographs? In Proc. CHI (Ft. Lauderdale, FL, April 2003), 409--416.
15. Shatford, S. Analyzing the Subject of a Picture: A Theoretical Approach. Cataloguing & Classification Quarterly, 6, 3 (Spring 1986), 39--62.
16. Shneiderman, B., Byrd, D., and Croft, W.B. Clarifying Search: A User-Interface Framework for Text Searches. D-Lib Magazine, 3, 1 (January 1997), <http://www.dlib.org>.
17. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., and Jain, R. Content based image retrieval at the end of the early years. IEEE Trans. Pattern Analysis and Machine Intelligence, 22, 12 (2000), 1349--1380.
18. Soo, V.-W., Lee, C.-Y., Li, C.-C., Chen, S.L., and Chen, C. Automated Semantic Annotation and Retrieval Based on Sharable Ontology and Case-based Learning Techniques. In Proc. JCDL (Houston, May 2003), 61--72.
19. Urban, J., Jose, J.M., and van Rijgsbergen, C.J. An Adaptive Technique for Content-Based Image Retrieval. Multimedia Tools and Applications, 25 (2005).
20. Worring, M., Smeulders, A.W.M, and Santini, S. Interaction in content-based retrieval: an evaluation of the state-of-the-art. LNCS 1929, Springer-Verlag (2000), 26--36.
21. Yang, M., Wildemuth, B., and Marchionini, G. The relative effectiveness of concept-based versus content-based video retrieval. In Proc. ACM Multimedia (Oct. 2004), 368--371.
22. Yee, K.-P., Swearingen, K., Li, K., and Hearst, M. Faceted Metadata for Image Search and Browsing. In Proc. CHI (Ft. Lauderdale, FL, April 2003), 401--408.
23. Zhou, X.S., and Huang, T. Relevance Feedback in Image Retrieval: A Comprehensive Review. ACM Multimedia Systems Journal, 8, 6 (2003), 536--544.

Index Terms

• Applied computing

  • Computers in other domains

    • Digital libraries and archives

• Information systems

  • Information retrieval

  • Information systems applications

    • Digital libraries and archives