Alexander G. Hauptmann
Michael G. Christel
ABSTRACT
This paper reviews successful approaches in evaluations of video retrieval over the last three years. The task involves the search and retrieval of shots from MPEG digitized video recordings using a combination of automatic speech, image and video analysis and information retrieval technologies. The search evaluations are grouped into interactive (with a human in the loop) and non-interactive (where the human merely enters the query into the system) submissions. Most non-interactive search approaches have relied extensively on text retrieval, and only recently have image-based features contributed reliably to improved search performance. Interactive approaches have substantially outperformed all non-interactive approaches, with most systems relying heavily on the user's ability to refine queries and reject spurious answers. We will examine both the successful automatic search approaches and the user interface techniques that have enabled high performance video retrieval.
- Boldareva, L., de Vries, A., and Hiemstra, D. Monitoring User-System Performance in Interactive Retrieval Tasks. Proc. RIAO 2004 (Avignon, France, April 2004), pp. 474--483.Google Scholar
- Boldareva, L., and Hiemstra, D. Interactive Content-Based Retrieval Using Pre-computed Object-Object Similarities. In Proc. CIVR 2004 (Dublin, Ireland, July 2004).Google ScholarCross Ref
- Christel, M., Huang, C., Moraveji, N., and Papernick, N. A Comparative Study of Evidence Combination Strategies. In Proc. ICASSP (Montreal, May 2004), pp. 1032--1035.Google Scholar
- Christel, M., Moraveji, N., and Huang, C. Evaluating Content-Based Filters for Image and Video Retrieval. In Proc. SIGIR (Sheffield, U.K., July 2004). Google ScholarDigital Library
- Christel, M. submitted to ACM Multimedia 2004.Google Scholar
- Enser, P.G.B. and Sandom, C.J. Retrieval of Archival Moving Imagery - CBIR Outside the Frame? In Image and Video Retrieval (CIVR 2002 Proceedings), Lecture Notes in Computer Science 2383, Springer-Verlag, Berlin, 206--214. Google ScholarDigital Library
- Gauvain, J.L., Lamel, L., and Adda, G. The LIMSI Broadcast News Transcription System. Speech Communication, 37(1-2):89--108, 2002. Google ScholarDigital Library
- Lee, H. and Smeaton, A.F. Designing the User Interface for the Fischlar Digital Video Library. J. Digital Info 2(4), May 2002.Google Scholar
- NIST, Digital Video Retrieval at NIST: TREC Video Retrieval Evaluation, 2001-2004, http://www-nlpir.nist.gov/projects/trecvid/.Google Scholar
- Rautiainen, M., Ojala, T., and Seppanen, T. Cluster-temporal browsing of large news video databases. In Proc. IEEE ICME (Taipei, Taiwan, June 2004).Google ScholarCross Ref
- Rowe, L.A. and Jain, R. ACM SIGMM Retreat Report on Future Directions in Multimedia Research, March 2004, www.acm.org/sigmm/main/events/sigmm_retreat/sigmm-retreat03-final.htm.Google Scholar
- Westerveld, T. and de Vries, A. Experimental Evaluation of a Generative Probabilistic Image Retrieval Model on 'Easy' Data. In Proc. Multimedia Information Retrieval Workshop associated with ACM SIGIR 2003 (Toronto, Aug. 2003), Google ScholarDigital Library
- Wildemuth, B., Yang, M., Hughes, A., Gruss, R., Geisler, G., and Marchionini, G. Access via Features vs. Access via Transcripts: User Performance and Satisfaction. In Proc. TRECVID (Gaithersburg, MD, Nov. 2003),Google Scholar
- Yavlinsky, A., Pickering, M., Heesch, D., Ruger, S. A Comparative Study of Evidence Combination Strategies. In Proc. ICASSP (Montreal, May 2004), pp. 1040--1043.Google Scholar
- A. F. Smeaton, P. Over, C. Costello, A. P. de Vries, D. S. Doermann, A. G. Hauptmann, M. E. Rorvig, J. R. Smith, and L. Wu: The TREC2001 Video Track: Information Retrieval on Digital Video. ECDL 2002: 266--275, 2002 Google ScholarDigital Library
- R. Ruiloba, P. Joly, S. Marchand-Maillet, G. Quénot : "Towards a Standard Protocol for the Evaluation of Video-to-Shots Segmentation Algorithms", International Workshop in Content-Based Multimedia Indexing (CBMI), Toulouse France.Google Scholar
- NIST TREC 2002, Results of the Video Track, http://trec.nist.gov/pubs/trec10/appendices/video_results.htmlGoogle Scholar
- J. Smith, S. Srinivasan, A. Amir, S. Basu, G. Iyengar, C. Lin, Milind Naade, D. Ponceleon, and B Tseng, "Integrating Features, Models, and Semantics for TREC Video Retrieval," NIST TREC-10 Text Retrieval Conference, Gaithersburg, Maryland, November 2001.Google Scholar
- J. Baan, A. van Ballegooij, J-M. Geusebroek, D. Hiemstra, J. den Hartog, J. List, C. Snoek, I. Patras, S. Raaijmakers, L. Todoran, J. Vendrig, A. de Vries, T. Westerveld and M. Worring, Lazy Users and Automatic Video Retrieval Tools in (the) Lowlands. In Proceedings of the 10th Text Retrieval Conference (TREC), November 2001.Google Scholar
- Hauptmann, A., Jin, R., N. Papernick, D. Ng, Y. Qi, Houghton, R Thornton, S. Video Retrieval with the Informedia Digital Video Library System, The Tenth Text Retrieval Conference (TREC-2001) Gaithersburg, Maryland, November 13-16, 2001Google Scholar
- J. R. Smith, W.H. Adams, A. Amir, C. Dorai, S. Ghosal, G. Iyengar, A. Jaimes, C. Lang, C.-Y.Lin, A. Natsev, C. Neti, H. J. Nock, H. Permuter, R. Singh, S. Srinivasan, B. L. Tseng, AT Varadaraju, D. Zhang,"IBM Research TREC-2002 Video Retrieval System," NIST Text Retrieval Conference (TREC-2002), Nov., 2002.Google Scholar
- J. Vendrig, J. den Hartog, D. van Leeuwen, I. Patras, S. Raaijmakers, C. Snoek, J. van Rest and M. Worring, TREC Feature Extraction by Active Learning. In Proceedings of the 11th Text Retrieval Conference (TREC), Nov. 2002.Google Scholar
- X.-S. Hua, P. Yin, H. Wang, J. Chen, L. Lu, M. Li, H.-J. Zhang, "MSR-Asia at TREC-11 Video Track," TRECVID 2002.Google Scholar
- Rautiainen M, Penttilä J, Vorobiev D, Noponen K, Väyrynen P, Hosio M, Matinmikko E, Mäkelä SM, Peltola J, Ojala T & Seppäänen T (2002) TREC 2002 Video Track experiments at MediaTeam Oulu and VTT. Proc. Text Retrieval Conference TREC 2002 Video Track, Gaithersburg, MD, Nov. 2002.Google Scholar
- Wolf, C., Doermann, D., and Rautiainen, M. Video Indexing and Retrieval at UMD. Proceedings of the Text Retrieval Conference (TREC) 2002. November 19th-22th, 2002, Gaithersburg, MD, USAGoogle Scholar
- A. Hauptmann, R.V. Baron, M.-Y. Chen, M. Christel, P. Duygulu, C. Huang, R. Jin, W.-H. Lin, T. Ng, N. Moraveji, N. Papernick, C.G.M. Snoek, G. Tzanetakis, J. Yang, R. Yan, and H.D. Wactlar, Informedia at TRECVID 2003: Analyzing and Searching Broadcast News Video, Proceedings of (VIDEO) TREC 2003, November 2003.Google Scholar
- Chen, M-Y., and Hauptmann, A., Searching for a Specific Person in Broadcast News Video, International Conference on Acoustics, Speech, and Signal Processing (ICASSP'04), Montreal, Canada, May 17-21, 2004Google Scholar
- Westerveld, T,. Ianeva, T., Boldareva, L., de Vries, A.P. and Hiemstra, D. Combining Information Sources for Video Retrieval In: TRECVID 2003 Workshop, Nov., 2003Google Scholar
- A. Amir, W, Hsu, G. Iyengar, C.-Y.Lin, M. Naade, A. Natsev, C. Neti, H. J. Nock, J. R. Smith, B. L. Tseng, Y. Wu, D. Zhang, "IBM Research TRECVID-2003 System," Proc. NIST Text Retrieval Conf. (TREC), Gaithersburg, MD, Nov., 2003.Google Scholar
- The Internet Archive Movie Archive Home Page. (2002) URL: www.archive.org/moviesGoogle Scholar
- Open Video Digital Library, http://www.open-video.org/Google Scholar
- Yan, R., Hauptmann, A.G. and Jin, R., Pseudo-Relevance Feedback for Multimedia Retrieval, in Video Mining, Rosenfeld, A., Doermann, D., and DeMenthon, D. (eds), Kluwer, Boston, pp. 309--338, 2003.Google Scholar
Index Terms
Successful approaches in the TREC video retrieval evaluations
Recommendations
News video retrieval by learning multimodal semantic information
VISUAL'07: Proceedings of the 9th international conference on Advances in visual information systemsWith the explosion of multimedia data especially that of video data, requirement of efficient video retrieval has becoming more and more important. Years of TREC Video Retrieval Evaluation (TRECVID) research gives benchmark for video search task. The ...
The relative effectiveness of concept-based versus content-based video retrieval
MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on MultimediaThree video search systems were compared in the interactive search task at the TRECVID 2003 workshop: a <i>text-only</i> system, which searched video shots through transcripts; a <i>features-only</i> system, which searched video shots through 16 video ...
Multimodal Video Retrieval with the 2017 IMOTION System
ICMR '17: Proceedings of the 2017 ACM on International Conference on Multimedia RetrievalThe IMOTION system is a multimodal content-based video search and browsing application offering a rich set of query modes on the basis of a broad range of different features. It is able to scale with the size of the collection due to its underlying ...
Comments