Merging storyboard strategies and automatic retrieval for improving interactive video search

ABSTRACT

The Carnegie Mellon University Informedia group has enjoyed consistent success with TRECVID interactive search using traditional storyboard interfaces for shot-based retrieval. For TRECVID 2006 the output of automatic search was included for the first time with storyboards, both as an option for an interactive user and in a different run as the sole means of access. The automatic search makes use of relevance-based probabilistic retrieval models to determine weights for combining retrieval sources when addressing a given topic. Storyboard-based access using automatic search output outperformed extreme video retrieval interfaces of manual browsing with resizable pages and rapid serial visualization of keyframes that used the same output. Further, the full Informedia interface with automatic search results as an option along with other query mechanisms scored significantly better than all other TRECVID 2006 interactive search systems. Attributes of the automatic search and interactive search systems are discussed to further optimize shot-based retrieval from news corpora.

References

1. Campbell, M., Ebadollahi, S., Naphade, M., Natsev, A., Smith, J. R., Tesic, J., Xie, L., Haubold, A. IBM Research TRECVID-2006 Video Retrieval System. In NIST TREC Video Retrieval Online Proceedings. NIST, 2006, //www-nlpir.nist.gov/projects/tvpubs/tv6.papers/ibm.pdf.
2. Christel, M., and Moraveji, N. Finding the Right Shots: Assessing Usability and Performance of a Digital Video Library Interface. In Proc. ACM Multimedia (New York, NY, Oct. 2004), ACM Press, New York, 2004, 732--739.
3. Christel, M., and Conescu, R. Mining Novice User Activity with TRECVID Interactive Retrieval Tasks. In LNCS 4071: Proc. Image and Video Retrieval (CIVR) (Tempe, AZ, July 2006), Springer, Berlin, 2006, 21--30.
4. Chua, T.-S., Neo, S.-Y., Li, K.-Y., Wang, G., Shi, R., Zhao, M., Xu, H. TRECVID 2004 Search and Feature Extraction Task by NUS PRIS. In NIST TREC Video Retrieval Online Proceedings. NIST, Gaithersburg, MD, 2004, http://www-nlpir.nist.gov/projects/tvpubs/tvpapers04/nus.pdf.
5. Hauptmann, A. G., Lin, W.-H., Yan, R., Yang, J., and Chen, M.-Y. Extreme video retrieval: joint maximization of human and computer performance. In Proc. ACM Multimedia (Santa Barbara, CA, Oct. 2006), ACM Press, New York, NY, 2006, 385--394.
6. Kraaij, W., Over, P., Ianeva, T., and Smeaton, A. TRECVID 2006 - An Introduction. In TRECVID Online Proceedings, //www-nlpir.nist.gov/projects/tvpubs/tv6.papers/tv6intro.pdf.
7. Naphade, M., Smith, J. R., Tesic, J., Chang, S.-F., Hsu, W., Kennedy, L., Hauptmann, A. and Curtis, J. Large-Scale Concept Ontology for Multimedia. IEEE MultiMedia 13(3), 2006, 86--91.
8. Snoek, C., Worring, M., Koelma, D., and Smeulders, A. Learned Lexicon-Driven Interactive Video Retrieval. In LNCS 4071: Proc. Image and Video Retrieval (CIVR) (Tempe, AZ, July 2006), Springer, Berlin, 2006, 11--20.
9. Snoek, C., Worring, M., Koelma, D., and Smeulders, A. A Learned Lexicon-Driven Paradigm for Interactive Video Retrieval. IEEE Trans. Multimedia 9(2), Feb. 2007, 280--292.
10. Yan, R., Yang, J., and Hauptmann, A. Learning Query-Class Dependent Weights in Automatic Video Retrieval. In Proc. ACM Multimedia (New York, NY, Oct. 2004), ACM Press, New York, NY, 2004, 548--555.
11. Yan, R., and Hauptmann, A. G. Efficient margin-based rank learning algorithms for information retrieval. In LNCS 4071: Proc. Image and Video Retrieval (CIVR) (Tempe, AZ, July 2006), Springer, Berlin, 2006, 113--122.

Index Terms

• Information systems

  • Information retrieval

  • Information systems applications

    • Multimedia information systems