Evaluating audio skimming and frame rate acceleration for summarizing BBC rushes

ABSTRACT

For the first time in 2007, TRECVID considered structured evaluation of automated video summarization, utilizing BBC rushes video. In 2007, we conducted user evaluations with the published TRECVID summary assessment procedure to rate a cluster method for producing summaries, a 25x (sampling every 25th frame), and pz (emphasizing pans and zooms). Data from 4 human assessors shows significant differences between the cluster, pz, and 25x approaches. The best coverage (text inclusion performance) is obtained by 25x, but at the expense of 25x taking the most time to evaluate and judged as being the most redundant. Method pz was easier to use than cluster and rated best on redundancy. A question following the TRECVID workshop was whether simple speed-ups would still work at 50x or 100x, leading to a study with 15 human assessors looking at pzA (pz but with better audio), 25x, 50x, and 100x summaries (these latter 3 with an unsynchronized more comprehensive audio track as well). 100x gives the fastest time on task but with poor usability and performance. PzA gives the best usability measures but poor time on task and performance. 25x does well on performance as before, with 50x doing just as well but with much less time on task and better ease of use and redundancy scores. Based on these results, 50x with its audio skimming is recommended as the best way to summarize video rushes materials.

References

1. Proc. ACM Int'l Workshop on TRECVID Video Summarization (Augsburg, Germany, in conjunction with ACM Multimedia, Sept. 28, 2007), ISBN: 978-1-59593-780-3.
2. Arons, B. SpeechSkimmer: A System for Interactively Skimming Recorded Speech. ACM TOCHI 4(1), 1997, 3--38.
3. Christel, M.G., Smith, M.A., Taylor, C.R, & Winkler, D.B. Evolving Video Skims into Useful Multimedia Abstractions. In Proc. ACM CHI '98 (Los Angeles, April 1998), 171--178.
4. Hanjalic, A. Shot-Boundary Detection: Unraveled or Resolved? IEEE Transactions on Circuits and Systems for Video Technology 12(2), 2002, 90--105.
5. Hauptmann, A.G., Christel, M.G., Lin, W.-H., Maher, B., Yang, J., Baron, R.V., and Xiang, G. Clever Clustering vs. Simple Speed-Up for Summarizing BBC Rushes. In Proc. ACM Workshop on TRECVID Video Summarization (Augsburg, Germany, Sept. 2007), 20--24.
6. Over, P., Smeaton, A.F., and Kelly, P. The TRECVID 2007 BBC Rushes Summarization Evaluation Pilot. In Proc. ACM Workshop on TRECVID Video Summarization (Augsburg, Germany, Sept. 2007), 1--15.
7. Song, Y., and Marchionini, G. Effects of Audio and Visual Surrogates for Making Sense of Digital Video. In Proc. ACM CHI '07 (San Jose, CA, April-May 2007), 867--876.
8. Taskiran, C.M., Pizlo, Z., Amir, A., Ponceleon, D., and Delp, E. J. Automated Video Program Summarization Using Speech Transcripts. IEEE Transactions on Multimedia 8(4), 2006, 775--791.
9. Truong, B.T., and Venkatesh, S. Video Abstraction: A Systematic Review and Classification. ACM Trans. Multimedia Computing, Communications, and Applications (TOMCCAP) 3(1), 2007, 1--37.
10. Wildemuth, B.M., Marchionini, G., Yang, M., Geisler, G., Wilkens, T., Hughes, A., and Gruss, R. How Fast Is Too Fast? Evaluating Fast Forward Surrogates for Digital Video. In Proc. Joint Conf. Digital Libraries (Houston, TX, May 2003), 221--230.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition