skip to main content
research-article

Sparse transfer learning for interactive video search reranking

Authors Info & Claims
Published:06 August 2012Publication History
Skip Abstract Section

Abstract

Visual reranking is effective to improve the performance of the text-based video search. However, existing reranking algorithms can only achieve limited improvement because of the well-known semantic gap between low-level visual features and high-level semantic concepts. In this article, we adopt interactive video search reranking to bridge the semantic gap by introducing user's labeling effort. We propose a novel dimension reduction tool, termed sparse transfer learning (STL), to effectively and efficiently encode user's labeling information. STL is particularly designed for interactive video search reranking. Technically, it (a) considers the pair-wise discriminative information to maximally separate labeled query relevant samples from labeled query irrelevant ones, (b) achieves a sparse representation for the subspace to encodes user's intention by applying the elastic net penalty, and (c) propagates user's labeling information from labeled samples to unlabeled samples by using the data distribution knowledge. We conducted extensive experiments on the TRECVID 2005, 2006 and 2007 benchmark datasets and compared STL with popular dimension reduction algorithms. We report superior performance by using the proposed STL-based interactive video search reranking.

References

  1. Bian, W. and Tao, D. 2011. Max-Min distance analysis by using sequential SDP relaxation for dimension reduction. IEEE Trans. Patt. Anal. Mach. Intell. 33, 5, 1037--1050. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Cai, D., He, X., and Han, J. 2005. Using graph model for face analysis. Tech. rep., Computer Science Department, University of Illinois at Urbana-Champaign.Google ScholarGoogle Scholar
  3. Cai, D., He, X., and Han, J. 2007a. Semi-supervised discriminant analysis. In Proceedings of the IEEE International Conference on Computer Vision. 1--8.Google ScholarGoogle Scholar
  4. Cai D., HE X., and Han J. 2007b. Spectral regression: A unified approach for sparse subspace learning. In Proceedings of ICDM. 73--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Carneiro, G., Chan, A. B., Moreno, P. J., and Vasconcelos, N. 2007. Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Patt. Anal. Mach. Intell. 394--410. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Chang, T. and Kuo, C.-C. 1993. Texture analysis and classification with tree-structured wavelet transform. IEEE Trans. Image Process. 2, 429--441. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Chang, H. S., Sull, S., and Lee, S. U. 1999. Efficient video indexing scheme for content-based retrieval. IEEE Trans. Circuits Syst. Video Tech. 1269--1279. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Efron, B., Hastie, T., and Tibshirani, R. 2004. Least angle regression. Ann. Stat. 32, 68--73.Google ScholarGoogle Scholar
  9. Fisher, R. A. 1936. The use of multiple measurements in taxonomic problems. Ann. Eugen. 179--188.Google ScholarGoogle Scholar
  10. Guan, N., Tao, D., Luo, Z., and Yuan, B. 2011. Manifold Regularized Discriminative Non-negative Matrix Factorization with Fast Gradient Descent. IEEE Trans. Image Proc. DOI: 10.1109/TIP.2011.2105496. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. He, X. and Niyogi, P. 2003. Locality preserving projections. In Adv. Neural Inf. Proc. Syst.Google ScholarGoogle Scholar
  12. Hotteling, H. 1933. Analysis of a complex of statistical variables into principal components. J. Educat. Psych. 24, 417--441.Google ScholarGoogle ScholarCross RefCross Ref
  13. Hsu, W. H., Kennedy, L. S., and Chang, S.-F. 2006. Video search reranking via information bottleneck principle. In Proceedings of the ACM International Conference on Multimedia. 35--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hsu, W. H., Kennedy, L. S., and Chang, S.-F. 2007. Video search reranking through random walk over document-level context graph. In Proceedings of the ACM International Conference on Multimedia. 971--980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Huang, J., Kumar, S.-R., Mitra, M., Zhu, W.-J., and Zabih, R. 1997. Image indexing using color correlograms. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 762--768. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jeon J., Lavrenko V., and Manmatha R. 2003. Automatic image annotation and retrieval using cross-media relevance models. ACM Special Interest Group on Information Retrieval. 119--126.Google ScholarGoogle Scholar
  17. Jing, Y. and Baluja, S. 2008. Visualrank: Applying pagerank to large-scale image search. IEEE Trans. Patt. Anal. Mach. Intell. 30, 1877--1890. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kotoulas, L. and Andreadis, I. 2003. Color histogram content based image retrieval and hardware implementation. IEE Proc. Circuits Devices Syst. 387--93.Google ScholarGoogle Scholar
  19. Lew, M. S., Sebe, N., Djeraba, C., and Jain R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimed. Comput. Commun. Appl. 2, 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Lin, Y.-Y., Liu, T.-L., and Chen, H.-T. 2005. Semantic manifold learning for image retrieval. In Proceedings of the ACM International Conference on Multimedia. 06--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Liu, J., Lai, W., Hua, X.-S., Huang, Y., and Li, S. 2007. Video search re-ranking via multi-graph propagation. In Proceedings of the ACM International Conference on Multimedia. 208--217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Liu, L., Rui, Y., Sun, L.-F., Yang, B., Zhang, J., and Yang, S.-Q. 2008. Topic mining on web-shared videos. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 2145--2148.Google ScholarGoogle Scholar
  23. Ma, W.-Y. and Zhang, H.-J. 1998. Benchmarking of image features for content-based retrieval. In Proceedings of the Record of the 32nd Asilomar Conference on Signals, Systems & Computers, vol. 1, 253--257.Google ScholarGoogle Scholar
  24. Neumaier, A. 1998. Solving ill-conditioned and singular linear systems: a tutorial on regularization. SIAM Rev. 40, 636--666. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Natsev, A., Haubold, A., Tesic, J., Xie, L., and Yan, R. 2007. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In Proceedings of the ACM International Conference on Multimedia. 991--1000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Nguyen, G. P. and Worring, M. 2008. Optimization of interactive visual-similarity-based search. ACM Trans. Multimed. Comput. Communicat. Appl. 4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., Wang, M., and Zhang, H.-J. 2008. Correlative multilabel video annotation with temporal kernels. ACM Trans. Multimed. Comput. Commun. Appl. 5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Robertson, S. E., Walker, S., Hancock-Beaulieu, M., Gatford, M., and Payne A. 1997. Simple, proven approaches to text retrieval. Cambridge University Computer Laboratory, Tech. rep. TR356.Google ScholarGoogle Scholar
  29. Rocchio, J. J. 1971. Relevance feedback in information retrieval. In SMART Retrieval System - Experiments in Automatic Document Processing. Prentice-Hall, 313--323.Google ScholarGoogle Scholar
  30. Roweis, S. T. and Saul, L. K. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323--2326.Google ScholarGoogle ScholarCross RefCross Ref
  31. Rui, Y., Huang, T.-S., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in MARS. In Proceedings of the IEEE International Conference on Image Processing. 815--818.Google ScholarGoogle Scholar
  32. Rui, Y., Huang, T.-S., Ortega, M., and Mehrotra, S. 1998. Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans. Circuits Syst. Video Technol. Special Issue on Segmentation, Description, and Retrieval of Video Content, 8, 644--655. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Si, S., Tao, D., and Geng, B. 2010. Bregman divergence based regularization for transfer subspace learning. IEEE Trans. Knowl. Data Eng. 22, 7, 929--942. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Slonim, N. and Tishby, N. 1999. Agglomerative information bottleneck. In Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  35. Smeulders, A. W., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 1349--1380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Tao, D., Tang, X., Li, X., and Rui, Y. 2006. Direct kernel biased discriminant analysis: A new content-based image retrieval relevance feedback algorithm. IEEE Trans. Multimed. 716--727. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Tenenbaum, J. B., de Silva, V., and Langford, J. C. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319--2323.Google ScholarGoogle ScholarCross RefCross Ref
  38. TRECVID.TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid. Trec measures. In Proceedings of Trec-10. Appendix on common evaluation measures. http://trec.nist.gov/pubs/trec10/appendices/meaures.pdf.Google ScholarGoogle Scholar
  39. Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., and Hua, X.-S. 2008. Bayesian video search reranking. In Proceedings of the ACM International Conference on Multimedia. 131--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Tian, X., Tao, D., Hua, X.-S., and Wu, X. 2010. Active reranking for Web image search. IEEE Trans. Image Process. 19, 3, 805--820. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yan, S. and Hauptmann, A. G. 2004. Co-retrieval: a boosted reranking approach for video retrieval. In Proceedings of the ACM International Conference on Content-Based Image and Video Retrieval. 60--69.Google ScholarGoogle Scholar
  42. Yan, S., Hu, Y., Xu, D., Zhang, B., Zhang, H., and Cheng, Q. 2007. Nonlinear discriminant analysis on embedded manifold. IEEE Trans. Circ. Syst. Video Tech. 17, 4, 468--477. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yang, B. 2010. DSI: A model for distributed multimedia semantic indexing and content integration. ACM Trans. Multi. Computing Communi. Appl. 6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Zhang, T., Tao, D., Li, X., and Yang, J. 2008. A unifying framework for spectral analysis based dimensionality reduction. In Proceedings of the IEEE International Conference on Neural Networks. 1670--1677.Google ScholarGoogle Scholar
  45. Zhang, Z. and Zha, H. 2004. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Sci. Comput. 26, 313--338. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Zhou, X. S. and Huang, T. S. 2001. Small sample learning during multimedia retrieval using biasmap. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Vol. 1. 11--17.Google ScholarGoogle Scholar
  47. Zhou, T., Tao, D., and Wu, X. 2011. Manifold elastic net: A unified framework for sparse dimension reduction. Data Mining Knowl. Disc. 22, 3, 340--371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Zou, H. and Hastie, T. 2005. Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. B, 67, 301--320.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Sparse transfer learning for interactive video search reranking

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 8, Issue 3
      July 2012
      143 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/2240136
      Issue’s Table of Contents

      Copyright © 2012 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 6 August 2012
      • Accepted: 1 March 2011
      • Revised: 1 December 2010
      • Received: 1 June 2010
      Published in tomm Volume 8, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!