Abstract
Visual reranking is effective to improve the performance of the text-based video search. However, existing reranking algorithms can only achieve limited improvement because of the well-known semantic gap between low-level visual features and high-level semantic concepts. In this article, we adopt interactive video search reranking to bridge the semantic gap by introducing user's labeling effort. We propose a novel dimension reduction tool, termed sparse transfer learning (STL), to effectively and efficiently encode user's labeling information. STL is particularly designed for interactive video search reranking. Technically, it (a) considers the pair-wise discriminative information to maximally separate labeled query relevant samples from labeled query irrelevant ones, (b) achieves a sparse representation for the subspace to encodes user's intention by applying the elastic net penalty, and (c) propagates user's labeling information from labeled samples to unlabeled samples by using the data distribution knowledge. We conducted extensive experiments on the TRECVID 2005, 2006 and 2007 benchmark datasets and compared STL with popular dimension reduction algorithms. We report superior performance by using the proposed STL-based interactive video search reranking.
- Bian, W. and Tao, D. 2011. Max-Min distance analysis by using sequential SDP relaxation for dimension reduction. IEEE Trans. Patt. Anal. Mach. Intell. 33, 5, 1037--1050. Google Scholar
Digital Library
- Cai, D., He, X., and Han, J. 2005. Using graph model for face analysis. Tech. rep., Computer Science Department, University of Illinois at Urbana-Champaign.Google Scholar
- Cai, D., He, X., and Han, J. 2007a. Semi-supervised discriminant analysis. In Proceedings of the IEEE International Conference on Computer Vision. 1--8.Google Scholar
- Cai D., HE X., and Han J. 2007b. Spectral regression: A unified approach for sparse subspace learning. In Proceedings of ICDM. 73--82. Google Scholar
Digital Library
- Carneiro, G., Chan, A. B., Moreno, P. J., and Vasconcelos, N. 2007. Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Patt. Anal. Mach. Intell. 394--410. Google Scholar
Digital Library
- Chang, T. and Kuo, C.-C. 1993. Texture analysis and classification with tree-structured wavelet transform. IEEE Trans. Image Process. 2, 429--441. Google Scholar
Digital Library
- Chang, H. S., Sull, S., and Lee, S. U. 1999. Efficient video indexing scheme for content-based retrieval. IEEE Trans. Circuits Syst. Video Tech. 1269--1279. Google Scholar
Digital Library
- Efron, B., Hastie, T., and Tibshirani, R. 2004. Least angle regression. Ann. Stat. 32, 68--73.Google Scholar
- Fisher, R. A. 1936. The use of multiple measurements in taxonomic problems. Ann. Eugen. 179--188.Google Scholar
- Guan, N., Tao, D., Luo, Z., and Yuan, B. 2011. Manifold Regularized Discriminative Non-negative Matrix Factorization with Fast Gradient Descent. IEEE Trans. Image Proc. DOI: 10.1109/TIP.2011.2105496. Google Scholar
Digital Library
- He, X. and Niyogi, P. 2003. Locality preserving projections. In Adv. Neural Inf. Proc. Syst.Google Scholar
- Hotteling, H. 1933. Analysis of a complex of statistical variables into principal components. J. Educat. Psych. 24, 417--441.Google Scholar
Cross Ref
- Hsu, W. H., Kennedy, L. S., and Chang, S.-F. 2006. Video search reranking via information bottleneck principle. In Proceedings of the ACM International Conference on Multimedia. 35--44. Google Scholar
Digital Library
- Hsu, W. H., Kennedy, L. S., and Chang, S.-F. 2007. Video search reranking through random walk over document-level context graph. In Proceedings of the ACM International Conference on Multimedia. 971--980. Google Scholar
Digital Library
- Huang, J., Kumar, S.-R., Mitra, M., Zhu, W.-J., and Zabih, R. 1997. Image indexing using color correlograms. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 762--768. Google Scholar
Digital Library
- Jeon J., Lavrenko V., and Manmatha R. 2003. Automatic image annotation and retrieval using cross-media relevance models. ACM Special Interest Group on Information Retrieval. 119--126.Google Scholar
- Jing, Y. and Baluja, S. 2008. Visualrank: Applying pagerank to large-scale image search. IEEE Trans. Patt. Anal. Mach. Intell. 30, 1877--1890. Google Scholar
Digital Library
- Kotoulas, L. and Andreadis, I. 2003. Color histogram content based image retrieval and hardware implementation. IEE Proc. Circuits Devices Syst. 387--93.Google Scholar
- Lew, M. S., Sebe, N., Djeraba, C., and Jain R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimed. Comput. Commun. Appl. 2, 1--19. Google Scholar
Digital Library
- Lin, Y.-Y., Liu, T.-L., and Chen, H.-T. 2005. Semantic manifold learning for image retrieval. In Proceedings of the ACM International Conference on Multimedia. 06--11. Google Scholar
Digital Library
- Liu, J., Lai, W., Hua, X.-S., Huang, Y., and Li, S. 2007. Video search re-ranking via multi-graph propagation. In Proceedings of the ACM International Conference on Multimedia. 208--217. Google Scholar
Digital Library
- Liu, L., Rui, Y., Sun, L.-F., Yang, B., Zhang, J., and Yang, S.-Q. 2008. Topic mining on web-shared videos. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 2145--2148.Google Scholar
- Ma, W.-Y. and Zhang, H.-J. 1998. Benchmarking of image features for content-based retrieval. In Proceedings of the Record of the 32nd Asilomar Conference on Signals, Systems & Computers, vol. 1, 253--257.Google Scholar
- Neumaier, A. 1998. Solving ill-conditioned and singular linear systems: a tutorial on regularization. SIAM Rev. 40, 636--666. Google Scholar
Digital Library
- Natsev, A., Haubold, A., Tesic, J., Xie, L., and Yan, R. 2007. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In Proceedings of the ACM International Conference on Multimedia. 991--1000. Google Scholar
Digital Library
- Nguyen, G. P. and Worring, M. 2008. Optimization of interactive visual-similarity-based search. ACM Trans. Multimed. Comput. Communicat. Appl. 4. Google Scholar
Digital Library
- Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., Wang, M., and Zhang, H.-J. 2008. Correlative multilabel video annotation with temporal kernels. ACM Trans. Multimed. Comput. Commun. Appl. 5. Google Scholar
Digital Library
- Robertson, S. E., Walker, S., Hancock-Beaulieu, M., Gatford, M., and Payne A. 1997. Simple, proven approaches to text retrieval. Cambridge University Computer Laboratory, Tech. rep. TR356.Google Scholar
- Rocchio, J. J. 1971. Relevance feedback in information retrieval. In SMART Retrieval System - Experiments in Automatic Document Processing. Prentice-Hall, 313--323.Google Scholar
- Roweis, S. T. and Saul, L. K. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323--2326.Google Scholar
Cross Ref
- Rui, Y., Huang, T.-S., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in MARS. In Proceedings of the IEEE International Conference on Image Processing. 815--818.Google Scholar
- Rui, Y., Huang, T.-S., Ortega, M., and Mehrotra, S. 1998. Relevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans. Circuits Syst. Video Technol. Special Issue on Segmentation, Description, and Retrieval of Video Content, 8, 644--655. Google Scholar
Digital Library
- Si, S., Tao, D., and Geng, B. 2010. Bregman divergence based regularization for transfer subspace learning. IEEE Trans. Knowl. Data Eng. 22, 7, 929--942. Google Scholar
Digital Library
- Slonim, N. and Tishby, N. 1999. Agglomerative information bottleneck. In Advances in Neural Information Processing Systems.Google Scholar
- Smeulders, A. W., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 1349--1380. Google Scholar
Digital Library
- Tao, D., Tang, X., Li, X., and Rui, Y. 2006. Direct kernel biased discriminant analysis: A new content-based image retrieval relevance feedback algorithm. IEEE Trans. Multimed. 716--727. Google Scholar
Digital Library
- Tenenbaum, J. B., de Silva, V., and Langford, J. C. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319--2323.Google Scholar
Cross Ref
- TRECVID.TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid. Trec measures. In Proceedings of Trec-10. Appendix on common evaluation measures. http://trec.nist.gov/pubs/trec10/appendices/meaures.pdf.Google Scholar
- Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., and Hua, X.-S. 2008. Bayesian video search reranking. In Proceedings of the ACM International Conference on Multimedia. 131--140. Google Scholar
Digital Library
- Tian, X., Tao, D., Hua, X.-S., and Wu, X. 2010. Active reranking for Web image search. IEEE Trans. Image Process. 19, 3, 805--820. Google Scholar
Digital Library
- Yan, S. and Hauptmann, A. G. 2004. Co-retrieval: a boosted reranking approach for video retrieval. In Proceedings of the ACM International Conference on Content-Based Image and Video Retrieval. 60--69.Google Scholar
- Yan, S., Hu, Y., Xu, D., Zhang, B., Zhang, H., and Cheng, Q. 2007. Nonlinear discriminant analysis on embedded manifold. IEEE Trans. Circ. Syst. Video Tech. 17, 4, 468--477. Google Scholar
Digital Library
- Yang, B. 2010. DSI: A model for distributed multimedia semantic indexing and content integration. ACM Trans. Multi. Computing Communi. Appl. 6. Google Scholar
Digital Library
- Zhang, T., Tao, D., Li, X., and Yang, J. 2008. A unifying framework for spectral analysis based dimensionality reduction. In Proceedings of the IEEE International Conference on Neural Networks. 1670--1677.Google Scholar
- Zhang, Z. and Zha, H. 2004. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Sci. Comput. 26, 313--338. Google Scholar
Digital Library
- Zhou, X. S. and Huang, T. S. 2001. Small sample learning during multimedia retrieval using biasmap. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Vol. 1. 11--17.Google Scholar
- Zhou, T., Tao, D., and Wu, X. 2011. Manifold elastic net: A unified framework for sparse dimension reduction. Data Mining Knowl. Disc. 22, 3, 340--371. Google Scholar
Digital Library
- Zou, H. and Hastie, T. 2005. Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. B, 67, 301--320.Google Scholar
Cross Ref
Index Terms
Sparse transfer learning for interactive video search reranking
Recommendations
Reranking search results for sparse queries
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge managementIt is well known that clickthrough data can be used to improve the effectiveness of search results: broadly speaking, a query's past clicks are a predictor of future clicks on documents. However, when a new or unusual query appears, or when a system is ...
Bayesian video search reranking
MM '08: Proceedings of the 16th ACM international conference on MultimediaContent-based video search reranking can be regarded as a process that uses visual content to recover the "true" ranking list from the noisy one generated based on textual information. This paper explicitly formulates this problem in the Bayesian ...
Graph-Based Pairwise Learning to Rank for Video Search
MMM '09: Proceedings of the 15th International Multimedia Modeling Conference on Advances in Multimedia ModelingLearning-based ranking is a promising approach to a variety of search tasks, which is aimed at automatically creating the ranking model based on training samples and machine learning techniques. However, the problem of lacking training samples labeled ...






Comments