skip to main content
research-article

Robust image annotation via simultaneous feature and sample outlier pursuit

Published:19 August 2013Publication History
Skip Abstract Section

Abstract

Graph-based semi-supervised image annotation has achieved great success in a variety of studies, yet it essentially and intuitively suffers from both the irrelevant/noisy features (referred to as feature outliers) and the unusual/corrupted samples (referred to as sample outliers). In this work, we investigate how to derive robust sample affinity matrix via simultaneous feature and sample outlier pursuit. This task is formulated as a Dual-outlier and Prior-driven Low-Rank Representation (DP-LRR) problem, which possesses convexity in objective function. In DP-LRR, the clean data are assumed to be self-reconstructible with low-rank coefficient matrix as in LRR; while the error matrix is decomposed as the sum of a row-wise sparse matrix and a column-wise sparse matrix, the ℓ2,1-norm minimization of which encourages the pursuit of feature and sample outliers respectively. The DP-LRR is further regularized by the priors from side information, that is, the inhomogeneous data pairs. An efficient iterative procedure based on linearized alternating direction method is presented to solve the DP-LRR problem, with closed-form solutions within each iteration. The derived low-rank reconstruction coefficient matrix is then fed into any graph based semi-supervised label propagation algorithm for image annotation, and as a by-product, the cleaned data from DP-LRR can also be utilized as a better image representation to generally boost image annotation performance. Extensive experiments on MIRFlickr, Corel30K, NUS-WIDE-LITE and NUS-WIDE databases well demonstrate the effectiveness of the proposed formulation for robust image annotation.

References

  1. Belkin, M., Niyogi, P., and Sindhwani, V. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bertsekas, D. P. 1982. Constrained Optimization and Lagrange Multiplier Methods. Athena Scientific.Google ScholarGoogle Scholar
  3. Blum, A. and Langley, P. 1997. Selection of relevant features and examples in machine learning. Artif. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Cai, D., Zhang, C., and He, X. 2010b. Unsupervised feature selection for multi-cluster data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cai, J.-F., Candès, E. J., and Shen, Z. 2010a. A singular value thresholding algorithm for matrix completion. SIAM J. Optim. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Candès, E. J., Li, X., Ma, Y., and Wright, J. 2011. Robust principal component analysis? J. ACM.Google ScholarGoogle Scholar
  7. Candès, E. J. and Recht, B. 2009. Exact matrix completion via convex optimization. Found. Comput. Math. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Carneiro, G., Chan, A. B., Moreno, P. J., and Vasconcelos, N. 2007. Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chang, C.-C. and Lin, C.-J. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1--27:27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Chen, X., Mu, Y., Yan, S., and Chua, T. 2010. Efficient large-scale image annotation by probabilistic collaborative multi-label propagation. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chen, X., Yuan, X., Yan, S., Tang, J., Rui, Y., and Chua, T. 2011b. Towards multi-semantic image annotation with graph regularized exclusive group lasso. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chen, X., Yuan, X.-T., Chen, Q., Yan, S., and Chua, T.-S. 2011a. Multi-label visual classification with label exclusive context. In Proceedings of the IEEE International Conference on Computer Vision. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cheng, B., Yang, J., Yan, S., Fu, Y., and Huang, T. S. 2010. Learning with l1-graph for image analysis. IEEE Trans. Image Process. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., and Zheng, Y.-T. 2009. NUS-WIDE: A real-world web image database from National University of Singapore. In Proceedings of the International Conference on Image and Video Retrieval. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Collobert, R., Sinz, F., Weston, J., and Bottou, L. 2006. Large scale transductive SVMs. J. Mach. Learn. Res. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Gao, Y., Fan, J., Xue, X., and Jain, R. 2006. Automatic image annotation by incorporating feature hierarchy and boosting to scale up SVM classifiers. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Guillaumin, M., Verbeek, J. J., and Schmid, C. 2010. Multimodal semi-supervised learning for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  19. He, B., Tao, M., and Yuan, X. 2011. Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim.Google ScholarGoogle Scholar
  20. Hsieh, L.-C. and Hsu, W. H. 2010. Search-Based Automatic Image Annotation via Flickr Photos Using Tag Expansion. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing.Google ScholarGoogle Scholar
  21. Huiskes, M. J. and Lew, M. S. 2008. The MIR Flickr Retrieval Evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Kohavi, R. and John, G. H. 1997. Wrappers for feature subset selection. Artif. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Lang, C., Liu, G., Yu, J., and Yan, S. 2012. Saliency detection by multitask sparsity pursuit. Trans. Image Process. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Li, X., Chen, L., Zhang, L., Lin, F., and Ma, W. 2006. Image annotation by large-scale content-based image retrieval. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Lin, Z., Chen, M., and Wu, L. 2010. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. Math. Program.Google ScholarGoogle Scholar
  26. Lin, Z., Liu, R., and Su, Z. 2011. Alternating direction method with adaptive penalty for low-rank representation. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  27. Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., and Ma, Y. 2010b. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Liu, G., Lin, Z., and Yu, Y. 2010a. Robust Subspace Segmentation by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Liu, H. and Motoda, H. 1998. Feature Selection for Knowledge Discovery and Data Mining. Kluwer Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Liu, J., Ji, S., and Ye, J. 2009. SLEP: Sparse Learning with Efficient Projections. Arizona State University.Google ScholarGoogle Scholar
  31. Ma, Z., Yang, Y., Nie, F., Uijlings, J., and Sebe, N. 2011. Exploiting the entire feature space with sparsity for automatic image annotation. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Ni, B., Yan, S., Kassim, A. A., and Cheong, L. F. 2008. Learning by Propagability. In Proceedings of the IEEE International Conference on Data Mining. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Nie, F., Huang, H., Cai, X., and Ding, C. 2010. Efficient and Robust Feature Selection via Joint l2,1-Norms Minimization. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  34. Peng, Y., Ganesh, A., Wright, J., Xu, W., and Ma, Y. 2011. RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Subramanya, A. and Bilmes, J. A. 2009. Entropic graph regularization in non-parametric semi-supervised classification. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  36. Tang, J., Chen, Z., Fu, A. W., and Cheung, D. W. 2007. Capabilities of outlier detection schemes in large datasets, framework and methodologies. Knowledge Inf. Syst. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Tang, J., Yan, S., Hong, R., Qi, G. J., and Chua, T. S. 2009. Inferring semantic concepts from community-contributed images and noisy tags. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Tao, M. and Yuan, X. 2011. Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Wang, C., Yan, S., Zhang, L., and Zhang, H.-J. 2009b. Multi-label sparse coding for automatic image annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  40. Wang, F. and Zhang, C. 2008. Label Propagation through Linear Neighborhoods. IEEE Trans. Knowl. Data Eng. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Wang, J., Wang, F., Zhang, C., Shen, H. C., and Quan, L. 2009a. Linear neighborhood propagation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Wang, X.-J., Zhang, L., Jing, F., and Ma, W.-Y. 2006. AnnoSearch: Image auto-annotation by search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Wang, X.-J., Zhang, L., Li, X., and Ma, W.-Y. 2008. Annotating images by mining image search results. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Xu, H., Caramanis, C., and Sanghavi, S. 2010. Robust PCA via outlier pursuit. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  45. Yang, J. and Yuan, X. 2011. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math. Comput.Google ScholarGoogle Scholar
  46. Yang, Y., Yang, Y., Huang, Z., Shen, H. T., and Nie, F. 2011. Tag localization with spatial correlations and joint group sparsity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Yang, Y., Wu, F., Nie, F., Shen, H. T., Zhuang, Y., and Hauptmann, A. G. 2012. Web and personal image annotation by mining label correlation with relaxed visual graph embedding. Trans. Image Process. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Zhao, Z., Wang, L., and Liu, H. 2010. Efficient spectral feature selection with minimum redundancy. In Proceedings of the National Conference on Artificial Intelligence.Google ScholarGoogle Scholar
  49. Zhou, D., Bousquet, O., Lal, T. N., Weston, J., and Schölkopf, B. 2003. Learning with Local and Global Consistency. In Proceedings of the Conference on Advances in Neural Information Processing Systems.Google ScholarGoogle Scholar
  50. Zhu, G., Yan, S., and Ma, Y. 2010. Image tag refinement towards low-rank, content-tag prior and error sparsity. In Proceedings of the International Conference on Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Zhu, X. and Ghahramani, Z. 2002. Learning from labeled and unlabeled data with label propagation. Tech. rep. CMU-CALD-02-107, CMU.Google ScholarGoogle Scholar
  52. Zhu, X., Ghahramani, Z., and Lafferty, J. D. 2003. Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In Proceedings of the International Conference on Machine Learning.Google ScholarGoogle Scholar

Index Terms

  1. Robust image annotation via simultaneous feature and sample outlier pursuit

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 9, Issue 4
          August 2013
          168 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/2501643
          Issue’s Table of Contents

          Copyright © 2013 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 19 August 2013
          • Accepted: 1 December 2012
          • Revised: 1 October 2012
          • Received: 1 July 2012
          Published in tomm Volume 9, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!