skip to main content
research-article

Multifeature analysis and semantic context learning for image classification

Published:10 May 2013Publication History
Skip Abstract Section

Abstract

This article introduces an image classification approach in which the semantic context of images and multiple low-level visual features are jointly exploited. The context consists of a set of semantic terms defining the classes to be associated to unclassified images. Initially, a multiobjective optimization technique is used to define a multifeature fusion model for each semantic class. Then, a Bayesian learning procedure is applied to derive a context model representing relationships among semantic classes. Finally, this context model is used to infer object classes within images. Selected results from a comprehensive experimental evaluation are reported to show the effectiveness of the proposed approaches.

References

  1. Aksoy, S., Koperski, K., Tusk, C., Marchisio, G., and Tilton, J. 2005. Learning bayesian classifiers for scene classification. IEEE Trans. Geosci. Remote Sensing 43, 3, 581--589.Google ScholarGoogle ScholarCross RefCross Ref
  2. Athanasiadis, T., Mylonas, P., Avrithis, Y., and Kollias, S. 2007. Semantic image segmentation and object labeling. IEEE Trans. Circ. Syst. Video Technol. 17, 3, 298--312. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Berman, A. and Shapiro, L. 1999. A flexible image database system for content-based retrieval. Comput. Vis. Image Understand. 75, 1--2, 175--195. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Chang, S., Sikora, T., and Purl, A. 2001. Overview of the mpeg-7 standard. IEEE Trans. Circ. Syst. Video Technol. 11, 6, 688--695. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Chun, Y., Kim, N., and Jang, I. 2008. Content-Based image retrieval using multiresolution color and texture features. IEEE Trans. Multimedia 10, 6, 1073--1084. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Cooper, G. and Herskovits, E. 1992. A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9, 4, 309--347. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. 2004. Visual categorization with bags of keypoints. In Proceedings of the Workshop on Statistical Learning in Computer Vision. Vol. 1, 1--22.Google ScholarGoogle Scholar
  8. Dagli, C. and Huang, T. 2004. A framework for grid-based image retrieval. In Proceedings of the 17th IEEE International Conference on Pattern Recognition. Vol. 2, 1021--1024. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Datta, R., Joshi, D., Li, J., and Wang, J. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. 40, 2, 5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. de Jong, F., Westerveld, T., and de Vries, A. 2007. Multimedia search without visual analysis: the value of linguistic and contextual information. IEEE Trans. Circ. Syst. Video Technol. 17, 3, 365--371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Deselaers, T., Keysers, D., and Ney, H. 2008. Features for image retrieval: an experimental comparison. Inf. Retr. 11, 2, 77--107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Everingham, M., van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. 2010. The pascal visual object classes challenge. Int. J. Comput. Vis. 88, 2, 303--338. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Fan, J., Gao, Y., Luo, H., and Jain, R. 2008. Mining multilevel image semantics via hierarchical classification. IEEE Trans. Multimedia 10, 2, 167--187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Gehler, P. and Nowozin, S. 2009. On feature combination for multiclass object classification. In Proceedings of the 12th IEEE International Conference on Computer Vision. 221--228.Google ScholarGoogle Scholar
  15. Huiskes, M. and Lew, M. 2008. The mir flickr retrieval evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. 39--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Huiskes, M. J., Thomee, B., and Lew, M. S. 2010. New trends and ideas in visual concept detection: the mir flickr retrieval evaluation initiative. In Proceedings of the International Conference on Multimedia Information Retrieval. 527--536. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jain, A. and Vailaya, A. 1996. Image retrieval using color and shape. Pattern Recogn. 29, 8, 1233--1244.Google ScholarGoogle ScholarCross RefCross Ref
  18. Jiang, W., Chang, S.-F., and Loui, A. 2007. Context-Based concept fusion with boosted conditional random fields. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'07). Vol. 1. I--949--I--952.Google ScholarGoogle Scholar
  19. Jiang, Y.-G., Wang, J., Chang, S.-F., and Ngo, C.-W. 2009. Domain adaptive semantic diffusion for large scale context-based video annotation. In Proceedings of the 12th IEEE International Conference on Computer Vision. 1420--1427.Google ScholarGoogle Scholar
  20. Jing, F., Li, M., Zhang, H., and Zhang, B. 2004. Relevance feedback in region-based image retrieval. IEEE Trans. Circ. Syst. Video Technol. 14, 5, 672--681. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Kherfi, M. and Ziou, D. 2007. Image collection organization and its application to indexing, browsing, summarization, and semantic retrieval. IEEE Trans. Multimedia 9, 4, 893--900. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Knowles, J. and Corne, D. 2000. Approximating the nondominated front using the Pareto archived evolution strategy. Evol. Comput. 8, 2, 149--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Koskela, M., Smeaton, A., and Laaksonen, J. 2007. Measuring concept similarities in multimedia ontologies: Analysis and evaluations. IEEE Trans. Multimedia 9, 5, 912--922. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Lavrenko, V., Feng, S., and Manmatha, R. 2004. Statistical models for automatic video annotation and retrieval. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. Vol. 3, IEEE, 1044--1047.Google ScholarGoogle Scholar
  25. Levinshtein, A., Stere, A., Kutulakos, K., Fleet, D., Dickinson, S., and Siddiqi, K. 2009. Turbopixels: Fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell. 31, 12, 2290--2297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Li, F., Dai, Q., Xu, W., and Er, G. 2008. Multilabel neighborhood propagation for region-based image retrieval. IEEE Trans. Multimedia 10, 8, 1592--1604. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Li, J. and Wang, J. 2003. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Pattern Anal. Mach. Intell. 25, 9, 1075--1088. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 2, 91--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Mikolajczyk, K. and Schmid, C. 2005. A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27, 10, 1615--1630. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Natsev, A., Rastogi, R., and Shim, K. 2004. Walrus: A similarity retrieval algorithm for image databases. IEEE Trans. Knowl. Data Engin. 16, 3, 301--316. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Qi, G.-J., Hua, X.-S., Rui, Y., Tang, J., Mei, T., and Zhang, H.-J. 2007. Correlative multi-label video annotation. In Proceedings of the 15th International Conference on Multimedia. 17--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. 2007. Objects in context. In Proceedings of the 11th IEEE International Conference on Computer Vision. 1--8.Google ScholarGoogle Scholar
  33. Rasiwasia, N. and Vasconcelos, N. 2009. Holistic context modeling using semantic co-occurrences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1889--1895.Google ScholarGoogle Scholar
  34. Rui, Y., Huang, T., Ortega, M., and Mehrotra, S. 1998. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans. Circ. Syst. Video Technol. 8, 5, 644--655. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Russell, B., Torralba, A., Murphy, K., and Freeman, W. 2008. Labelme: A database and web-based tool for image annotation. Int. J. Comput. Vis. 77, 1, 157--173. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Steuer, R. 1986. Multiple Criteria Optimization. Theory, Computation, and Application. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, Wiley.Google ScholarGoogle Scholar
  37. Swain, M. and Ballard, D. 1991. Color indexing. Int. J. Comput. Vis. 7, 1, 11--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Tang, J., Hua, X.-S., Wang, M., Gu, Z., Qi, G.-J., and Wu, X. 2009a. Correlative linear neighborhood propagation for video annotation. IEEE Trans. Syst. Man Cybernet. 39, 2, 409--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Tang, J., Yan, S., Hong, R., Qi, G.-J., and Chua, T.-S. 2009b. Inferring semantic concepts from community-contributed images and noisy tags. In Proceedings of the 17th ACM International Conference on Multimedia (MM'09). 223--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Torres, R. D. S., Falcao, A. X., Goncalves, M. A., Papa, J. A. P., Zhang, B., Fan, W., and Fox, E. A. 2009. A genetic programming framework for content-based image retrieval. Pattern Recogn. 42, 2, 283--292. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Tuceryan, M. and Jain, A. 1993. Texture analysis. In Handbook of Pattern Recognition and Computer Vision, 235--276. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Vailaya, A., Jain, A., and Zhang, H. 1998. On image classification: City images vs. landscapes. Pattern Recogn. 31, 12, 1921--1935.Google ScholarGoogle ScholarCross RefCross Ref
  43. Vedaldi, A., Gulshan, V., Varma, M., and Zisserman, A. 2009. Multiple kernels for object detection. In Proceedings of the 12th IEEE International Conference on Computer Vision. 606--613.Google ScholarGoogle Scholar
  44. Vogel, J. and Schiele, B. 2007. Semantic modeling of natural scenes for content-based image retrieval. Int. J. Comput. Vis. 72, 2, 133--157. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Wang, J., Li, J., and Wiederhold, G. 2001. Simplicity: Semantics-Sensitive integrated matching for picture libraries. IEEE Trans. Pattern Anal. Mach. Intell. 23, 9, 947--963. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Zhang, J. and Ye, L. 2009. Content based image retrieval using unclean positive examples. IEEE Trans. Image Process. 18, 10, 2370--2375. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Zhang, L., Zhang, K., and Dong, X. 2011. Online sparse learning utilizing multi-feature combination for image classification. In Proceedings of the 18th IEEE International Conference on Image Processing. 197--200.Google ScholarGoogle Scholar
  48. Zhang, Q. and Izquierdo, E. 2007. Combining low-level features for semantic inference in image retrieval. EURASIP J. Adv. Signal Process. 1, 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Zhang, R. and Zhang, Z. 2007. Effective image retrieval based on hidden concept discovery in image database. IEEE Trans. Image Process. 16, 2, 562--572. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Zhu, X., Wu, X., Elmagarmid, A., Feng, Z., and Wu, L. 2005. Video data mining: Semantic indexing and event detection from the association perspective. IEEE Trans. Knowl. Data Engin. 17, 5, 665--677. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Multifeature analysis and semantic context learning for image classification

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 9, Issue 2
        May 2013
        144 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/2457450
        Issue’s Table of Contents

        Copyright © 2013 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 10 May 2013
        • Accepted: 1 August 2012
        • Revised: 1 July 2012
        • Received: 1 February 2012
        Published in tomm Volume 9, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!