skip to main content
research-article

Constructing visual phrases for effective and efficient object-based image retrieval

Authors Info & Claims
Published:30 October 2008Publication History
Skip Abstract Section

Abstract

The explosion of multimedia data necessitates effective and efficient ways for us to get access to our desired ones. In this article, we draw an analogy between image retrieval and text retrieval and propose a visual phrase-based approach to retrieve images containing desired objects (object-based image retrieval). The visual phrase is defined as a pair of frequently co-occurred adjacent local image patches and is constructed using data mining. We design methods on how to construct visual phrase and how to index/search images based on visual phrase. We demonstrate experiments to show our visual phrase-based approach can be very efficient and more effective than current visual word-based approach.

References

  1. Agrawal, R. and Srikant, R. 1994. Fast algorithms for mining association rules. In Proceeding of the 20th International Conference on VLDB, J. B. Bocca, M. Jarke, C., and Zaniolo, Eds. Morgan Kaufmann, Los Altos, CA. 487--499. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Agarwal, S., Awan, A., and roth, D. 2004. Learning to detect objects in images via a sparse, part-based representation. IEEE Trans. Patt. Anal. Machine Intell. 26, 11, 1475--1490. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Carson, C., Thomas, M., Belongie, S., and Malik, J. 2002. Blobworld: Image segmentation using expectation maximization and its application to image querying. IEEE Trans. Patt. Anal. Machine Intell. 24, 8, 1026--1038. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ceglar, A. and Roddick, J. F. 2006. Association mining. ACM Comput. Surv. 38, 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Cox, I. J., Miller, M. L., Minka, T. P., Papathomas, T. V., and Yianilos, P. N. 2000. The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Trans. Image Process. 9, 1, 20--37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Datta, R., Joshi, D., Li, J., and Wang, J. 2006. Image retrieval: ideas, influences, and trends of the new age. Penn Sate University Tech. rep. CSE 06--009.Google ScholarGoogle Scholar
  7. Enser, P. G. B. and Sandom, C. J. 2003. Towards a comprehensive survey of the semantic gap in visual image retrieval. In Proceedings of the 2nd International Conference on Image and video retrieval. E. M. Bakker, T. S. Huang, M. S. Lew, N. Sebe, and X. Zhou, Eds. Springer-Verlag, London, UK. 291--299 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Fergus, R., Li, F. --F., Perona, P., and Zisserman, A. 2005. Learning object categories from Google's image search. In Proceedings of the 10th IEEE International Conference on Computer Vision. IEEE Computer Society, Vol. 2, 1816--1823. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. FlickneR, M., Sawheny, H., Niblack, W., Ashley, J., Qian Huang Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Comput. 28, 9, 23--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Garcya-Perez, D., Mosquera, A., Berretti, S., and Del Bimbo, A. 2006. Object-based image retrieval using active nets. In Proceedings of the 18th International Conference on Pattern Recognition, Vol. 4, 750--753. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Goethals, B. 2003. Survey on frequent pattern mining. Helsinki Institute for Information Technology tech. rep.Google ScholarGoogle Scholar
  12. Grabner, M., Grabner, H., and Bischof, H. 2006. Fast Approximated SIFT. In Proceedings of 7th Asian Conference on Computer Vision, P. J. Narayanan, S. K. Nayar, H.--Y. Shum, Eds. Springer-Verlag, London, UK. 918--927. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Hammouda, K. M. and Kamel, M. S. 2004. Efficient phrase-based document indexing for web document clustering. IEEE Trans. Knowl. Data Engin. 16, 10, 1279--1296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hoiem, R., Sukthankar, H., Schneiderman, H., and Huston, L. 2004. Object-based image retrieval using the statistical structure of images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. Vol. 2, 490--497. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jansen, M. B. J., Spink, A., and Saracevic, T. 2000. Real life, real users, and real needs: A study and analysis of user queries on the web. Inform. Process. Manag. Int. J. 36, 2, 207--227. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jin, R, Chai, J. Y., and Si, L. 2004. Effective automatic image annotation via a coherent language model and active learning. In Proceedings of 12th ACM International Conference on Multimedia. ACM Press, New York, 892--899. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jing, F., Li, M., Zhang, H.-J., and Zhang, B. 2004, An efficient and effective region-based image retrieval framework. IEEE Trans. Image Process. 13, 5, 699--709. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jurie, F. and Triggs, B. 2005. Creating efficient codebooks for visual recognition. In Proceedings of International Conference on Computer Vision. IEEE Computer Society, Washington, DC, Vol. 1, 604--610. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Ke, Y., Sukthankar, R., and Huston, L. 2004. Efficient near-duplicate detection and sub-image retrieval. In Proceedings of the 12th ACM International Conference on Multimedia. ACM Press, New York, NY. 869--876. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Kherfi, M. L., Ziou, D., and Bernardi, A. 2004. Image retrieval from the World Wide Web: Issues, techniques, and systems. ACM Comput. Surv. 36, 1, 35--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimed. Comput. Comm. Appl. 2, 1, 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Li, J. and Wang, J. 2003. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Patt. Anal. Mach. Intell. 25, 9, 1075--1088. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Luo, J. and Nascimento, M. 2003. Content based subimage retrieval via hierarchical tree matching. In Proceedings of the 1st ACM International Workshop on Multimedia Databases. ACM Press, New York. 63--69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Marszalek, M. and Schmid, C. 2006. Spatial weighting for bag-of-features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Press, Washington, DC, Vol. 2, 2118--2125. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Mikolajczyk, K. and Schmid, C. 2005. A performance evaluation of local descriptors. IEEE Trans. Patt. Anal. Mach. Intell. 27, 10, 1615--1630. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Pantofaru, C., Dorko, G., Schmid, C., and Hebert, M. 2006. Combining regions and patches for object class localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop. IEEE Computer Society Press, Washington, DC. 23--30 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Papageorgious, C., Oren, M., and Poggio, T. 1998. A general framework for object detection. In Proceedings of the 6th International Conference on Computer Vision. IEEE Computer Society, Washington, DC. 555--562. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Pentland, A., Picard, R. W., and Sclaroff, S. 1996. Photobook: Content-based manipulation of image databases. Int. J. Comput. Vision 18, 3, 233--254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Rui, Y., Huang, T. S., and Chang, S.--F. 1999. Image retrieval: Current techniques, promising directions and open Issues. J. Visual Comm. Image Represent. 10, 1, 39--62.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Sebe, N., Lew, M., and Huijsmans, D. 1999. Multi-scale subimage search. In Proceedings of the 7th ACM International Conference on Multimedia. ACM Press, New York. 79--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Sivic, J. and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the 9th IEEE International Conference on Computer Vision. IEEE Computer Society, Washington, DC. Vol. 2, 1470--1477. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Smeulders, A., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 12, 1349--1380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Smith, J. R. and Chang, S.--F. 1996. Querying by color regions using the VisualSEEK content-based visual query system. In Intell. Multimed. Inform. Retri. Mark T. Maybury, Ed. AAAI Press, 23--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Smyth, W. F., Lam, C. P., Chen, X., and Maxville, V. 2003. Heuristics for image retrieval using spatial configurations. In Proceedings of the 7th Digital Image Computing: Techniques and Applications. Sydney. 909--918.Google ScholarGoogle Scholar
  36. Squire, D., Muller, W., Muller, H., and Pun, T. 2000. Content-based visual query of image databases: Inspirations from text retrieval. Patt. Recogn. Lett. 21, 13--14, 1193--1198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Sun, Z, Bebis, G., and Miller, R. 2006. On-road vehicle detection: A review. IEEE Trans. Patt. Anal. Mach. Intell. 28, 5, 694--711. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Sung, K.--K. and Poggio, T. 1998, Example-based learning for view-based face detection. IEEE Trans. Patt. Anal. Mach. Intell. 20, 1, 39--51. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Veltkamp, R. C. and Tanase, M. 2000. Content-based image retrieval systems: A survey. Utrecht University technical rep. UU-CS-2000-34.Google ScholarGoogle Scholar
  40. Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. 511--518.Google ScholarGoogle Scholar
  41. Wang, G., Zhang, Y., and Li, F.-F. 2006. Using dependent regions for object categorization in generative framework. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Press, Washington, DC, Vol. 2, 1597--1604. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Williams, H. E., Zobel, J., and BAHLE, D. 2004. Fast phrase querying with combined indexes. ACM Trans. Inform. Syst. 22, 4, 573--594. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Winn, J., Criminisi, A., and Minka, T. 2005. Object categorization by learned universal visual dictionary. In Proceedings of 10th IEEE Conference on Computer Vision. IEEE Computer Society, Washington DC, Vol. 2, 1800--1807. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Witter, I. H., Moffat, A., and Bell, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, Los Altos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Yan, X., Yu, P. S., and Han, J. 2005. Graph indexing based on discriminative frequent structure analysis. ACM Trans. Datab. Syst. 30, 4, 960--993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Yang, M.--H., Kriegman, D. J., and Ahuja, N. 2002. Detecting faces in images: A survey. IEEE Trans. Patt. Anal. Mach. Intell. 24, 1, 34--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Yeh, T., Tollmar, K., and Darrell, T. 2004. Searching the Web with mobile images for location recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. Vol. 2, 76--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Zheng, Q.-F, Wang, W.-Q, and Gao, W. 2006. Effective and efficient object-based images retrieval using visual phrases. In Proceedings of the 14th ACM International Conference on Multimedia. ACM Press, New York. 77--80. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Constructing visual phrases for effective and efficient object-based image retrieval

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!