Abstract
The explosion of multimedia data necessitates effective and efficient ways for us to get access to our desired ones. In this article, we draw an analogy between image retrieval and text retrieval and propose a visual phrase-based approach to retrieve images containing desired objects (object-based image retrieval). The visual phrase is defined as a pair of frequently co-occurred adjacent local image patches and is constructed using data mining. We design methods on how to construct visual phrase and how to index/search images based on visual phrase. We demonstrate experiments to show our visual phrase-based approach can be very efficient and more effective than current visual word-based approach.
- Agrawal, R. and Srikant, R. 1994. Fast algorithms for mining association rules. In Proceeding of the 20th International Conference on VLDB, J. B. Bocca, M. Jarke, C., and Zaniolo, Eds. Morgan Kaufmann, Los Altos, CA. 487--499. Google Scholar
Digital Library
- Agarwal, S., Awan, A., and roth, D. 2004. Learning to detect objects in images via a sparse, part-based representation. IEEE Trans. Patt. Anal. Machine Intell. 26, 11, 1475--1490. Google Scholar
Digital Library
- Carson, C., Thomas, M., Belongie, S., and Malik, J. 2002. Blobworld: Image segmentation using expectation maximization and its application to image querying. IEEE Trans. Patt. Anal. Machine Intell. 24, 8, 1026--1038. Google Scholar
Digital Library
- Ceglar, A. and Roddick, J. F. 2006. Association mining. ACM Comput. Surv. 38, 2. Google Scholar
Digital Library
- Cox, I. J., Miller, M. L., Minka, T. P., Papathomas, T. V., and Yianilos, P. N. 2000. The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Trans. Image Process. 9, 1, 20--37. Google Scholar
Digital Library
- Datta, R., Joshi, D., Li, J., and Wang, J. 2006. Image retrieval: ideas, influences, and trends of the new age. Penn Sate University Tech. rep. CSE 06--009.Google Scholar
- Enser, P. G. B. and Sandom, C. J. 2003. Towards a comprehensive survey of the semantic gap in visual image retrieval. In Proceedings of the 2nd International Conference on Image and video retrieval. E. M. Bakker, T. S. Huang, M. S. Lew, N. Sebe, and X. Zhou, Eds. Springer-Verlag, London, UK. 291--299 Google Scholar
Digital Library
- Fergus, R., Li, F. --F., Perona, P., and Zisserman, A. 2005. Learning object categories from Google's image search. In Proceedings of the 10th IEEE International Conference on Computer Vision. IEEE Computer Society, Vol. 2, 1816--1823. Google Scholar
Digital Library
- FlickneR, M., Sawheny, H., Niblack, W., Ashley, J., Qian Huang Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. 1995. Query by image and video content: The QBIC system. IEEE Comput. 28, 9, 23--32. Google Scholar
Digital Library
- Garcya-Perez, D., Mosquera, A., Berretti, S., and Del Bimbo, A. 2006. Object-based image retrieval using active nets. In Proceedings of the 18th International Conference on Pattern Recognition, Vol. 4, 750--753. Google Scholar
Digital Library
- Goethals, B. 2003. Survey on frequent pattern mining. Helsinki Institute for Information Technology tech. rep.Google Scholar
- Grabner, M., Grabner, H., and Bischof, H. 2006. Fast Approximated SIFT. In Proceedings of 7th Asian Conference on Computer Vision, P. J. Narayanan, S. K. Nayar, H.--Y. Shum, Eds. Springer-Verlag, London, UK. 918--927. Google Scholar
Digital Library
- Hammouda, K. M. and Kamel, M. S. 2004. Efficient phrase-based document indexing for web document clustering. IEEE Trans. Knowl. Data Engin. 16, 10, 1279--1296. Google Scholar
Digital Library
- Hoiem, R., Sukthankar, H., Schneiderman, H., and Huston, L. 2004. Object-based image retrieval using the statistical structure of images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. Vol. 2, 490--497. Google Scholar
Digital Library
- Jansen, M. B. J., Spink, A., and Saracevic, T. 2000. Real life, real users, and real needs: A study and analysis of user queries on the web. Inform. Process. Manag. Int. J. 36, 2, 207--227. Google Scholar
Digital Library
- Jin, R, Chai, J. Y., and Si, L. 2004. Effective automatic image annotation via a coherent language model and active learning. In Proceedings of 12th ACM International Conference on Multimedia. ACM Press, New York, 892--899. Google Scholar
Digital Library
- Jing, F., Li, M., Zhang, H.-J., and Zhang, B. 2004, An efficient and effective region-based image retrieval framework. IEEE Trans. Image Process. 13, 5, 699--709. Google Scholar
Digital Library
- Jurie, F. and Triggs, B. 2005. Creating efficient codebooks for visual recognition. In Proceedings of International Conference on Computer Vision. IEEE Computer Society, Washington, DC, Vol. 1, 604--610. Google Scholar
Digital Library
- Ke, Y., Sukthankar, R., and Huston, L. 2004. Efficient near-duplicate detection and sub-image retrieval. In Proceedings of the 12th ACM International Conference on Multimedia. ACM Press, New York, NY. 869--876. Google Scholar
Digital Library
- Kherfi, M. L., Ziou, D., and Bernardi, A. 2004. Image retrieval from the World Wide Web: Issues, techniques, and systems. ACM Comput. Surv. 36, 1, 35--67. Google Scholar
Digital Library
- Lew, M. S., Sebe, N., Djeraba, C., and Jain, R. 2006. Content-based multimedia information retrieval: State of the art and challenges. ACM Trans. Multimed. Comput. Comm. Appl. 2, 1, 1--19. Google Scholar
Digital Library
- Li, J. and Wang, J. 2003. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Patt. Anal. Mach. Intell. 25, 9, 1075--1088. Google Scholar
Digital Library
- Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110. Google Scholar
Digital Library
- Luo, J. and Nascimento, M. 2003. Content based subimage retrieval via hierarchical tree matching. In Proceedings of the 1st ACM International Workshop on Multimedia Databases. ACM Press, New York. 63--69. Google Scholar
Digital Library
- Marszalek, M. and Schmid, C. 2006. Spatial weighting for bag-of-features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Press, Washington, DC, Vol. 2, 2118--2125. Google Scholar
Digital Library
- Mikolajczyk, K. and Schmid, C. 2005. A performance evaluation of local descriptors. IEEE Trans. Patt. Anal. Mach. Intell. 27, 10, 1615--1630. Google Scholar
Digital Library
- Pantofaru, C., Dorko, G., Schmid, C., and Hebert, M. 2006. Combining regions and patches for object class localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop. IEEE Computer Society Press, Washington, DC. 23--30 Google Scholar
Digital Library
- Papageorgious, C., Oren, M., and Poggio, T. 1998. A general framework for object detection. In Proceedings of the 6th International Conference on Computer Vision. IEEE Computer Society, Washington, DC. 555--562. Google Scholar
Digital Library
- Pentland, A., Picard, R. W., and Sclaroff, S. 1996. Photobook: Content-based manipulation of image databases. Int. J. Comput. Vision 18, 3, 233--254. Google Scholar
Digital Library
- Rui, Y., Huang, T. S., and Chang, S.--F. 1999. Image retrieval: Current techniques, promising directions and open Issues. J. Visual Comm. Image Represent. 10, 1, 39--62.Google Scholar
Digital Library
- Sebe, N., Lew, M., and Huijsmans, D. 1999. Multi-scale subimage search. In Proceedings of the 7th ACM International Conference on Multimedia. ACM Press, New York. 79--82. Google Scholar
Digital Library
- Sivic, J. and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the 9th IEEE International Conference on Computer Vision. IEEE Computer Society, Washington, DC. Vol. 2, 1470--1477. Google Scholar
Digital Library
- Smeulders, A., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of early years. IEEE Trans. Patt. Anal. Mach. Intell. 22, 12, 1349--1380. Google Scholar
Digital Library
- Smith, J. R. and Chang, S.--F. 1996. Querying by color regions using the VisualSEEK content-based visual query system. In Intell. Multimed. Inform. Retri. Mark T. Maybury, Ed. AAAI Press, 23--42. Google Scholar
Digital Library
- Smyth, W. F., Lam, C. P., Chen, X., and Maxville, V. 2003. Heuristics for image retrieval using spatial configurations. In Proceedings of the 7th Digital Image Computing: Techniques and Applications. Sydney. 909--918.Google Scholar
- Squire, D., Muller, W., Muller, H., and Pun, T. 2000. Content-based visual query of image databases: Inspirations from text retrieval. Patt. Recogn. Lett. 21, 13--14, 1193--1198. Google Scholar
Digital Library
- Sun, Z, Bebis, G., and Miller, R. 2006. On-road vehicle detection: A review. IEEE Trans. Patt. Anal. Mach. Intell. 28, 5, 694--711. Google Scholar
Digital Library
- Sung, K.--K. and Poggio, T. 1998, Example-based learning for view-based face detection. IEEE Trans. Patt. Anal. Mach. Intell. 20, 1, 39--51. Google Scholar
Digital Library
- Veltkamp, R. C. and Tanase, M. 2000. Content-based image retrieval systems: A survey. Utrecht University technical rep. UU-CS-2000-34.Google Scholar
- Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. 511--518.Google Scholar
- Wang, G., Zhang, Y., and Li, F.-F. 2006. Using dependent regions for object categorization in generative framework. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Press, Washington, DC, Vol. 2, 1597--1604. Google Scholar
Digital Library
- Williams, H. E., Zobel, J., and BAHLE, D. 2004. Fast phrase querying with combined indexes. ACM Trans. Inform. Syst. 22, 4, 573--594. Google Scholar
Digital Library
- Winn, J., Criminisi, A., and Minka, T. 2005. Object categorization by learned universal visual dictionary. In Proceedings of 10th IEEE Conference on Computer Vision. IEEE Computer Society, Washington DC, Vol. 2, 1800--1807. Google Scholar
Digital Library
- Witter, I. H., Moffat, A., and Bell, T. C. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, Los Altos, CA. Google Scholar
Digital Library
- Yan, X., Yu, P. S., and Han, J. 2005. Graph indexing based on discriminative frequent structure analysis. ACM Trans. Datab. Syst. 30, 4, 960--993. Google Scholar
Digital Library
- Yang, M.--H., Kriegman, D. J., and Ahuja, N. 2002. Detecting faces in images: A survey. IEEE Trans. Patt. Anal. Mach. Intell. 24, 1, 34--58. Google Scholar
Digital Library
- Yeh, T., Tollmar, K., and Darrell, T. 2004. Searching the Web with mobile images for location recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC. Vol. 2, 76--81. Google Scholar
Digital Library
- Zheng, Q.-F, Wang, W.-Q, and Gao, W. 2006. Effective and efficient object-based images retrieval using visual phrases. In Proceedings of the 14th ACM International Conference on Multimedia. ACM Press, New York. 77--80. Google Scholar
Digital Library
Index Terms
Constructing visual phrases for effective and efficient object-based image retrieval
Recommendations
Effective and efficient object-based image retrieval using visual phrases
MM '06: Proceedings of the 14th ACM international conference on MultimediaIn this paper, we draw an analogy between image retrieval and text retrieval and propose a visual phrase-based approach to retrieve images containing desired objects. The visual phrase is defined as a pair of adjacent local image patches and is ...
Efficient Indexing for Mobile Image Retrieval
ICDMW '11: Proceedings of the 2011 IEEE 11th International Conference on Data Mining WorkshopsIn this work, we present a novel indexing method, which contains visual phrase quantization, two-dimensional inverted index and Approximate RANSAC (ARANSAC), for mobile image retrieval. First of all, visual phrase quantization is proposed by mapping the ...
Toward a higher-level visual representation for object-based image retrieval
We propose a higher-level visual representation, visual synset, for object-based image retrieval beyond visual appearances. The proposed visual representation improves the traditional part-based bag-of-words image representation, in two aspects. First, ...






Comments