skip to main content
research-article

Optimization of interactive visual-similarity-based search

Published:11 February 2008Publication History
Skip Abstract Section

Abstract

At one end of the spectrum, research in interactive content-based retrieval concentrates on machine learning methods for effective use of relevance feedback. On the other end, the information visualization community focuses on effective methods for conveying information to the user. What is lacking is research considering the information visualization and interactive retrieval as truly integrated parts of one content-based search system. In such an integrated system, there are many degrees of freedom like the similarity function, the number of images to display, the image size, different visualization modes, and possible feedback modes. To base the optimal values for all of those on user studies is unfeasible. We therefore develop search scenarios in which tasks and user actions are simulated. From there, the proposed scheme is optimized based on objective constraints and evaluation criteria. In such a manner, the degrees of freedom are reduced and the remaining degrees can be evaluated in user studies. In this article, we present a system that integrates advanced similarity based visualization with active learning. We have performed extensive experimentation on interactive category search with different image collections. The results using the proposed simulation scheme show that indeed the use of advanced visualization and active learning pays off in all of these datasets.

References

  1. Basseville, M. 1989. Distance measures for signal processing and pattern recognition. Sig. Process 18, 349--369. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bederson, B. 2001. PhotoMesa: A zoomable image browser using quantum treemaps and bubblemaps. In Proceedings of the ACM Symposium on User Interface Software and Technology, CHI Letters. Vol. 3. 71--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Chang, C. and Lin, C. 2001. Libsvm: A library for support vector machines. Software available at http://www.csie.ntu.edu. tw/~jlin/libsvm.Google ScholarGoogle Scholar
  4. Chen, M., Christel, M., Hauptmann, A., and Wactlar, H. 2005. Putting active learning into multimedia applications: Dynamic definition and refinement of concept classifiers. In MULTIMEDIA '05: Proceedings of the 13th Annual ACM International Conference on Multimedia. ACM, New York, 902--911. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Chen, Y., Zhou, X., and Huang, T. 2001. One-class SVM for learning in image retrieval. In Proceedings of the International Conference on Image Processing. Vol. 1. 34--37.Google ScholarGoogle Scholar
  6. Dubes, R. 1987. How many clusters are best?---An experiment. Pattern Recog. 20, 6, 645--663. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Geusebroek, J., Burghouts, G., and Smeulders, A. 2005. The Amsterdam library of object images. Int. J. Comput. Vision 61, 1, 103--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Geusebroek, J. M. and Smeulders, A. W. M. 2005. A six-stimulus theory for stochastic texture. Int. J. Comput. Vision 62, 1/2, 7--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Gosselin, P. and Cord, M. 2004. RETIN AL: An active learning strategy for image category retrieval. In Proceedings of the International Conference on Image Processing.Google ScholarGoogle Scholar
  10. Guo, G., Jain, A., Ma, W., and Zhang, H. 2001. Learning similarity measure for natural image retrieval with relevance feedback. In Proceedings of the International Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  11. Heesch, D. and Ruger, S. 2004. Three interfaces for content-based access to image collections. In Proceedings of the International Conference on Image and Video Retrieval. 491--499.Google ScholarGoogle Scholar
  12. Ivory, Y. and Hearst, M. 2001. The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33, 4, 470--516. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Keim, D. 2002. Information visualization and visual data mining. IEEE Trans. Visual. Comput. Graph. 7, 1, 100--107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Manevitz, L. and Yousef, M. 2004. One-class SVMs for document classification. J. Mach. Learn. Res. 2, 139--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Moghaddam, B., Tian, Q., Lesh, N., Shen, C., and Huang, T. 2004. Visualization and user-modeling for browsing personal photo libraries. Int. J. Comput. Vision 56, 1-2, 109--130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Naphade, M., Basu, S., Smith, J., Lin, C., and Tseng, B. 2002. Modeling semantic concepts to support query by keywords in video. In Proceedings of the IEEE International Conference on Image Processing. IEEE Computer Society Press, Los Alamitos, CA.Google ScholarGoogle Scholar
  17. Nguyen, G. and Worring, M. 2005. Similarity based visualization of image collections. In Proceedings of the 7th International Workshop on Audio-Visual Content and Information Visualization in Digital Libraries.Google ScholarGoogle Scholar
  18. Nguyen, H. and Smeulders, A. 2004. Everything gets better all the time, apart from the amount of data. In Proceedings of the International Conference on Image and Video Retrieval.Google ScholarGoogle Scholar
  19. Rodden, K., Basalaj, W., Sinclair, D., and Wood, K. 2001. Does organisation by similarity assist image browsing? In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, 190--197. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Rubner, Y. 1999. Perceptual metrics for database navigation. Ph.D. dissertation, Stanford University, Stanford, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Rubner, Y., Puzicha, J., Tomasi, C., and Buhmann, J. 2001. Empirical evaluation of dissimilarity measures for color and texture. Comput. Vision Image Understand. J. 84, 1, 25--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Rui, Y., Huang, T., and Mehrotra, S. 1998. Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans. Curcuits Video Tech. 8, 5, 644--655. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Rumelhart, D. and Zipser, D. 1985. Feature discovery by competitive learning. Cogn. Sci. 9, 75--112.Google ScholarGoogle ScholarCross RefCross Ref
  24. Santini, S., Gupta, A., and Jain, R. 2001. Emergent semantics through interaction in image databases. IEEE Trans. Knowl. Data Eng. 13, 3, 337--351. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Shneiderman, B. 1996. The eyes have it: A task by data type taxonomy for information visualizations. In Proceedings of the IEEE Conference on Visual Languages. IEEE Computer Society Press, Los Alamitos, CA, 336--343. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Smeulders, A., Worring, M., Santini, S., Gupta, A., and Jain, R. 2000. Content-based image retrieval at the end of early years. IEEE Trans. Pattern Anal. Mach. Intel. 22, 12, 1349--1380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Snoek, C., Worring, M., van Gemert, J., Geusebroek, J., Koelma, D., Nguyen, G., de Rooij, O., and Seinstra, F. 2005. Mediamill: Exploring news video archives based on learned semantics. In Proceedings of ACM Multimedia (Best Technical Demonstration Award). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Swain, M. and Ballard, D. 1991. Color indexing. Int. J. Comput. Vision 7, 1, 11--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Tong, S. and Chang, E. 2001. Support vector machine active learning for image retrieval. In Proceedings of the ACM Internaltional Conference on Multimedia. Vol. 9. ACM, New York, 107--118. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Voorhees, E. and Harman, D. 2001. Trec-10 proceedings appendix on common evaluation measures. http://trec.nist.gov/pubs/trec10/appendices/measures.pdf.Google ScholarGoogle Scholar
  31. Zhang, L., Lin, F., and Zhang, B. 2001. Support vector machine learning for image retrieval. In Proceedings of the International Conference on Image Processing. 721--724.Google ScholarGoogle Scholar
  32. Zhou, X. and Huang, T. 2003. Relevance feedback in image retrieval: A comprehensive overview. Multimed. Syst. 8, 6, 536--544.Google ScholarGoogle ScholarCross RefCross Ref
  33. Zhu, M. 2004. Recall, precision and average precision. http://www.stats.uwaterloo.ca/stats_navigation/techreports/04WorkingPapers/2004-09.pdf.Google ScholarGoogle Scholar

Index Terms

  1. Optimization of interactive visual-similarity-based search

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 4, Issue 1
      January 2008
      197 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/1324287
      Issue’s Table of Contents

      Copyright © 2008 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 11 February 2008
      • Accepted: 1 November 2006
      • Revised: 1 August 2006
      • Received: 1 December 2005
      Published in tomm Volume 4, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!