Sketch-based shape retrieval

Abstract

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.

References

1. Bay, H., Tuytelaars, T., and Gool, L. J. V. 2006. SURF: Speeded up robust features. In ECCV, 404--417.
2. Bülthoff, H., and Edelman, S. 1992. Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc. National Academy of Sciences 89, 1, 60--64.
3. Candès, E. J., and Donoho, D. L. 1999. Curvelets -- a surprisingly effective nonadaptive representation for objects with edges. In Int'l. Conf. Curves and Surfaces, 105--120.
4. Canny, J. 1986. A computational approach to edge detection. IEEE TPAMI 8, 6, 679--698.
5. Chen, D.-Y., Tian, X.-P., Shen, Y.-T., and Ouhyoung, M. 2003. On visual similarity based 3d model retrieval. Comput. Graph. Forum (Proc. Eurographics) 22, 3, 223--232.
6. Chen, T., Cheng, M., Tan, P., Shamir, A., and Hu, S. 2009. Sketch2Photo: internet image montage. ACM TOG (Proc. SIGGRAPH ASIA) 28, 5, 124:1--124:10.
7. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H. S., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? ACM TOG (Proc. SIGGRAPH) 27, 3, 88:1--88:11.
8. Daras, P., and Axenopoulos, A. 2010. A 3D shape retrieval framework supporting multimodal queries. IJCV 89, 2, 229--247.
9. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., and Santella, A. 2003. Suggestive contours for conveying shape. ACM TOG (Proc. SIGGRAPH) 22, 3, 848--855.
10. Dutagaci, H., Cheung, C. P., and Godil, A. 2010. A benchmark for best view selection of 3D objects. In Proc. ACM workshop on 3D object retrieval, 45--50.
11. Eitz, M., Richter, R., Hildebrand, K., Boubekeur, T., and Alexa, M. 2011. Photosketcher: interactive sketch-based image synthesis. IEEE CG&A 31, 6, 56--66.
12. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., and Jacobs, D. 2003. A search engine for 3D models. ACM TOG 22, 1, 83--105.
13. Hou, S., and Ramani, K. 2006. Sketch-based 3D engineering part class browsing and retrieval. In Sketch-Based Interfaces and Modeling, 131--138.
14. Jacobs, C. E., Finkelstein, A., and Salesin, D. H. 1995. Fast multiresolution image querying. In Proc. SIGGRAPH 95, 277--286.
15. Lee, J., and Funkhouser, T. 2008. Sketch-based search and composition of 3D models. In Sketch-Based Interfaces and Modeling, 97--104.
16. Lee, Y., Zitnick, C., and Cohen, M. 2011. ShadowDraw: real-time user guidance for freehand drawing. ACM TOG (Proc. SIGGRAPH) 30, 4, 27:1--27:10.
17. Lloyd, S. P. 1982. Least squares quantization in PCM. IEEE Trans. Information Theory 28, 2, 129--137.
18. Löffler, J. 2000. Content-based retrieval of 3D models in distributed web databases by visual shape information. In Int'l. Conf. Information Visualization, 82--87.
19. Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV 60, 2, 91--110.
20. Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 3, 145--175.
21. Pu, J., Lou, K., and Ramani, K. 2005. A 2D sketch-based user interface for 3D CAD model retrieval. Computer-Aided Design and Applications 2, 6, 717--725.
22. Schölkopf, B., and Smola, A. 2002. Learning with Kernels. The MIT Press.
23. Secord, A., Lu, J., Finkelstein, A., Singh, M., and Nealen, A. 2011. Perceptual models of viewpoint preference. ACM TOG 30, 5, 109:1--109:12.
24. Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., and Guo, B. 2011. Discriminative sketch-based 3D model retrieval via robust shape matching. Computer Graphics Forum (Proc. Pacific Graphics) 30, 7, 2011--2020.
25. Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton Shape Benchmark. In Proc. Shape Modeling International, 167--178.
26. Shin, H., and Igarashi, T. 2007. Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. In Proc. Graphics Interface, 63--70.
27. Sivic, J., and Zisserman, A. 2003. Video Google: a text retrieval approach to object matching in videos. In ICCV, 1470--1477.
28. Squire, D., Mueller, W., Mueller, H., and Raki, J. 1999. Content-based query of image databases. In Scand. Conf. on Image Analysis, 143--149.
29. Tangelder, J., and Veltkamp, R. 2008. A survey of content based 3D shape retrieval methods. Multimedia Tools and Applications 39, 441--471.
30. Witten, I., Moffat, A., and Bell, T. 1999. Managing gigabytes: compressing and indexing documents and images. Morgan Kaufmann.
31. Yoon, S., Scherer, M., Schreck, T., and Kuijper, A. 2010. Sketch-based 3d model retrieval using diffusion tensor fields of suggestive contours. In Int'l. Conf. Multimedia, 193--200.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Shape modeling

      • Parametric curve and surface models

• Information systems

  • Information retrieval

    • Retrieval models and ranking