Context-based search for 3D models

Abstract

Large corpora of 3D models, such as Google 3D Warehouse, are now becoming available on the web. It is possible to search these databases using a keyword search. This makes it possible for designers to easily include existing content into new scenes. In this paper, we describe a method for context-based search of 3D scenes. We first downloaded a large set of scene graphs from Google 3D Warehouse. These scene graphs were segmented into individual objects. We also extracted tags from the names of the models. Given the object shape, tags, and spatial relationship between pairs of objects, we can predict the strength of a relationship between a candidate model and an existing object in the scene. Using this function, we can perform context-based queries. The user specifies a region in the scene they are modeling using a 3D bounding box, and the system returns a list of related objects. We show that context-based queries perform better than keyword queries alone, and that without any keywords our algorithm still returns a relevant set of models.

References

1. Chen, D., Tian, X., Shen, Y., and Ouhyoung, M. 2003. On visual similarity based 3D model retrieval. In Computer graphics forum, vol. 22, Amsterdam: North Holland, 1982-, 223--232.
2. Fellbaum, C., et al. 1998. WordNet: An electronic lexical database. MIT press Cambridge, MA.
3. Funkhouser, T., and Shilane, P. 2006. Partial matching of 3D shapes with priority-driven search. In Proceedings of the fourth Eurographics symposium on Geometry processing, Eurographics Association, 142.
4. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., and Jacobs, D. 2003. A search engine for 3D models. ACM Transactions on Graphics 22, 1, 83--105.
5. Funkhouser, T., Kazhdan, M., Shilane, P., Min, P., Kiefer, W., Tal, A., Rusinkiewicz, S., and Dobkin, D. 2004. Modeling by example. In ACM SIGGRAPH, ACM, 663.
6. Galleguillos, C., Rabinovich, A., and Belongie, S. 2008. Object categorization using co-occurrence, location and appearance. In IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, 1--8.
7. Goldfeder, C., and Allen, P. 2008. Autotagging to improve text search for 3d models. In JCDL '08: Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries, ACM, New York, NY, USA, 355--358.
8. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (SIGGRAPH 2007) 26, 3.
9. He, X., Zemel, R., and Carreira-Perpinán, M. 2004. Multiscale conditional random fields for image labeling. CVPR, 695--702.
10. Malisiewicz, T., and Efros, A. A. 2009. Beyond categories: The visual memex model for reasoning about object relationships. In NIPS.
11. Min, P., Kazhdan, M., and Funkhouser, T. 2004. A comparison of text and shape matching for retrieval of online 3D models. Research and Advanced Technology for Digital Libraries, 209--220.
12. Novotni, M., and Klein, R. 2003. 3D Zernike descriptors for content based shape retrieval. In Solid modeling and applications, ACM, 225.
13. Papadakis, P., Pratikakis, I., Trafalis, T., Theoharis, T., and Perantonis, S. 2008. Relevance Feedback in Content-based 3D Object Retrieval: A Comparative Study. Computer-Aided Design and Applications Journal 5, 5.
14. Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. 2007. Objects in context. IEEE 11th International Conference on Computer Vision, 1--8.
15. Russell, B., Torralba, A., Murphy, K., and Freeman, W. 2008. LabelMe: a database and web-based tool for image annotation. International Journal of Computer Vision 77, 1, 157--173.
16. Salton, G., Wong, A., and Yang, C. S. 1975. A vector space model for automatic indexing. Commun. ACM 18, 11, 613--620.
17. Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The princeton shape benchmark. In Shape Modeling International.
18. Strat, T. M., and Fischler, M. A. 1991. Context-based vision: Recognizing objects using information from both 2d and 3d imagery. IEEE PAMI 13, 1050--1065.
19. Torralba, A., 2010. The context challenge. http://web.mit.edu/torralba/www/carsAndFacesInContext.html.

Index Terms

• Computing methodologies

  • Computer graphics

    • Shape modeling