Abstract
In a Content-based Image Retrieval (CBIR) System, the task is to retrieve similar images from a large database given a query image. The usual procedure is to extract some useful features from the query image and retrieve images that have a similar set of features. For this purpose, a suitable similarity measure is chosen, and images with high similarity scores are retrieved. Naturally, the choice of these features play a very important role in the success of this system, and high-level features are required to reduce the “semantic gap.”
In this article, we propose to use features derived from pre-trained network models from a deep-learning convolution network trained for a large image classification problem. This approach appears to produce vastly superior results for a variety of databases, and it outperforms many contemporary CBIR systems. We analyse the retrieval time of the method and also propose a pre-clustering of the database based on the above-mentioned features, which yields comparable results in a much shorter time in most of the cases.
- Stanford University. [n.d.]. CS231n: Convolutional Neural Networks for Visual Recognition. Retrieved from http://cs231n.stanford.edu/.Google Scholar
- Khawaja Ahmed, Shahida, and Muhammad Iqbal. 2018. Content-based image retrieval using image features information fusion. Info. Fusion 51 (Nov. 2018), 76–99. https://doi.org/10.1016/j.inffus.2018.11.004Google Scholar
- K. T. Ahmed, S. A. H. Naqvi, A. Rehman, and T. Saba. 2019. Convolution, approximation and spatial information based object and color signatures for content based image retrieval. In Proceedings of the International Conference on Computer and Information Sciences (ICCIS’19). 1–6. https://doi.org/10.1109/ICCISci.2019.8716437Google Scholar
- S. Aksoy and R. M. Haralick. 2000. Probabilistic vs. geometric similarity measures for image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’00), Vol. 2. 357–362. https://doi.org/10.1109/CVPR.2000.854847Google Scholar
- Mutasem K. Alsmadi. 2020. Content-based image retrieval using color, shape and texture descriptors and features. Arab. J. Sci. Eng. 45, 4 (2020), 3317–3330. https://doi.org/10.1007/s13369-020-04384-yGoogle Scholar
- Rehan Ashraf, Mudassar Ahmed, Sohail Jabbar, Shehzad Khalid, Awais Ahmad, Sadia Din, and Gwangil Jeon. 2018. Content based image retrieval by using color descriptor and discrete wavelet transform. J. Med. Syst. 17, 6 (Mar. 2018), 3552–3580. https://doi.org/10.1007/s10916-017-0880-7Google Scholar
- Rehan Ashraf, Khalid Bashir, Aun Irtaza, and Muhammad Mahmood. 2015. Content based image retrieval using embedded neural networks with bandletized regions. Entropy 17 (June 2015), 3552–3580. https://doi.org/10.3390/e17063552Google Scholar
- Mohamed Uvaze Ahamed Ayoobkhan, C. Eswaran, and Kannan Ramakrishnan. 2017. CBIR system based on prediction errors. J. Info. Sci. Eng. 33 (Mar. 2017), 347–365. https://doi.org/10.1688/JISE.2017.33.2.5Google Scholar
- Smarajit Bose, Amita Pal, Disha Chakrabarti, and Taranga Mukherjee. 2017. Improved content-based image retrieval via discriminant analysis. Int. J. Mach. Learn. Comput. 7 (June 2017), 44–48. https://doi.org/10.18178/ijmlc.2017.7.3.618Google Scholar
- Alfredo Canziani, Adam Paszke, and Eugenio Culurciello. 2016. An analysis of deep neural network models for practical applications. Retrieved from https://arXiv:1605.07678.Google Scholar
- François Chollet. 2016. Xception: Deep learning with depthwise separable convolutions. Retrieved from https://arXiv:1610.02357.Google Scholar
- Francois Chollet. 2017. Deep Learning with Python (1st ed.). Manning Publications, Greenwich, CT. Google Scholar
Digital Library
- J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 248–255. https://doi.org/10.1109/CVPR.2009.5206848Google Scholar
Cross Ref
- Meenakshi Garg and Gaurav Dhiman. 2021. A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants. Neural Comput. Appl. 33 (2021), 1311–1328. https://doi.org/10.1007/s00521-020-05017-zGoogle Scholar
Digital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. Retrieved from https://arXiv:1512.03385.Google Scholar
- Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. 2016. Densely connected convolutional networks. Retrieved from https://arXiv:1608.06993.Google Scholar
- Jing Huang. 1998. Color-spatial Image Indexing and Applications. Ph.D. Dissertation. Ithaca, NY. Advisor(s) Zabih, Ramin. Google Scholar
- D. Mansoor Hussain and D. Surendran. 2020. The efficient fast-response content-based image retrieval using spark and MapReduce model framework. J. Ambient Intell. Human. Comput. 12, 3 (2020), 4049–4056. https://doi.org/10.1007/s12652-020-01775-9Google Scholar
- Safia Jabeen, Zahid Mehmood, Toqeer Mahmood, Tanzila Saba, Amjad Rehman, and Muhammad Mahmood. 2018. An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-words model. PLoS ONE 13, 4 (Mar. 2018). https://doi.org/10.1371/journal.pone.0194526Google Scholar
- Aamir Khan and Anand Jalal. 2021. A visual saliency-based approach for content-based image retrieval. International J. Cogn. Info. Natural Intell. 15 (Jan. 2021), 1–15. https://doi.org/10.4018/IJCINI.2021010101Google Scholar
- Suman Khokhar and Satya Verma. 2017. Content based image retrieval with multi-feature classification by back-propagation neural network. Int. J. Comput. Appl. Technol. Res. 6 (July 2017), 278–284. https://doi.org/10.7753/IJCATR0607.1002Google Scholar
- Harald Kosch. 2003. Distributed Multimedia Database Technologies Supported by MPEG-7 and MPEG-21. CRC Press. Google Scholar
- Li Fei-Fei, R. Fergus, and P. Perona. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop. 178–178. https://doi.org/10.1109/CVPR.2004.383Google Scholar
- Mr. Yogen Mahesh Lohite and Prof. Sushant J. Pawar. 2017. A novel method for content based image retrieval using local features and SVM classifier. Int. Res. J. Eng. Technol. 4, 7 (2017).Google Scholar
- B. S. Manjunath, J. Ohm, V. V. Vasudevan, and A. Yamada. 2001. Color and texture descriptors. IEEE Trans. Circ. Syst. Video Technol. 11, 6 (June 2001), 703–715. https://doi.org/10.1109/76.927424Google Scholar
Digital Library
- Pedro Marcelino. [n.d.]. Transfer learning from pre-trained models. Retrieved from https://towardsdatascience.com/transfer-learning-from-pre-trained-models- f2393f124751.Google Scholar
- Zahid Mehmood, Toqeer Mahmood, and Muhammad Arshad Javid. 2018. Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine. Appl. Intell. 48, 1 (Jan. 2018), 166–181. https://doi.org/10.1007/s10489-017-0957-5Google Scholar
- Wayne Niblack, Ronald Barber, William Equitz, Myron Flickner, Eduardo Glasman, Dragutin Petkovic, Peter Yanker, Christos Faloutsos, and Gabriel Taubin. 1993. The QBIC Project: Querying images by content, using color, texture, and shape. InProceedings of the SPIE Conference on Storage and Retrieval for Image and Video Databases. 173–187.Google Scholar
- A. Obulesu, Vakulabharanam Vijaya Kumar, and Sumalatha Lingamgunta. 2018. Content based image retrieval using multi motif co-occurrence matrix. Int. J. Image Graph. Signal Process. 10 (Apr. 2018), 59–72. https://doi.org/10.5815/ijigsp.2018.04.07Google Scholar
- T. Ojala, Mika Rautiainen, Esa Matinmikko, and M. Aittola. 2001. Semantic image retrieval with HSV correlograms. (Jan. 2001).Google Scholar
- Michael Ortega-Binderberger. [n.d.]. Corel Image Features Data Set. Retrieved from https://archive.ics.uci.edu/ml/datasets/corel+image+features.Google Scholar
- Jing Peng, Bir Bhanu, and Shan Qing. 1999. Probabilistic feature relevance learning for content-based image retrieval. Comput. Vision Image Understand. 75 (1999), 150–164.Google Scholar
Digital Library
- Soumya Rana, Maitreyee Dey, and Siarry Patrick. 2018. Boosting content based image retrieval performance through integration of parametric & nonparametric approaches. J. Visual Commun. Image Represent. 58 (Nov. 2018), 205–219. https://doi.org/10.1016/j.jvcir.2018.11.015Google Scholar
- A. Rashno and S. Sadri. 2017. Content-based image retrieval with color and texture features in neutrosophic domain. In Proceedings of the 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA’17). 50–55. https://doi.org/10.1109/PRIA.2017.7983063Google Scholar
- Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted residuals and linear bottlenecks. Retrieved from https://arXiv:1801.04381.Google Scholar
- Amna Sarwar, Zahid Mehmood, Tanzila Saba, Khurram Ashfaq Qazi, Ahmed Adnan, and Habibullah Jamal. 2019. A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine. J. Info. Sci. 45, 1 (2019), 117–135. https://doi.org/10.1177/0165551518782825arXiv:https://doi.org/10.1177/0165551518782825Google Scholar
Digital Library
- G. V. Satya Kumar and P. G. Krishna Mohan. 2018. Local mean differential excitation pattern for content based image retrieval. SN Appl. Sci. 1, 1 (Nov. 2018), 46. https://doi.org/10.1007/s42452-018-0047-2Google Scholar
- Seong-O Shim and Tae-Sun Choi. 2003. Image indexing by modified color co-occurrence matrix. In Proceedings of the International Conference on Image Processing, Vol. 3. III–493. https://doi.org/10.1109/ICIP.2003.1247289Google Scholar
- Uzma Sharif, Zahid Mehmood, Toqeer Mahmood, Dr. Javid, Amjad Rehman, and Tanzila Saba. 2018. Scene analysis and search using local features and support vector machine for effective content-based image retrieval. Artific. Intell. Rev. 52, 2 (June 2018), 901–925. https://doi.org/10.1007/s10462-018-9636-0Google Scholar
- Jonathon Shlens. 2014. A tutorial on principal component analysis. Retrieved from https://arXiv:1404.1100.Google Scholar
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. Retrieved from https://arXiv:1409.1556.Google Scholar
- Sachendra Singh and Shalini Batra. 2020. An efficient bi-layer content based image retrieval system. Multimedia Tools Appl. 79, 25 (July 2020), 17731–17759. https://doi.org/10.1007/s11042-019-08401-7Google Scholar
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi. 2016. Inception-v4, inception-resnet and the impact of residual connections on learning. Retrieved from https://arXiv:1602.07261.Google Scholar
- Jian Xu, Cunzhao Shi, Chengzuo Qi, Chunheng Wang, and Baihua Xiao. 2017. Unsupervised part-based weighting aggregation of deep convolutional features for image retrieval. Retrieved from https://arXiv:1705.01247.Google Scholar
- G. Yosr, N. Baklouti, H. Hagras, M. Ben ayed, and A. M. Alimi. 2021. Interval Type-2 beta fuzzy near sets approach to content-based image retrieval. IEEE Trans. Fuzzy Syst. (2021), 1–1. https://doi.org/10.1109/TFUZZ.2021.3049900Google Scholar
- Muhammad Yousuf, Zahid Mehmood, Hafiz Adnan Habib, Toqeer Mahmood, Tanzila Saba, Amjad Rehman, and Muhammad Rashid. 2018. A novel technique based on visual words fusion analysis of sparse features for effective content-based image retrieval. Math. Problems Eng. 2018 (Mar. 2018), 13. https://doi.org/10.1155/2018/2134395Google Scholar
- M. D. Zeiler, G. W. Taylor, and R. Fergus. 2011. Adaptive deconvolutional networks for mid- and high-level feature learning. In Proceedings of the International Conference on Computer Vision. 2018–2025. https://doi.org/10.1109/ICCV.2011.6126474Google Scholar
- Wengang Zhou, Houqiang Li, and Qi Tian. 2017. Recent advance in content-based image retrieval: A literature survey. Retrieved from https://arXiv:1706.06064.Google Scholar
- Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2017. Learning transferable architectures for scalable image recognition. Retrieved from https://arXiv:1707.07012.Google Scholar
Index Terms
CBIR Using Features Derived by Deep Learning
Recommendations
Deep convolutional learning for Content Based Image Retrieval
In this paper we propose a model retraining method for learning more efficient convolutional representations for Content Based Image Retrieval. We employ a deep CNN model to obtain the feature representations from the activations of the convolutional ...
Deep features based medical image retrieval
AbstractIn this paper, deep convolutional neural network model is developed for classification and image retrieval purpose. Deep learning techniques obtained deep features from different abstraction levels represented using multiple hidden layers. This ...
Image Retrieval Using Fused Deep Convolutional Features
This paper proposes an image retrieval using fused deep convolutional features to solve the semantic gap between low-level features and high-level semantic features of traditional contend-based image retrieval method. Firstly, the improved network ...






Comments