Abstract
The pansharpening is a combination of multispectral (MS) and panchromatic (PAN) images that produce a high-spatial-spectral-resolution MS images. In multiresolution analysis–based pansharpening schemes, some spatial and spectral distortions are found. It can be reduced by adding spatial detail images of the PAN image into MS images. In the convolution neural network– (CNN) based method, the lowpass filter image extracted by the CNN model when MS and PAN images are directly applied into the input. The feature values are very high and reduce the conversion efficiency. In the proposed scheme, bi-dimensional empirical mode decomposition is used to extract the spatial detail information of the PAN image to reduce the feature values of the input. This extracted PAN image information is applied to the CNN to produce the non-linear changes in the image pixels and transformed into the perfect spatial detail image. It identifies the spatial and spectral detail quantity for the proposed scheme and it also varies with the different datasets automatically of the same satellite images. Simulation results in the context of qualitative and quantitative analysis demonstrate the effectiveness of proposed scheme applied on datasets collected by different satellites.
- [1] . 2003. An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas. In Proceedings of the 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas. IEEE, 90–94.Google Scholar
Cross Ref
- [2] . 2006. MTF-tailored multiscale fusion of high-resolution MS and pan imagery. Photogram. Eng. Remote Sens. 72, 5 (2006), 591–596.Google Scholar
Cross Ref
- [3] . 2013. Bi-cubic interpolation for shift-free pan-sharpening. ISPRS J. Photogram. Remote Sensing 86 (2013), 65–76.Google Scholar
Cross Ref
- [4] . 2008. Multispectral and panchromatic data fusion assessment without reference. Photogram. Eng. Remote Sens. 74, 2 (2008), 193–200.Google Scholar
Cross Ref
- [5] . 2007. Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Trans. Geosci. Remote Sens. 45, 10 (2007), 3012–3021.Google Scholar
Cross Ref
- [6] . 2019. Convolutional autoencoder-based multispectral image fusion. IEEE Access 7 (2019), 35673–35683.Google Scholar
Cross Ref
- [7] . 1991. Comparison of three different methods to merge multiresolution and multispectral data- landsat TM and SPOT panchromatic. Photogram. Eng. Remote Sens. 57, 3 (1991), 295–303.Google Scholar
- [8] . 2014. Wavelet packet mixing for image fusion and pan-sharpening. In Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XX, Vol. 9088. International Society for Optics and Photonics, 908803.Google Scholar
- [9] . 2015. Challenges and opportunities of multimodality and data fusion in remote sensing. Proc. IEEE 103, 9 (2015), 1585–1601.Google Scholar
Cross Ref
- [10] . 2014. A bidimensional empirical mode decomposition method for fusion of multispectral and panchromatic remote sensing images. Remote Sens. 6, 9 (2014), 8446–8467.Google Scholar
Cross Ref
- [11] . 2004. Empirical mode decomposition as a filter bank. IEEE Sign. Process. Lett. 11, 2 (2004), 112–114.Google Scholar
Cross Ref
- [12] . 1987. Color enhancement of highly correlated images. II. Channel ratio and “chromaticity” transformation techniques. Remote Sens’ Environ’ 22, 3 (1987), 343–365.Google Scholar
Cross Ref
- [13] . 2004. Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans’ Geosci. Remote Sens. 42, 6 (2004), 1291–1299.Google Scholar
Cross Ref
- [14] . 2019. Pansharpening via detail injection based convolutional neural networks. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. (2019).Google Scholar
- [15] . 2015. An integrated scheme to improve pan-sharpening visual quality of satellite images. Egypt. Inf. J. 16, 1 (2015), 121–131.Google Scholar
- [16] . 1998. The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis. In Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, Vol. 454. The Royal Society, 903–995.Google Scholar
Cross Ref
- [17] . 2009. MAP estimation for multiresolution fusion in remotely sensed images using an IGMRF prior model. IEEE Trans. Geosci. Remote Sens. 48, 3 (2009), 1245–1255.Google Scholar
- [18] . 2013. Pansharpening with matting model. IEEE Trans. Geosci. Remote Sens. 52, 8 (2013), 5088–5099.Google Scholar
Cross Ref
- [19] . 2018. Revised spatial transformer network towards improved image super-resolutions. In Proceedings of the 24th International Conference on Pattern Recognition (ICPR’18). IEEE, 2688–2692.Google Scholar
Cross Ref
- [20] . 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1646–1654.Google Scholar
Cross Ref
- [21] . 2010. Fast and efficient panchromatic sharpening. IEEE Trans. Geosci. Remote Sens. 48, 1 (2010), 155–163.Google Scholar
Cross Ref
- [22] . 2010. A new pan-sharpening method using a compressed sensing technique. IEEE Trans. Geosci. Remote Sens. 49, 2 (2010), 738–746.Google Scholar
Cross Ref
- [23] . 2013. Remote sensing image fusion via sparse representations over learned dictionaries. IEEE Trans. Geosci. Remote Sens. 51, 9 (2013), 4779–4789.Google Scholar
Cross Ref
- [24] . 2020. Multi-focus image fusion using geometric algebra based discrete fourier transform. IEEE Access 8 (2020), 60019–60028.Google Scholar
Cross Ref
- [25] . 2016. Pansharpening by convolutional neural networks. Remote Sens. 8, 7 (2016), 594.Google Scholar
Cross Ref
- [26] . 2003. Image analysis by bidimensional empirical mode decomposition. Image Vis. Comput. 21, 12 (2003), l019–1026.Google Scholar
Cross Ref
- [27] . 2005. Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods. IEEE Trans. Geosci. Remote Sens. 43, 10 (2005), 2376–2385.Google Scholar
Cross Ref
- [28] . 2016. Fusion of multispectral and panchromatic images based on morphological operators. IEEE Trans. Image Process. 25, 6 (2016), 2882–2895.Google Scholar
- [29] . 2021. A pansharpening scheme using spectral graph wavelet transforms and convolutional neural networks. Int. J. Remote Sens. 42, 8 (2021), 2898–2919.Google Scholar
Cross Ref
- [30] . 2021. Pansharpening scheme using spatial detail injection based convolutional neural networks. IET Image Process. (2021).Google Scholar
- [31] . 2017. A hybrid approach for pansharpening using hilbert vibration decomposition. In Proceedings of the IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI’17). IEEE, 2801–2805.Google Scholar
Cross Ref
- [32] . 2019. A two-dimensional discrete fractional fourier transform-based pansharpening scheme. Int. J. Remote Sens. 40, 16 (2019), 6098–6115.Google Scholar
Cross Ref
- [33] . 2017. Pansharpening approach using hilbert vibration decomposition. IET Image Process. 11, 12 (2017), 1152–1162.Google Scholar
Cross Ref
- [34] . 2018. Pansharpening scheme using filtering in two-dimensional discrete fractional fourier transform. IET Image Process. 12, 6 (2018), 1013–1019.Google Scholar
Cross Ref
- [35] . 2018. Target-adaptive CNN-based pansharpening. IEEE Trans. Geosci. Remote Sens. 56, 9 (2018), 5443–5457.Google Scholar
Cross Ref
- [36] . 2008. An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Trans. Geosci. Remote Sens. 46, 5 (2008), 1323–1335.Google Scholar
Cross Ref
- [37] . 2015. Improving the spatial resolution of landsat TM/ETM+ through fusion with SPOT5 images via learning-based super-resolution. IEEE Trans. Geosci. Remote Sens. 53, 3 (2015), 1195–1204.Google Scholar
- [38] . 2003. An algorithm for mapping burnt areas in Australia using SPOT-VEGETATION data. IEEE Trans. Geosci. Remote Sens. 41, 4 (
April 2003), 907–909.DOI: Google ScholarCross Ref
- [39] . 2018. Scale-recurrent network for deep image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8174–8182.Google Scholar
Cross Ref
- [40] . 2008. Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics. IEEE Trans. Geosci. Remote Sens. 46, 5 (2008), 1301–1312.Google Scholar
Cross Ref
- [41] . 2018. Deep learning for image denoising: A survey. In International Conference on Genetic and Evolutionary Computing. Springer, 563–572.Google Scholar
- [42] . 2004. A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery. IEEE Geosci. Remote Sens. Lett. 1, 4 (2004), 309–312.Google Scholar
Cross Ref
- [43] . 2020. A detail-preserving cross-scale learning strategy for CNN-based pansharpening. Remote Sens. 12, 3 (2020), 348.Google Scholar
Cross Ref
- [44] . 2015. A critical comparison among pansharpening algorithms. IEEE Trans. Geosci. Remote Sens. 53, 5 (2015), 2565–2586.Google Scholar
Cross Ref
- [45] . 2018. Full scale regression-based injection coefficients for panchromatic sharpening. IEEE Trans. Image Process. 27, 7 (2018), 3418–3431.Google Scholar
- [46] . 2014. Contrast and error-based fusion schemes for multispectral image pansharpening. IEEE Geosci. Remote Sens. Lett. 11, 5 (2014), 930–934.Google Scholar
Cross Ref
- [47] . 2002. Data Fusion: Definitions and Architectures: Fusion of Images of Different Spatial Resolutions. Presses des MINES.Google Scholar
- [48] . 2002. A universal image quality index. IEEE Sign. Process. Lett. 9, 3 (2002), 81–84.Google Scholar
Cross Ref
- [49] . 2017. Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci. Remote Sens. Lett. 14, 10 (2017), 1795–1799.Google Scholar
Cross Ref
- [50] . 2012. Image denoising and inpainting with deep neural networks. In Advances in Neural Information Processing Systems. 341–349.Google Scholar
Digital Library
- [51] . 1992. Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. In Proc. Summaries 3rd Annu. JPL Airborne Geosci. Workshop, vol. 1. 147–149.Google Scholar
- [52] . 2005. Using FCMC, FVS, and PCA techniques for feature extraction of multispectral images. IEEE Geosci. Remote Sens. Lett. 2, 2 (
April 2005), 108–112.DOI: Google ScholarCross Ref
- [53] . 2012. Adjustable model-based fusion method for multispectral and panchromatic images. IEEE Trans. Syst. Man Cybernet. B 42, 6 (2012), 1693–1704.Google Scholar
Digital Library
- [54] . 2013. An object-based approach for urban land cover classification: Integrating LiDAR height and intensity data. IEEE Geosci. Remote Sens. Lett. 10, 4 (
July 2013), 928–931.DOI: Google ScholarCross Ref
- [55] . 2012. A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans. Geosci. Remote Sens. 51, 5 (2012), 2827–2836.Google Scholar
Cross Ref
- [56] . 2016. Strategies combining spectral angle mapper and change vector analysis to unsupervised change detection in multispectral images. IEEE Geosci. Remote Sens. Lett. 13, 5 (
May 2016), 681–685.DOI: Google ScholarCross Ref
Index Terms
Pansharpening Scheme Using Bi-dimensional Empirical Mode Decomposition and Neural Network
Recommendations
Video super resolution using convolutional neural network and image fusion techniques
Recently, huge progress has been achieved in the field of single image super resolution which augments the resolution of images. The idea behind super resolution is to convert low-resolution images into high-resolution images. SRCNN (Single ...
Compression of multispectral images with inverse pyramid decomposition
KES'10: Proceedings of the 14th international conference on Knowledge-based and intelligent information and engineering systems: Part IIIIn the paper is presented a new method for compression of multispectral images, based on the Inverse Difference Pyramid decomposition. The method is applicable for any number of multispectral images of same object. The processing is performed as ...
Bi-dimensional empirical mode decomposition (BEMD) algorithm based on particle swarm optimization-fractal interpolation
The performance of interpolation algorithm used in bi-dimensional empirical mode decomposition directly affects its popularization and application. Therefore, the research on interpolation algorithm is more reasonable, accurate and fast. So far, in the ...






Comments