Abstract
High-resolution hyperspectral (HS) reconstruction has recently achieved significantly progress, among which the method based on the fusion of the RGB and HS images of the same scene can greatly improve the reconstruction performance compared with those based on the individually spectral or spatial enhancement. It is well known that the HS image is obtained only via the costly hypersoectral sensor, whereas the RGB images can be provided by low-price RGB cameras and the spectral sensitivity (SS) functions of RGB cameras are usually different. Thus, this study proposes a HS reconstruction, which fuses merely two RGB images with redundant spectral responses. In this work, we design a new RGB camera via shifting the SS of an existed RGB camera, which can provide similar strength of spectral response with different spectral centers of SS, and fuse the new achieved color image with an existed RGB image by a deep ResNet. Experiments validate that fusion of two existed RGB images can provide impressive HS reconstruction performance and further improvement can be achieved by integrating the color image of the simulated SS with the RGB image.
- J. Aeschbacher, J. Wu, and R. Timofte. 2017. In defense of shallow learned spectral reconstruction from RGB images. In Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW’17).Google Scholar
- N. Akhtar, F. Shafait, and A. Mian. 2015. Bayesian sparse representation for hyperspectral image super resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 3631--3640.Google Scholar
- Aitor Alvarez-Gila, Joost van de Weijer, and Estibaliz Garrote. 2017. Adversarial networks for spatial context-aware spectral image reconstruction from RGB. In Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW’17).Google Scholar
- B. Arad and O. Ben-Shahar. 2016. Sparse recovery of hyperspectral signal from natural RGB images. In Proceedings of the European Conference on Computer Vision.19--34.Google Scholar
- J. M. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, N. M. Nasrabadi, and J. Chanussot. 2013. Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine 1, 2 (2013), 6--36.Google Scholar
Cross Ref
- A. Chakrabarti and T. Zickler. 2011. Statistics of real-world hyperspectral images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). 193--200.Google Scholar
- H. Chang, D. Yeung, and Y. Xiong. 2004. Super-resolution through neighbor embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’04).Google Scholar
- L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L Yuille. 2016. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. arXiv:1606.00915.Google Scholar
- R. W. Dian, S. Li, A. Guo, and L. Fang. 2018. Deep hyperspectral image sharpening. IEEE Transactions on Neural Networks and Learning Systems 29, 11 (2018), 5345--5355.Google Scholar
Cross Ref
- C. Dong, C. C. Loy, K. M. He, and X. O. Tang. 2015. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 2 (2015), 295--307.Google Scholar
- C. Dong, C. C. Loy, and X. O. Tang. 2016. Accelerating the super-resolution convolutional neural network. In Proceedings of the European Conference on Computer Vision (ECCV’16).Google Scholar
- W. S. Dong, F. Z. Fu, G. M. Shi, X. Cao, J. J. Wu, G. Y. Li, and X. Li. 2016. Hyperspectral image super-resolution via non-negative strutured sparse representation. IEEE Transactions on Image Processing 25, 3 (2016), 2337--2352.Google Scholar
Digital Library
- M. Fauvel, Y. Tarabalka, J. Benediktsson, J. Chanusssot, and J. Tilton. 2013. Advances in spectral-spatial classification of hyperspectral images. Proceedings of the IEEE 101, 3 (2013), 652--675.Google Scholar
Cross Ref
- S. Galliani, C. Lanaras, D. Marmanis, E. Baltsavias, and K. Schindler. 2017. Learned spectral super-resolution. arXiv:1703.09470.Google Scholar
- X. L. Han, J. Yu, J. Q. Luo, and W. D. Sun. 2019. Hyperspectral and multispectral image fusion using cluster-based multi-branch BP neural networks. Remote Sensing 11, 10 (2019), 1--13.Google Scholar
Cross Ref
- X.-H. Han and Y.-W. Chen. 2019. Deep residual network of spectral and spatial fusion for hyperspectral image super-resolution. In Proceedings of the IEEE International Conference on Multimedia Big Data (BigMM’19). 266--270.Google Scholar
Cross Ref
- X.-H. Han, B. X. Shi, and Y. Q. Zheng. 2018. Self-similarity constrained sparse representation for hyperspectral image super-resolution. IEEE Transactions on Image Processing 27, 11 (2018), 5625--5637.Google Scholar
Digital Library
- X.-H. Han, B. X. Shi, and Y. Q. Zheng. 2018. SSF-CNN: Spatial and spectral fusion with CNN for hyperspectral image super-resolution. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP’18). 2506--2510.Google Scholar
- R. C. Hardie, M. T. Eismann, and G. L. Wilson. 2004. Map estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Transactions on Image Processing 13, 9 (2004), 1174--1184.Google Scholar
Digital Library
- K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google Scholar
- A. Hore and D. Ziou. 2010. Image quality metrics: PSNR vs. SSIM. In Proceedings of the 20th International Conference on Pattern Recognition (ICPR’10). 2366--2369.Google Scholar
- J. Jiang, D. Y. Liu, J. W. Gu, and S. Susstrunk. 2013. What is the space of spectral sensitivity functions for digital color cameras? In Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV’13). 4321--4328.Google Scholar
- J. Kim, J. K. Lee, and K. M. Lee. 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).Google Scholar
- C. Kwan, J. H. Choi, S. H. Chan, J. Zhou, and B. Budavari. 2018. A super-resolution and fusion approach to enhancing hyperspectral images. Remote Sensing 10, 9 (2018), 1--28.Google Scholar
Cross Ref
- C. Lanaras, E. Baltsavias, and K. Schindler. 2015. Hyperspectral super-resolution by coupled spectral unmixing. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’15). 3586--3595.Google Scholar
- Y. S. Li, J. Hua, X. Zhao, W. Y. Xie, and J. J. Li. 2017. Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing 266 (2017), 29--41.Google Scholar
Cross Ref
- B. Lim, A. Son, H. Kim, S. Nah, and K. M. Lee. 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17).Google Scholar
- L. Loncan, L. B. Almeida, J. M. Bioucas-Dias, X. Briottet, J. Chanussot, N. Dobigeon, S. Fabre, et al. 2015. Hyperspectral pansharpening: A review. IEEE Geoscience and Remote Sensing Magazine 3, 3 (2015), 27--46.Google Scholar
Cross Ref
- S. H. Mei, X. Yuan, J. Y. Ji, and Y. F. Zhang. 2017. Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Neurocomputing 9, 11 (2017), 1139.Google Scholar
- Y. Tarabalka, J. Chanusssot, and J. Benediktsson. 2010. Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 40, 5 (2010), 1267--1279.Google Scholar
Digital Library
- Q. Wei, J. Bioucas-Dias, N. Dobigeon, and J. Y. Toureret. 2015. Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing 53, 7 (2015), 3658--3668.Google Scholar
Cross Ref
- J. Yang, J. Wright, T. Huang, and Y. Ma. 2008. Image super-resolution as sparse representation of raw image patches. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08), Vol. 2. 1--8.Google Scholar
- F. Yasuma, D. Iso, and S. Nayar. 2010. Generalized assorted pixel camera: Post-capture control of resolution, dynamic range and spectrum. IEEE Transactions on Image Processing 19, 9 (2010), 2241--2253.Google Scholar
Digital Library
- C. Yi, Y. Q. Zhao, J. C. Chan, F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar. 2019. Spectral super-resolution for multispectral image based on spectral improvement strategy and spatial preservation strategy. IEEE Transactions on Geoscience and Remote Sensing 57, 11 (2019), 9010--9024.Google Scholar
Cross Ref
- N. Yokoya, T. Yairi, and A. Iwasaki. 2012. Coupled nonnegative matrix factorization for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing 50, 2 (2012), 528--537.Google Scholar
Cross Ref
- J. Zhou, C. Kwan, and B. Budavari. 2016. Hyperspectral image super-resolution: A hybrid color mapping approach. Journal of Applied Remote Sensing 10, 31 (2016), 035024-1--035024-20.Google Scholar
Cross Ref
- Y. Zhou, H. Chang, K. Barner, P. Spellman, and B. Parvin. 2014. Classification of histology sections via multispectral convolutional sparse coding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 3081--3088.Google Scholar
Index Terms
Hyperspectral Reconstruction with Redundant Camera Spectral Sensitivity Functions
Recommendations
High-quality hyperspectral reconstruction using a spectral prior
We present a novel hyperspectral image reconstruction algorithm, which overcomes the long-standing tradeoff between spectral accuracy and spatial resolution in existing compressive imaging approaches. Our method consists of two steps: First, we learn ...
EMCCD color correction based on spectral sensitivity analysis
The sensor response function of a color camera is very essential to understand an overall camera imaging pipeline and to process captured images. It is also true when we characterize underlying imaging behaviors of the electron multiplying charge ...
Camera spectral sensitivity, illumination and spectral reflectance estimation for a hybrid hyperspectral image capture system
2017 IEEE International Conference on Image Processing (ICIP)A variety of methods have been proposed to restore high resolution hyperspectral image (HSI) from a hybrid camera system, which captures high spatial resolution RGB images and low spatial resolution HSI. They focused unanimously on HSI super-resolution ...






Comments