Abstract
Reconstructing a high-resolution hyperspectral (HR-HS) image via merging a low-resolution hyperspectral (LR-HS) image and a high-resolution RGB (HR-RGB) image has become a hot research topic, and can greatly benefit for different subsequent high-level vision tasks. Recently, deep learning–based approaches have evolved for HS image reconstruction and validated impressive performance. However, to learn a good reconstruction model in the deep learning–based methods, it is mandatory to previously collect large-scale training triplets consisting of the LR-HS, HR-RGB, and HR-HS images, which is difficult to be collected in real applications. This study proposes a deep self-supervised HS image reconstruction framework (DSSH), which does not have to depend on any handcrafted prior and previously collected training triplets at all. The proposed DSSH method leverages the designed network architecture itself for capturing the prior of the underlying structure in the latent HR-HS image and employs the observed LR-HS and HR-RGB images only for network parameter learning. Experiments on two benchmark HS image datasets validated that the proposed DSSH method manifests very impressive reconstruction performance, and is even better than some state-of-the-art supervised learning approaches.
- [1] . 2014. Sparse spatio-spectral representation for hyperspectral image super-resolution. In European Conference on Computer Vision. Springer, 63–78.Google Scholar
Cross Ref
- [2] . 2015. Bayesian sparse representation for hyperspectral image super resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3631–3640.Google Scholar
Cross Ref
- [3] . 2016. Hierarchical beta process with Gaussian process prior for hyperspectral image super resolution. In European Conference on Computer Vision. Springer, 103–120.Google Scholar
Cross Ref
- [4] . 2018. Systems and methods for hyperspectral medical imaging using real-time projection of spectral information. (
Feb. 2018).US Patent No. 9,883,833. Filed May 13, 2009. Issued Feb. 6, 2018.Google Scholar - [5] . 2013. Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine 1, 2 (2013), 6–36.Google Scholar
Cross Ref
- [6] . 2011. Hyperspectral remote sensing for mineral exploration in Pulang, Yunnan Province, China. International Journal of Remote Sensing 32, 9 (2011), 2409–2426.Google Scholar
Digital Library
- [7] . 2011. Statistics of real-world hyperspectral images. In CVPR 2011. IEEE, 193–200.Google Scholar
Digital Library
- [8] . 2006. Self-supervised monocular road detection in desert terrain. In Robotics: Science and Systems, Vol. 38.Google Scholar
- [9] . 2018. Deep hyperspectral image sharpening. IEEE Transactions on Neural Networks and Learning Systems99 (2018), 1–11.Google Scholar
- [10] . 2016. Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Transactions on Image Processing 25, 5 (2016), 2337–2352.Google Scholar
Digital Library
- [11] . 2019. Hyperspectral image super-resolution with optimized RGB guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11661–11670.Google Scholar
Cross Ref
- [12] . 2019. Deep residual network of spectral and spatial fusion for hyperspectral image super-resolution. In 2019 IEEE 5th International Conference on Multimedia Big Data (BigMM’19). IEEE, 266–270.Google Scholar
Cross Ref
- [13] . 2018. Self-similarity constrained sparse representation for hyperspectral image super-resolution. IEEE Transactions on Image Processing 27, 11 (2018), 5625–5637.Google Scholar
Digital Library
- [14] . 2018. SSF-CNN: Spatial and spectral fusion with CNN for hyperspectral image super-resolution. In 2018 25th IEEE International Conference on Image Processing (ICIP’18). IEEE, 2506–2510.Google Scholar
Cross Ref
- [15] . 2019. Residual component estimating CNN for image super-resolution. In 2019 IEEE 5th International Conference on Multimedia Big Data (BigMM’19). IEEE, 443–447.Google Scholar
Cross Ref
- [16] . 2019. Multi-level and multi-scale spatial and spectral fusion CNN for hyperspectral image super-resolution. In Proceedings of the IEEE International Conference on Computer Vision Workshops.Google Scholar
Cross Ref
- [17] . 2015. Total-variation-regularized low-rank matrix factorization for hyperspectral image restoration. IEEE Transactions on Geoscience and Remote Sensing 54, 1 (2015), 178–188.Google Scholar
Cross Ref
- [18] . 2017. Generative adversarial networks-based semi-supervised learning for hyperspectral image classification. Remote Sensing 9, 10 (2017), 1042.Google Scholar
Cross Ref
- [19] . 2019. Self-supervised hyperspectral image restoration using separable image prior. arXiv:1907.00651. Google Scholar
Cross Ref
- [20] . 2019. Zero-shot hyperspectral image denoising with separable image prior. In Proceedings of the IEEE International Conference on Computer Vision Workshops.Google Scholar
Cross Ref
- [21] . 2011. High-resolution hyperspectral imaging via matrix factorization. In CVPR 2011. IEEE, 2329–2336.Google Scholar
Digital Library
- [22] . 2007. Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23, 12 (2007), 1495–1502.Google Scholar
Cross Ref
- [23] . 2017. Resolution enhancement for hyperspectral images: A super-resolution and fusion approach. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’17). IEEE, 6180–6184.Google Scholar
Digital Library
- [24] . 2000. Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. (
Jan. 2000).US Patent 6,011,875. Filed Apr. 29, 1998. Issued Jan. 4, 2000.Google Scholar - [25] . 2015. Hyperspectral super-resolution by coupled spectral unmixing. In Proceedings of the IEEE International Conference on Computer Vision. 3586–3594.Google Scholar
Digital Library
- [26] . 2017. Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing 266 (2017), 29–41.Google Scholar
Cross Ref
- [27] . 2020. Unsupervised multispectral and hyperspectral image fusion with deep spatial and spectral priors. In Proceedings of the Asian Conference on Computer Vision.Google Scholar
- [28] . 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing 42, 8 (2004), 1778–1790.Google Scholar
Cross Ref
- [29] . 2018. Unsupervised sparse Dirichlet-Net for hyperspectral image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2511–2520.Google Scholar
Cross Ref
- [30] . 2019. Deep hyperspectral prior: Denoising, inpainting, super-resolution. arXiv:1902.00301. Google Scholar
Cross Ref
- [31] . 2019. Deep hyperspectral prior: Denoising, inpainting, super-resolution. CoRR abs/1902.00301 (2019). arXiv:1902.00301. http://arxiv.org/abs/1902.00301.Google Scholar
- [32] . 2015. The effect of dictionary learning algorithms on super-resolution hyperspectral reconstruction. In 2015 XXV International Conference on Information, Communication and Automation Technologies (ICAT’15). IEEE, 1–5.Google Scholar
Digital Library
- [33] . 2020. Guided deep decoder: Unsupervised image pair fusion. In European Conference on Computer Vision. Springer, 87–102.Google Scholar
Digital Library
- [34] . 2018. Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9446–9454.Google Scholar
- [35] . 2004. A hyperspectral imaging system for in vivo optical diagnostics. IEEE Engineering in Medicine and Biology Magazine 23, 5 (2004), 40–49.Google Scholar
Cross Ref
- [36] . 2019. Hyperspectral image reconstruction using a deep spatial-spectral prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8032–8041.Google Scholar
Cross Ref
- [37] . 2013. A non-negative sparse promoting algorithm for high resolution hyperspectral imaging. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 1409–1413.Google Scholar
Cross Ref
- [38] . 2019. Multispectral and hyperspectral image fusion by MS/HS fusion net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1585–1594.Google Scholar
Cross Ref
- [39] . 2017. Comparison of hyperspectral imaging and computer vision for automatic differentiation of organically and conventionally farmed salmon. Journal of Food Engineering 196 (2017), 170–182.Google Scholar
Cross Ref
- [40] . 2010. Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing 19, 9 (2010), 2241–2253.Google Scholar
Digital Library
- [41] . 2011. Coupled non-negative matrix factorization (CNMF) for hyperspectral and multispectral data fusion: Application to pasture classification. In 2011 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 1779–1782.Google Scholar
Cross Ref
- [42] . 2017. Multisensor coupled spectral unmixing for time-series analysis. IEEE Transactions on Geoscience and Remote Sensing 55, 5 (2017), 2842–2857.Google Scholar
Cross Ref
- [43] . 2012. A super-resolution reconstruction algorithm for hyperspectral images. Signal Processing 92, 9 (2012), 2082–2096.Google Scholar
Digital Library
- [44] . 2020. Unsupervised adaptation learning for hyperspectral imagery super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3073–3082.Google Scholar
Cross Ref
- [45] . 2011. Hyperspectral imagery super-resolution by sparse representation and spectral regularization. EURASIP Journal on Advances in Signal Processing 2011, 1 (2011), 1–10.Google Scholar
Cross Ref
- [46] . 2020. Residual component estimating CNN for image super-resolution. IEEE Transactions on Image Processing 30 (2020), 1423–1428.Google Scholar
- [47] . 2020. Hyperspectral image super-resolution combining with deep learning and spectral unmixing. Signal Processing: Image Communication 84 (2020), 115833.Google Scholar
Cross Ref
Index Terms
Deep Self-Supervised Hyperspectral Image Reconstruction
Recommendations
Lightweight Hyperspectral Image Reconstruction Network with Deep Feature Hallucination
Computer Vision – ACCV 2022 WorkshopsAbstractHyperspectral image reconstruction from a compressive snapshot is an dispensable step in the advanced hyperspectral imaging systems to solve the low spatial and/or temporal resolution issue. Most existing methods extensively exploit various hand-...
Semi-Supervised Deep Learning Using Pseudo Labels for Hyperspectral Image Classification
Deep learning has gained popularity in a variety of computer vision tasks. Recently, it has also been successfully applied for hyperspectral image classification tasks. Training deep neural networks, such as a convolutional neural network for ...
Deformed2Self: Self-supervised Denoising for Dynamic Medical Imaging
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021AbstractImage denoising is of great importance for medical imaging system, since it can improve image quality for disease diagnosis and downstream image analyses. In a variety of applications, dynamic imaging techniques are utilized to capture the time-...






Comments