skip to main content
research-article

Boosting Hyperspectral Image Classification with Dual Hierarchical Learning

Authors Info & Claims
Published:05 January 2023Publication History
Skip Abstract Section

Abstract

Hyperspectral image (HSI) classification aims at predicting the pixel-wise labels in an image, where there are only a few labeled pixel samples (hard labels) for training. It is a challenging task since the classification process is susceptible to over-fitting under training with limited samples. To relieve this problem, we propose a method based on dual hierarchical learning. First, we employ a connectionist hyperspectral convolution (HC) network to capture the representations of the pixels from different receptive fields. Specifically, an HC is designed to learn the correlation among adjacent pixels and is further extended to a connectionist hierarchical structure. These operations use the correlation to enhance one-pixel learning from multiple receptive fields. Second, we analyze the properties in the hyperspectral image and introduce a hierarchical pseudo label generation algorithm to enrich the supervision of the label information. Finally, we design a dual hierarchical learning strategy to help all HC layers learn from both the hard labels and the hierarchical pseudo labels. In other words, it addresses the HSI classification problem from different views. For inference, we employ two fusion strategies to find a better prediction. The experimental results on four popular HSI benchmarks, i.e., Salinas-A, IndianPines, PaviaU, and PaviaC, demonstrate the effectiveness of the proposed method. Our code is publicly available on GitHub: https://github.com/ShuoWangCS/HSI-DHL.

REFERENCES

  1. [1] Achanta Radhakrishna, Shaji Appu, Smith Kevin, Lucchi Aurelien, Fua Pascal, and Süsstrunk Sabine. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 11 (2012), 22742282.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Appice Annalisa, Guccione Pietro, and Malerba Donato. 2016. Transductive hyperspectral image classification: Toward integrating spectral and relational features via an iterative ensemble system. Machine Learning 103, 3 (2016), 343375.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Appice Annalisa, Guccione Pietro, and Malerba Donato. 2017. A novel spectral-spatial co-training algorithm for the transductive classification of hyperspectral imagery data. Pattern Recognition 63 (2017), 229245.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Appice Annalisa and Malerba Donato. 2019. Segmentation-aided classification of hyperspectral data using spatial dependency of spectral bands. ISPRS Journal of Photogrammetry and Remote Sensing 147 (2019), 215231.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Camps-Valls Gustavo, Marsheva Tatyana V. Bandos, and Zhou Dengyong. 2007. Semi-supervised graph-based hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 45, 10 (2007), 30443054.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Castelluccio Marco, Poggi Giovanni, Sansone Carlo, and Verdoliva Luisa. 2017. Training convolutional neural networks for semantic classification of remote sensing imagery. In 2017 Joint Urban Remote Sensing Event (JURSE’17). IEEE, 14.Google ScholarGoogle Scholar
  7. [7] Chang C.-I.. 2000. An information-theoretic approach to spectral variability, similarity, and discrimination for hyperspectral image analysis. IEEE Transactions on Information Theory 46, 5 (2000), 19271932.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Cui Binge, Cui Jiandi, Hao Siyuan, Guo Nannan, and Lu Yan. 2020. Spectral-spatial hyperspectral image classification based on superpixel and multi-classifier fusion. International Journal of Remote Sensing 41, 16 (2020), 61576182.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Piane C. Delle, MacRae C., Wilson N., Torpy A., and Dewhurst D. N.. 2019. Silica diagenesis in the Marcellus Shale: A trace element and hyperspectral cathodoluminescence study. In 6th EAGE Shale Workshop, Vol. 2019. European Association of Geoscientists & Engineers, 15.Google ScholarGoogle Scholar
  10. [10] Dópido Inmaculada, Li Jun, Marpu Prashanth Reddy, Plaza Antonio, Dias José M. Bioucas, and Benediktsson Jon Atli. 2013. Semisupervised self-learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 51, 7 (2013), 40324044.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Du Qian and Ren Hsuan. 2003. Real-time constrained linear discriminant analysis to target detection and classification in hyperspectral imagery. Pattern Recognition 36, 1 (2003), 112.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Felzenszwalb Pedro F. and Huttenlocher Daniel P.. 2004. Efficient graph-based image segmentation. International Journal of Computer Vision 59, 2 (2004), 167181.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Gao Yue, Ji Rongrong, Cui Peng, Dai Qionghai, and Hua Gang. 2014. Hyperspectral image classification through bilayer graph-based learning. IEEE Transactions on Image Processing 23, 7 (2014), 27692778.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Green Robert O., Eastwood Michael L., Sarture Charles M., Chrien Thomas G., Aronsson Mikael, Chippendale Bruce J., Faust Jessica A., Pavri Betina E., Chovit Christopher J., Solis Manuel, et al. 1998. Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS). Remote Sensing of Environment 65, 3 (1998), 227248.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Hang Renlong, Zhou Feng, Liu Qingshan, and Ghamisi Pedram. 2020. Classification of hyperspectral images via multitask generative adversarial networks. IEEE Transactions on Geoscience and Remote Sensing 59, 2 (2020), 14241436.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Harris J. R., Rogge D., Hitchcock R., Ijewliw O., and Wright D.. 2005. Mapping lithology in Canada’s Arctic: Application of hyperspectral data using the minimum noise fraction transformation and matched filtering. Canadian Journal of Earth Sciences 42, 12 (2005), 21732193.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Haut J., Paoletti M., Paz-Gallardo Abel, Plaza Javier, Plaza Antonio, and Vigo-Aguiar J.. 2017. Cloud implementation of logistic regression for hyperspectral image classification. In Proceedings of the 17th International Conference on Computational and Mathematical Methods in Science and Engineering (CMMSE), Vol. 3. Cádiz, Spain: Costa Ballena (Rota), 10632321.Google ScholarGoogle Scholar
  18. [18] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770778.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] He Mingyi, Li Bo, and Chen Huahui. 2017. Multi-scale 3D deep convolutional neural network for hyperspectral image classification. In 2017 IEEE International Conference on Image Processing (ICIP’17). IEEE, 39043908.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Hecker Christoph, Meijde Mark Van der, Werff Harald van der, and Meer Freek D van der. 2008. Assessing the influence of reference spectra on synthetic SAM classification results. IEEE Transactions on Geoscience and Remote Sensing 46, 12 (2008), 41624172.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Huang Gao, Liu Zhuang, Maaten Laurens Van Der, and Weinberger Kilian Q.. 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 47004708.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Jiao Licheng, Liang Miaomiao, Chen Huan, Yang Shuyuan, Liu Hongying, and Cao Xianghai. 2017. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 55, 10 (2017), 55855599.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Kingma Diederik P. and Ba Jimmy. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. 113.Google ScholarGoogle Scholar
  24. [24] Krizhevsky Alex, Sutskever Ilya, and Hinton Geoffrey E.. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 10971105.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Kunkel B., Blechinger F., Lutz R., Doerffer R., Piepen H. Van der, and Schroder M.. 1988. ROSIS (reflective optics system imaging spectrometer)-A candidate instrument for polar platform missions. In Optoelectronic Technologies for Remote Sensing from Space, Vol. 868. International Society for Optics and Photonics, 134141.Google ScholarGoogle Scholar
  26. [26] Lee Hyungtae and Kwon Heesung. 2017. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing 26, 10 (2017), 48434855.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Li Shutao, Dian Renwei, Fang Leyuan, and Bioucas-Dias José M.. 2018. Fusing hyperspectral and multispectral images via coupled sparse tensor factorization. IEEE Transactions on Image Processing 27, 8 (2018), 41184130.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Liang Liang, Di Liping, Zhang Lianpeng, Deng Meixia, Qin Zhihao, Zhao Shuhe, and Lin Hui. 2015. Estimation of crop LAI using hyperspectral vegetation indices and a hybrid inversion method. Remote Sensing of Environment 165 (2015), 123134.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Liang Yi, Zhao Xin, Guo Alan J. X., and Zhu Fei. 2019. Hyperspectral image classification with deep metric learning and conditional random field. IEEE Geoscience and Remote Sensing Letters 17, 6 (2019), 10421046.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Liu Bing, Yu Xuchu, Yu Anzhu, Zhang Pengqiang, Wan Gang, and Wang Ruirui. 2018. Deep few-shot learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 57, 4 (2018), 22902304.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Liu Bing, Yu Xuchu, Zhang Pengqiang, Yu Anzhu, Fu Qiongying, and Wei Xiangpo. 2017. Supervised deep feature extraction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 56, 4 (2017), 19091921.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Long Jonathan, Shelhamer Evan, and Darrell Trevor. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 34313440.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Luo Haowen. 2018. Shorten spatial-spectral RNN with parallel-GRU for hyperspectral image classification. arXiv preprint arXiv:1810.12563 (2018).Google ScholarGoogle Scholar
  34. [34] Ma Li, Crawford Melba M., and Tian Jinwen. 2010. Local manifold learning-based \(k\)-nearest-neighbor for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 48, 11 (2010), 40994109.Google ScholarGoogle Scholar
  35. [35] Melgani Farid and Bruzzone Lorenzo. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing 42, 8 (2004), 17781790.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Paoletti Mercedes E., Haut Juan M., Plaza Javier, and Plaza Antonio. 2020. Scalable recurrent neural network for hyperspectral image classification. Journal of Supercomputing 76, 11 (2020), 88668882.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Roy Swalpa Kumar, Krishna Gopal, Dubey Shiv Ram, and Chaudhuri Bidyut B.. 2019. HybridSN: Exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters 17, 2 (2019), 277281.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Roy Swalpa Kumar, Manna Suvojit, Song Tiecheng, and Bruzzone Lorenzo. 2020. Attention-based adaptive spectral-spatial kernel resnet for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 59, 9 (2020), 78317843.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Santiago J. González, Schenkel Fabian, Gross Wolfgang, and Middelmann Wolfgang. 2020. An unsupervised labeling approach for hyperspectral image classification. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 43 (2020), 407415.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Shen Yu, Zhu Sijie, Chen Chen, Du Qian, Xiao Liang, Chen Jianyu, and Pan Delu. 2020. Efficient deep learning of nonlocal features for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 59, 7 (2020), 60296043.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Tan Kun, Hu Jun, Li Jun, and Du Peijun. 2015. A novel semi-supervised hyperspectral image classification approach based on spatial neighborhood information and classifier combination. ISPRS Journal of Photogrammetry and Remote Sensing 105 (2015), 1929.Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Timofte Radu, Rothe Rasmus, and Gool Luc Van. 2016. Seven ways to improve example-based single image super resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 18651873.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Villa Alberto, Benediktsson Jón Atli, Chanussot Jocelyn, and Jutten Christian. 2011. Hyperspectral image classification with independent component discriminant analysis. IEEE Transactions on Geoscience and Remote Sensing 49, 12 (2011), 48654876.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Wang Liguo, Hao Siyuan, Wang Qunming, and Wang Ying. 2014. Semi-supervised classification for hyperspectral imagery based on spatial-spectral label propagation. ISPRS Journal of Photogrammetry and Remote Sensing 97 (2014), 123137.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Zhang Jing, Zhang Tong, Dai Yuchao, Harandi Mehrtash, and Hartley Richard. 2018. Deep unsupervised saliency detection: A multiple noisy labeling perspective. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 90299038.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Zhang Lefei, Zhang Liangpei, Tao Dacheng, and Huang Xin. 2011. On combining multiple features for hyperspectral remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing 50, 3 (2011), 879893.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Zheng Zhuo, Zhong Yanfei, Ma Ailong, and Zhang Liangpei. 2020. FPGA: Fast patch-free global learning framework for fully end-to-end hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 58, 8 (2020), 56125626.Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Zhong Ping and Wang Runsheng. 2010. Learning conditional random fields for classification of hyperspectral images. IEEE Transactions on Image Processing 19, 7 (2010), 18901907.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Zhong Zilong, Li Jonathan, Luo Zhiming, and Chapman Michael. 2017. Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing 56, 2 (2017), 847858.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Zhou Feng, Hang Renlong, Liu Qingshan, and Yuan Xiaotong. 2019. Pyramid fully convolutional network for hyperspectral and multispectral image fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, 5 (2019), 15491558.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Zhou Yicong and Wei Yantao. 2015. Learning hierarchical spectral–spatial features for hyperspectral image classification. IEEE Transactions on Cybernetics 46, 7 (2015), 16671678.Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Zhu Qiqi, Deng Weihuan, Zheng Zhuo, Zhong Yanfei, Guan Qingfeng, Lin Weihua, Zhang Liangpei, and Li Deren. 2021. A spectral-spatial-dependent global learning framework for insufficient and imbalanced hyperspectral image classification. IEEE Transactions on Cybernetics (2021).Google ScholarGoogle Scholar
  53. [53] Zou Liang, Zhu Xingliang, Wu Changfeng, Liu Yong, and Qu Lei. 2020. A spectral-spatial-dependent global learning framework for insufficient and imbalanced hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13 (2020), 659674.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Boosting Hyperspectral Image Classification with Dual Hierarchical Learning

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 1
      January 2023
      505 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3572858
      • Editor:
      • Abdulmotaleb El Saddik
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 5 January 2023
      • Online AM: 12 March 2022
      • Accepted: 24 February 2022
      • Revised: 22 November 2021
      • Received: 23 April 2021
      Published in tomm Volume 19, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!