skip to main content
research-article

Context Prior Guided Semantic Modeling for Biomedical Image Segmentation

Published:15 March 2023Publication History
Skip Abstract Section

Abstract

Most state-of-the-art deep networks proposed for biomedical image segmentation are developed based on U-Net. While remarkable success has been achieved, its inherent limitations hinder it from yielding more precise segmentation. First, its receptive field is limited due to the fixed kernel size, which prevents the network from modeling global context information. Second, when spatial information captured by shallower layer is directly transmitted to higher layers by skip connections, the process inevitably introduces noise and irrelevant information to feature maps and blurs their semantic meanings. In this article, we propose a novel segmentation network equipped with a new context prior guidance (CPG) module to overcome these limitations for biomedical image segmentation, namely context prior guidance network (CPG-Net). Specifically, we first extract a set of context priors under the supervision of a coarse segmentation and then employ these context priors to model the global context information and bridge the spatial-semantic gap between high-level features and low-level features. The CPG module contains two major components: context prior representation (CPR) and semantic complement flow (SCF). CPR is used to extract pixels belonging to the same objects and hence produce more discriminative features to distinguish different objects. We further introduce deep semantic information for each CPR by the SCF mechanism to compensate the semantic information diluted during the decoding. We extensively evaluate the proposed CPG-Net on three famous biomedical image segmentation tasks with diverse imaging modalities and semantic environments. Experimental results demonstrate the effectiveness of our network, consistently outperforming state-of-the-art segmentation networks in all the three tasks. Codes are available at https://github.com/zzw-szu/CPGNet.

REFERENCES

  1. [1] Alom Md Zahangir, Hasan Mahmudul, Yakopcic Chris, Taha Tarek M., and Asari Vijayan K.. 2018. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:1802.06955. Retrieved from https://arxiv.org/abs/1802.06955.Google ScholarGoogle Scholar
  2. [2] Caicedo Juan C., Goodman Allen, Karhohs Kyle W., Cimini Beth A., Ackerman Jeanelle, Haghighi Marzieh, Heng CherKeng, Becker Tim, Doan Minh, McQuin Claire, et al. 2019. Nucleus segmentation across imaging experiments: The 2018 data science bowl. Nat. Methods 16, 12 (2019), 12471253.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Chen Liang-Chieh, Papandreou George, Kokkinos Iasonas, Murphy Kevin, and Yuille Alan L.. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2017), 834848.Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Chen Liang-Chieh, Papandreou George, Kokkinos Iasonas, Murphy Kevin, and Yuille Alan L.. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2017), 834848.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Çiçek Özgün, Abdulkadir Ahmed, Lienkamp Soeren S., Brox Thomas, and Ronneberger Olaf. 2016. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 424432.Google ScholarGoogle Scholar
  6. [6] Codella Noel, Rotemberg Veronica, Tschandl Philipp, Celebi M. Emre, Dusza Stephen, Gutman David, Helba Brian, Kalloo Aadi, Liopyris Konstantinos, Marchetti Michael, et al. 2019. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv:1902.03368. Retrieved from https://arxiv.org/abs/1902.03368.Google ScholarGoogle Scholar
  7. [7] Crum William R., Camara Oscar, and Hill Derek L. G.. 2006. Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans. Med. Imag. 25, 11 (2006), 14511461.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Ding Henghui, Jiang Xudong, Shuai Bing, Liu Ai Qun, and Wang Gang. 2018. Context contrasted feature and gated multi-scale aggregation for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 23932402.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Ding Henghui, Jiang Xudong, Shuai Bing, Liu Ai Qun, and Wang Gang. 2019. Semantic correlation promoted shape-variant context for segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 88858894.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Ding Henghui, Jiang Xudong, Shuai Bing, Liu Ai Qun, and Wang Gang. 2020. Semantic segmentation with context encoding and multi-path decoding. IEEE Trans. Image Process. 29 (2020), 35203533.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Feng Shuanglang, Zhao Heming, Shi Fei, Cheng Xuena, Wang Meng, Ma Yuhui, Xiang Dehui, Zhu Weifang, and Chen Xinjian. 2020. CPFNet: Context pyramid fusion network for medical image segmentation. IEEE Trans. Med. Imag. 39, 10 (2020), 30083018.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Fu Jun, Liu Jing, Tian Haijie, Li Yong, Bao Yongjun, Fang Zhiwei, and Lu Hanqing. 2019. Dual attention network for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 31463154.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Gu Zaiwang, Cheng Jun, Fu Huazhu, Zhou Kang, Hao Huaying, Zhao Yitian, Zhang Tianyang, Gao Shenghua, and Liu Jiang. 2019. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans. Med. Imag. 38, 10 (2019), 22812292.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770778.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Ibtehaz Nabil and Rahman M. Sohel. 2020. MultiResUNet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neur. Netw. 121 (2020), 7487.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Isensee Fabian, Jaeger Paul F., Kohl Simon A. A., Petersen Jens, and Maier-Hein Klaus H.. 2021. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 2 (2021), 203211.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Jaeger Stefan, Candemir Sema, Antani Sameer, Wáng Yì-Xiáng J., Lu Pu-Xuan, and Thoma George. 2014. Two public chest x-ray datasets for computer-aided screening of pulmonary diseases. Quant. Imag. Med. Surg. 4, 6 (2014), 475.Google ScholarGoogle Scholar
  18. [18] Kampffmeyer Michael, Dong Nanqing, Liang Xiaodan, Zhang Yujia, and Xing Eric P.. 2018. ConnNet: A long-range relation-aware pixel-connectivity network for salient segmentation. IEEE Trans. Image Process. 28, 5 (2018), 25182529.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Kingma Diederik P. and Ba Jimmy. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference for Learning Representations (ICLR’15).Google ScholarGoogle Scholar
  20. [20] Li Xiangtai, Zhao Houlong, Han Lei, Tong Yunhai, Tan Shaohua, and Yang Kuiyuan. 2020. Gated fully fusion for semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 1141811425.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Lin Guosheng, Milan Anton, Shen Chunhua, and Reid Ian. 2017. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 19251934.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Lin Guosheng, Milan Anton, Shen Chunhua, and Reid Ian. 2017. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 19251934.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Liu Jiang-Jiang, Hou Qibin, Cheng Ming-Ming, Feng Jiashi, and Jiang Jianmin. 2019. A simple pooling-based design for real-time salient object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 39173926.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Long Jonathan, Shelhamer Evan, and Darrell Trevor. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 34313440.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Milletari Fausto, Navab Nassir, and Ahmadi Seyed-Ahmad. 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the 4th International Conference on 3D Vision (3DV’16). IEEE, 565571.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Oktay Ozan, Schlemper Jo, Folgoc Loic Le, Lee Matthew, Heinrich Mattias, Misawa Kazunari, Mori Kensaku, McDonagh Steven, Hammerla Nils Y., Kainz Bernhard, et al. 2018. Attention u-net: Learning where to look for the pancreas. arXiv:1804.03999. Retrieved from https://arxiv.org/abs/1804.03999.Google ScholarGoogle Scholar
  27. [27] Peng Chao, Zhang Xiangyu, Yu Gang, Luo Guiming, and Sun Jian. 2017. Large kernel matters–improve semantic segmentation by global convolutional network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 43534361.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Punn Narinder Singh and Agarwal Sonali. 2020. Inception u-net architecture for semantic segmentation to identify nuclei in microscopy cell images. ACM Trans. Multimedia Comput. Commun. Appl. 16, 1 (2020), 115.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Rajput Amitesh Singh, Tanwar Vishesh Kumar, and Raman Balasubramanian. 2021. -Score-based secure biomedical model for effective skin lesion segmentation over ehealth cloud. ACM Trans. Multimedia Comput. Commun. Appl. 17, 2s (2021), 119.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Ren Xuhua, Ahmad Sahar, Zhang Lichi, Xiang Lei, Nie Dong, Yang Fan, Wang Qian, and Shen Dinggang. 2020. Task decomposition and synchronization for semantic biomedical image segmentation. IEEE Trans. Image Process. 29 (2020), 74977510.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Ronneberger Olaf, Fischer Philipp, and Brox Thomas. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 234241.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Shen Zhiqiang, Liu Zhuang, Li Jianguo, Jiang Yu-Gang, Chen Yurong, and Xue Xiangyang. 2019. Object detection from scratch with deep supervision. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2 (2019), 398412.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Soltaninejad Mohammadreza, Sturrock Craig J., Griffiths Marcus, Pridmore Tony P., and Pound Michael P.. 2020. Three dimensional root CT segmentation using multi-resolution encoder-decoder networks. IEEE Trans. Image Process. 29 (2020), 66676679.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Szegedy Christian, Ioffe Sergey, Vanhoucke Vincent, and Alemi Alexander. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Tanwar Vishesh Kumar, Raman Balasubramanian, Rajput Amitesh Singh, and Bhargava Rama. 2020. CryptoLesion: A privacy-preserving model for lesion segmentation using whale optimization over cloud. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 123.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Tschandl Philipp, Rosendahl Cliff, and Kittler Harald. 2018. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 1 (2018), 19.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Łukasz, and Polosukhin Illia. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 59986008.Google ScholarGoogle Scholar
  38. [38] Wang Xiaolong, Girshick Ross, Gupta Abhinav, and He Kaiming. 2018. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 77947803.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Wang Xiaohong, Jiang Xudong, Ding Henghui, and Liu Jun. 2019. Bi-directional dermoscopic feature learning and multi-scale consistent decision fusion for skin lesion segmentation. IEEE Trans. Image Process. 29 (2019), 30393051.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Wu Huisi, Pan Junquan, Li Zhuoying, Wen Zhenkun, and Qin Jing. 2020. Automated skin lesion segmentation via an adaptive dual attention module. IEEE Trans. Med. Imag. 40, 1 (2020), 357370.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Yang Zhenzhen, Xu Pengfei, Yang Yongpeng, and Bao Bing-Kun. 2021. A densely connected network based on u-net for medical image segmentation. ACM Trans. Multimedia Comput. Commun. Appl. 17, 3 (2021), 114.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Yu Lequan, Chen Hao, Dou Qi, Qin Jing, and Heng Pheng-Ann. 2016. Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans. Med. Imag. 36, 4 (2016), 9941004.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Zhang Fan, Chen Yanqin, Li Zhihang, Hong Zhibin, Liu Jingtuo, Ma Feifei, Han Junyu, and Ding Errui. 2019. Acfnet: Attentional class feature network for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision. 67986807.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Zhang Hang, Dana Kristin, Shi Jianping, Zhang Zhongyue, Wang Xiaogang, Tyagi Ambrish, and Agrawal Amit. 2018. Context encoding for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 71517160.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Zhang Zhijie, Fu Huazhu, Dai Hang, Shen Jianbing, Pang Yanwei, and Shao Ling. 2019. Et-net: A generic edge-attention guidance network for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 442450.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Zhang Zhilu and Sabuncu Mert. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neural Information Processing Systems. 87788788.Google ScholarGoogle Scholar
  47. [47] Zhao Hengshuang, Shi Jianping, Qi Xiaojuan, Wang Xiaogang, and Jia Jiaya. 2017. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 28812890.Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Zhou Sihang, Nie Dong, Adeli Ehsan, Yin Jianping, Lian Jun, and Shen Dinggang. 2019. High-resolution encoder–decoder networks for low-contrast medical image segmentation. IEEE Trans. Image Process. 29 (2019), 461475.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Zhou Zongwei, Siddiquee Md Mahfuzur Rahman, Tajbakhsh Nima, and Liang Jianming. 2019. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imag. 39, 6 (2019), 18561867.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Zhu Zhen, Xu Mengde, Bai Song, Huang Tengteng, and Bai Xiang. 2019. Asymmetric non-local neural networks for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision. 593602.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Context Prior Guided Semantic Modeling for Biomedical Image Segmentation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 2s
      April 2023
      545 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3572861
      • Editor:
      • Abdulmotaleb El Saddik
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 March 2023
      • Online AM: 25 August 2022
      • Accepted: 12 August 2022
      • Revised: 1 July 2022
      • Received: 25 October 2021
      Published in tomm Volume 19, Issue 2s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
    • Article Metrics

      • Downloads (Last 12 months)188
      • Downloads (Last 6 weeks)21

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!