Abstract
Most state-of-the-art deep networks proposed for biomedical image segmentation are developed based on U-Net. While remarkable success has been achieved, its inherent limitations hinder it from yielding more precise segmentation. First, its receptive field is limited due to the fixed kernel size, which prevents the network from modeling global context information. Second, when spatial information captured by shallower layer is directly transmitted to higher layers by skip connections, the process inevitably introduces noise and irrelevant information to feature maps and blurs their semantic meanings. In this article, we propose a novel segmentation network equipped with a new context prior guidance (CPG) module to overcome these limitations for biomedical image segmentation, namely context prior guidance network (CPG-Net). Specifically, we first extract a set of context priors under the supervision of a coarse segmentation and then employ these context priors to model the global context information and bridge the spatial-semantic gap between high-level features and low-level features. The CPG module contains two major components: context prior representation (CPR) and semantic complement flow (SCF). CPR is used to extract pixels belonging to the same objects and hence produce more discriminative features to distinguish different objects. We further introduce deep semantic information for each CPR by the SCF mechanism to compensate the semantic information diluted during the decoding. We extensively evaluate the proposed CPG-Net on three famous biomedical image segmentation tasks with diverse imaging modalities and semantic environments. Experimental results demonstrate the effectiveness of our network, consistently outperforming state-of-the-art segmentation networks in all the three tasks. Codes are available at https://github.com/zzw-szu/CPGNet.
- [1] . 2018. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:1802.06955. Retrieved from https://arxiv.org/abs/1802.06955.Google Scholar
- [2] . 2019. Nucleus segmentation across imaging experiments: The 2018 data science bowl. Nat. Methods 16, 12 (2019), 1247–1253.Google Scholar
Cross Ref
- [3] . 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2017), 834–848.Google Scholar
Cross Ref
- [4] . 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2017), 834–848.Google Scholar
Cross Ref
- [5] . 2016. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 424–432.Google Scholar
- [6] . 2019. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv:1902.03368. Retrieved from https://arxiv.org/abs/1902.03368.Google Scholar
- [7] . 2006. Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans. Med. Imag. 25, 11 (2006), 1451–1461.Google Scholar
Cross Ref
- [8] . 2018. Context contrasted feature and gated multi-scale aggregation for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2393–2402.Google Scholar
Cross Ref
- [9] . 2019. Semantic correlation promoted shape-variant context for segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8885–8894.Google Scholar
Cross Ref
- [10] . 2020. Semantic segmentation with context encoding and multi-path decoding. IEEE Trans. Image Process. 29 (2020), 3520–3533.Google Scholar
Digital Library
- [11] . 2020. CPFNet: Context pyramid fusion network for medical image segmentation. IEEE Trans. Med. Imag. 39, 10 (2020), 3008–3018.Google Scholar
Cross Ref
- [12] . 2019. Dual attention network for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3146–3154.Google Scholar
Cross Ref
- [13] . 2019. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans. Med. Imag. 38, 10 (2019), 2281–2292.Google Scholar
Cross Ref
- [14] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google Scholar
Cross Ref
- [15] . 2020. MultiResUNet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neur. Netw. 121 (2020), 74–87.Google Scholar
Digital Library
- [16] . 2021. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 2 (2021), 203–211.Google Scholar
Cross Ref
- [17] . 2014. Two public chest x-ray datasets for computer-aided screening of pulmonary diseases. Quant. Imag. Med. Surg. 4, 6 (2014), 475.Google Scholar
- [18] . 2018. ConnNet: A long-range relation-aware pixel-connectivity network for salient segmentation. IEEE Trans. Image Process. 28, 5 (2018), 2518–2529.Google Scholar
Cross Ref
- [19] . 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference for Learning Representations (ICLR’15).Google Scholar
- [20] . 2020. Gated fully fusion for semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11418–11425.Google Scholar
Cross Ref
- [21] . 2017. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1925–1934.Google Scholar
Cross Ref
- [22] . 2017. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1925–1934.Google Scholar
Cross Ref
- [23] . 2019. A simple pooling-based design for real-time salient object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3917–3926.Google Scholar
Cross Ref
- [24] . 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3431–3440.Google Scholar
Cross Ref
- [25] . 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the 4th International Conference on 3D Vision (3DV’16). IEEE, 565–571.Google Scholar
Cross Ref
- [26] . 2018. Attention u-net: Learning where to look for the pancreas. arXiv:1804.03999. Retrieved from https://arxiv.org/abs/1804.03999.Google Scholar
- [27] . 2017. Large kernel matters–improve semantic segmentation by global convolutional network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4353–4361.Google Scholar
Cross Ref
- [28] . 2020. Inception u-net architecture for semantic segmentation to identify nuclei in microscopy cell images. ACM Trans. Multimedia Comput. Commun. Appl. 16, 1 (2020), 1–15.Google Scholar
Digital Library
- [29] . 2021. -Score-based secure biomedical model for effective skin lesion segmentation over ehealth cloud. ACM Trans. Multimedia Comput. Commun. Appl. 17, 2s (2021), 1–19.Google Scholar
Digital Library
- [30] . 2020. Task decomposition and synchronization for semantic biomedical image segmentation. IEEE Trans. Image Process. 29 (2020), 7497–7510.Google Scholar
Digital Library
- [31] . 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 234–241.Google Scholar
Cross Ref
- [32] . 2019. Object detection from scratch with deep supervision. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2 (2019), 398–412.Google Scholar
Cross Ref
- [33] . 2020. Three dimensional root CT segmentation using multi-resolution encoder-decoder networks. IEEE Trans. Image Process. 29 (2020), 6667–6679.Google Scholar
Cross Ref
- [34] . 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.Google Scholar
Cross Ref
- [35] . 2020. CryptoLesion: A privacy-preserving model for lesion segmentation using whale optimization over cloud. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 1–23.Google Scholar
Digital Library
- [36] . 2018. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 1 (2018), 1–9.Google Scholar
Cross Ref
- [37] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998–6008.Google Scholar
- [38] . 2018. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7794–7803.Google Scholar
Cross Ref
- [39] . 2019. Bi-directional dermoscopic feature learning and multi-scale consistent decision fusion for skin lesion segmentation. IEEE Trans. Image Process. 29 (2019), 3039–3051.Google Scholar
Digital Library
- [40] . 2020. Automated skin lesion segmentation via an adaptive dual attention module. IEEE Trans. Med. Imag. 40, 1 (2020), 357–370.Google Scholar
Cross Ref
- [41] . 2021. A densely connected network based on u-net for medical image segmentation. ACM Trans. Multimedia Comput. Commun. Appl. 17, 3 (2021), 1–14.Google Scholar
Digital Library
- [42] . 2016. Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans. Med. Imag. 36, 4 (2016), 994–1004.Google Scholar
Cross Ref
- [43] . 2019. Acfnet: Attentional class feature network for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision. 6798–6807.Google Scholar
Cross Ref
- [44] . 2018. Context encoding for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7151–7160.Google Scholar
Cross Ref
- [45] . 2019. Et-net: A generic edge-attention guidance network for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 442–450.Google Scholar
Digital Library
- [46] . 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neural Information Processing Systems. 8778–8788.Google Scholar
- [47] . 2017. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2881–2890.Google Scholar
Cross Ref
- [48] . 2019. High-resolution encoder–decoder networks for low-contrast medical image segmentation. IEEE Trans. Image Process. 29 (2019), 461–475.Google Scholar
Digital Library
- [49] . 2019. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imag. 39, 6 (2019), 1856–1867.Google Scholar
Cross Ref
- [50] . 2019. Asymmetric non-local neural networks for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision. 593–602.Google Scholar
Cross Ref
Index Terms
Context Prior Guided Semantic Modeling for Biomedical Image Segmentation
Recommendations
Anatomical Model-Guided Segmentation of Brain MR Image Sequences
ICVGIP '14: Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image ProcessingThis paper presents a method for segmentation of human brain magnetic resonance (MR) image sequences based on a 3D human brain model (triangulated mesh). The brain model is composed of four components, namely, cerebrum, cerebellum, brain stem and ...
Interactive Lung Segmentation Algorithm for CT Chest Images Based on Live-Wire Model and Snake Model
ICECT '09: Proceedings of the 2009 International Conference on Electronic Computer TechnologyIn this paper, an interactive lung parenchyma segmentation algorithm is put forward with improved Live-Wire model, Snake model and contour interpolation, which takes full advantage of lung contours' slow change in adjacent CT image layers and operators' ...
Prostate Segmentation with Texture Enhanced Active Appearance Model
SITIS '10: Proceedings of the 2010 Sixth International Conference on Signal-Image Technology and Internet Based SystemsProstate contour segmented from Trans Rectal Ultra Sound (TRUS) and Magnetic Resonance (MR) images could improve inter-modality registration accuracy and reduce computational complexity of the procedure. However, prostate segmentation in each of these ...






Comments