skip to main content
research-article

Multimodal Brain Tumor Segmentation Based on an Intelligent UNET-LSTM Algorithm in Smart Hospitals

Authors Info & Claims
Published:16 June 2021Publication History
Skip Abstract Section

Abstract

Smart hospitals are important components of smart cities. An intelligent medical system for brain tumor segmentation is required to construct smart hospitals. To achieve intelligent brain tumor segmentation, morphological variety and serious category imbalance must be managed effectively. Conventional deep neural networks have difficulty in predicting high-accuracy segmentation images due to these issues. To solve these problems, we propose using multimodal brain tumor images combined with the UNET and LSTM models to construct a new network structure with a mixed loss function to solve sample imbalance and describe an intelligent segmentation process to identify brain tumors. To verify the practicability of this algorithm, we used the open source Brain Tumor Segmentation Challenge dataset to train and verify the proposed network. We obtained DSCs of 0.91, 0.82, and 0.80; sensitivities of 0.93, 0.85, and 0.82; and specificities of 0.99, 0.99, and 0.98 in three tumor regions, including the whole tumor (WT), tumor core (TC), and enhanced tumor (ET). We also compared the results of the proposed network with those of other brain tumor segmentation methods, and the results showed that the proposed algorithm could segment different tumor lesions more accurately, highlighting its potential application value in the clinical diagnosis of brain tumors.

References

  1. W. J. Niessen. 2016. MR brain image analysis in dementia: From quantitative imaging biomarkers to ageing brain models and imaging genetics. Medical Image Analysis 33 (2016), 107–113.Google ScholarGoogle ScholarCross RefCross Ref
  2. R. Lu, L. Qiang, G. Xin, et al. 2019. Three-dimensional segmentation of brain tumors in magnetic resonance imaging based on improved continuous max-flow. Laser & Optoelectronics Progress 55, 11 (2019), 111011.Google ScholarGoogle Scholar
  3. K. J. Shanthi and M. S. Kumar. 2007. Skull stripping and automatic segmentation of brain MRI using seed growth and threshold techniques. In 2007 International Conference on Intelligent and Advanced Systems. IEEE, 422–426.Google ScholarGoogle Scholar
  4. M. Kaleem, M. Sanaullah, M. A. Hussain, et al. 2012. Segmentation of brain tumor tissue using marker controlled watershed transform method. In International Multi Topic Conference. Springer, Berlin, Heidelberg, 222–227.Google ScholarGoogle ScholarCross RefCross Ref
  5. M. R. Kaus, S. K. Warfield, A. Nabavi, et al. 2001. Automated segmentation of MR images of brain tumors. Radiology 218, 2 (2001), 586–591.Google ScholarGoogle ScholarCross RefCross Ref
  6. D. Zikic, Y. Ioannou, M. Brown, et al. 2014. Segmentation of brain tumor tissues with convolutional neural networks. Proceedings MICCAI-BRATS (2014), 36–39.Google ScholarGoogle Scholar
  7. Y. Li, F. Jia, and J. Qin. 2016. Brain tumor segmentation from multimodal magnetic resonance images via sparse representation. Artificial Intelligence in Medicine 73 (2016), 1–13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. S. Pereira, A. Pinto, V. Alves, et al. 2016. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging 35, 5 (2016), 1240–1251.Google ScholarGoogle ScholarCross RefCross Ref
  9. D. Zikic, Y. Ioannou, M. Brown, et al. 2014. Segmentation of brain tumor tissues with convolutional neural networks. Proceedings MICCAI-BRATS (2014), 36–39.Google ScholarGoogle Scholar
  10. P. D. Chang. 2016. Fully convolutional neural networks with hyperlocal features for brain tumor segmentation. In Proceedings MICCAI-BRATS Workshop. 4–9.Google ScholarGoogle Scholar
  11. L. Zhao and K. Jia. 2016. Multiscale CNNs for brain tumor segmentation and diagnosis. Computational and Mathematical Methods in Medicine 2016 (2016).Google ScholarGoogle Scholar
  12. K. Kamnitsas, C. Ledig, V. F. J. Newcombe, et al. 2017. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 36 (2017), 61–78.Google ScholarGoogle ScholarCross RefCross Ref
  13. O. Ronneberger, P. Fischer, and T. Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 234–241.Google ScholarGoogle Scholar
  14. M. D. Zeiler, and R. Fergus. 2014. Visualizing and understanding convolutional networks. In European Conference on Computer Vision. Springer, Cham, 818–833.Google ScholarGoogle Scholar
  15. Q. Dou, L. Yu, H. Chen, et al. 2017. 3D deeply supervised network for automated segmentation of volumetric medical images. Medical Image Analysis 41 (2017), 40–54.Google ScholarGoogle ScholarCross RefCross Ref
  16. B. Yu, L. Zhou, L. Wang, et al. 2018. 3D cGAN based cross-modality MR image synthesis for brain tumor segmentation. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI’18). IEEE, 626–630.Google ScholarGoogle ScholarCross RefCross Ref
  17. W. Bao, J. Yue, and Y. Rao. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PLoS One 12, 7 (2017), e0180944.Google ScholarGoogle Scholar
  18. F. A. Gers, N. N. Schraudolph, and J. Schmidhuber. 2002. Learning precise timing with LSTM recurrent networks. Journal of Machine Learning Research 3 (Aug. 2002), 115–143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. F. A. Gers, J. Schmidhuber, and F. Cummins. 1999. Learning to forget: Continual prediction with LSTM. In 9th International Conference on Artificial Neural Networks, 850–855. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Gulli, and S. Pal. 2017. Deep Learning with Keras. Packt Publishing Ltd. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. B. H. Menze, A. Jakab, S. Bauer, et al. 2014. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging 34, 10 (2014), 1993–2024.Google ScholarGoogle ScholarCross RefCross Ref
  22. S. Bakas, H. Akbari, A. Sotiras, et al. 2017. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data 4 (2017), 170117.Google ScholarGoogle ScholarCross RefCross Ref
  23. S. Bakas, M. Reyes, A. Jakab, et al. 2018. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. https://arxiv.org/abs/1811.02629 Retrived November, 2018.Google ScholarGoogle Scholar
  24. B. Kayalibay, G. Jensen, and P. van der Smagt. 2017. CNN-based segmentation of medical imaging data. https://arxiv.org/abs/1701.03056 Retrived January, 2017.Google ScholarGoogle Scholar
  25. M. Havaei, A. Davy, D. Warde-Farley, et al. 2017. Brain tumor segmentation with deep neural networks. Medical Image Analysis 35 (2017), 18–31.Google ScholarGoogle ScholarCross RefCross Ref
  26. C. Zhang, X. Shen, H. Cheng, et al. 2019. Brain tumor segmentation based on hybrid clustering and morphological operations. International Journal of Biomedical Imaging 2019 (2019).Google ScholarGoogle Scholar
  27. Cheng'en He, Huijun Xu, Zhong Wang, and Ma Liping. 2020. Automatic segmentation algorithm for multimodal magnetic resonance-based brain tumor images. Acta Optica Sinica 40, 6 (2020), 0610001.Google ScholarGoogle ScholarCross RefCross Ref
  28. Konstantinos Kamnitsas et al. 2017. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 36 (2017), 61–78.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Multimodal Brain Tumor Segmentation Based on an Intelligent UNET-LSTM Algorithm in Smart Hospitals

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Internet Technology
      ACM Transactions on Internet Technology  Volume 21, Issue 3
      August 2021
      522 pages
      ISSN:1533-5399
      EISSN:1557-6051
      DOI:10.1145/3468071
      • Editor:
      • Ling Liu
      Issue’s Table of Contents

      Copyright © 2021 Association for Computing Machinery.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 16 June 2021
      • Revised: 1 February 2021
      • Accepted: 1 February 2021
      • Received: 1 August 2020
      Published in toit Volume 21, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!