skip to main content
research-article

An Explainable Deep Learning Ensemble Model for Robust Diagnosis of Diabetic Retinopathy Grading

Authors Info & Claims
Published:26 October 2021Publication History
Skip Abstract Section

Abstract

Diabetic retinopathy (DR) is one of the most common causes of vision loss in people who have diabetes for a prolonged period. Convolutional neural networks (CNNs) have become increasingly popular for computer-aided DR diagnosis using retinal fundus images. While these CNNs are highly reliable, their lack of sufficient explainability prevents them from being widely used in medical practice. In this article, we propose a novel explainable deep learning ensemble model where weights from different models are fused into a single model to extract salient features from various retinal lesions found on fundus images. The extracted features are then fed to a custom classifier for the final diagnosis of DR severity level. The model is trained on an APTOS dataset containing retinal fundus images of various DR grades using a cyclical learning rates strategy with an automatic learning rate finder for decaying the learning rate to improve model accuracy. We develop an explainability approach by leveraging gradient-weighted class activation mapping and shapely adaptive explanations to highlight the areas of fundus images that are most indicative of different DR stages. This allows ophthalmologists to view our model's decision in a way that they can understand. Evaluation results using three different datasets (APTOS, MESSIDOR, IDRiD) show the effectiveness of our model, achieving superior classification rates with a high degree of precision (0.970), sensitivity (0.980), and AUC (0.978). We believe that the proposed model, which jointly offers state-of-the-art diagnosis performance and explainability, will address the black-box nature of deep CNN models in robust detection of DR grading.

REFERENCES

  1. [1] Jeany Q. L., Welchowski T., Schmid M., Letow J., I. Wolpers, F. Pascual-Camps, F. G. Holz, and R. P. Finger. 2020. Prevalence, incidence and future projection of diabetic eye disease in Europe: A systematic review and meta-analysis. European Journal of Epidemiology 35, 1 (2020), 1123.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Cheung N., Wong T. Y., and Mitchell P.. 2010. Diabetic retinopathy. Lancet 376, 9735 (2010), 124136.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Carson L., Darvin Y., Margaret G., and Tony L.. 2017. Automated detection of diabetic retinopathy using deep learning. In Proceedings of AMIA Joint Summits on Translational Science 2017, 147155.Google ScholarGoogle Scholar
  4. [4] Alyoubi W. L., Shalash W. M., and Abulkhair M. F.. 2020. Diabetic retinopathy detection through deep learning techniques: A review. Informatics in Medicine Unlocked 20 (2020), 100377.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Krause J., Gulshan V., Rahimy E., Karth P., Widner K., Corrado G. S., Peng L., and Webster D. R.. 2017. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy, 2017, CoRR, abs/1710.01711.Google ScholarGoogle Scholar
  6. [6] Rahman M. A., Hossain M. S., Alrajeh N. A., and Gupta B. B.. 2021. A multimodal, multimedia point-of-care deep learning framework for COVID-19 diagnosis. ACM Trans. Multimedia Comput. Commun. 17, 1s, Article 18 (2021), 24 pages. DOI: DOI: https://doi.org/10.1145/3421725Google ScholarGoogle Scholar
  7. [7] Pratt H., Coenen F., M.Broadbent D., Harding S. P., and Zheng Y.. 2016. Convolutional neural networks for diabetic retinopathy. Procedia Computer Science 90, 2016, 200205.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Singh A., Dhillon A., Kumar N., Hossain M. S., G. Muhammad, and M. Kumar. 2021. eDiaPredict: An ensemble-based framework for diabetes prediction. ACM Trans. Multimedia Comput. Commun. Appl. 17, 2s, Article 66 (2021), 26 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Wan S., Liang Y., and Y. Zhang. 2018. Deep convolutional neural networks for diabetic retinopathy detection by image classification. Computers & Electrical Engineering 72, (2018), 274–282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Polyak B. T. and Juditsky A. B.. 1992. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization 30, 4 (1992), 838855. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Smith L. N.. 2017. Cyclical learning rates for training neural networks. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision 464472.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] APTOS. 2019. APTOS 2019 blindness detection. https://www.kaggle.com/c/aptos2019-blindness-detectionLast. accessed October 25, 2020.Google ScholarGoogle Scholar
  13. [13] Decencire E., Zhang X., Cazuguel G., Lay B., Cochener B., Trone C., Gain P., Ordonez R., Massin P., Erginay A., Charton B., and Klein J.-C.. 2014. Feedback on a publicly distributed database: The Messidor database. Image Analysis & Stereology 33, 3 (2014), 231234.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Porwal Prasanna, Pachade Samiksha, Manesh Kokare, Girish Deshmukh, Jaemin Son, Woong Bae, Lihong Liu, Jianzong Wang, Xinhui Liu, Liangxin Gao, TianBo Wu, Jing Xiao, Fengyan Wang, Baocai Yin, Yunzhi Wang, Gopichandh Danala, Linsheng He, Yoon Ho Choi, Yeong Chan Lee, Sang-Hyuk Jung, Zhongyu Li, Xiaodan Sui, Junyan Wu, Xiaolong Li, Ting Zhou, Janos Toth, Agnes Baran, Avinash Kori, Sai Saketh Chennamsetty, Mohammed Safwan, Varghese Alex, Xingzheng Lyu, Li Cheng, Qinhao Chu, Pengcheng Li, Xin Ji, Sanyuan Zhang, Yaxin Shen, Ling Dai, Oindrila Saha, Rachana Sathish, Tânia Melo, Teresa Araújo, Balazs Harangi, Bin Sheng, Ruogu Fang, Debdoot Sheet, Andras Hajdu, Yuanjie Zheng, Ana Maria Mendonça, Shaoting Zhang, Aurélio Campilho, Bin Zheng, Dinggang Shen, Luca Giancardo, Gwenolé Quellec, and Fabrice Mériaudeau. 2020. IDRiD: diabetic retinopathy – Segmentation and grading challenge. Med. Image Anal. 59 (2020), 101561.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Hossain M. S., Amin S. U., Muhammad G., and Al Sulaiman M.. 2019. Applying deep learning for epilepsy seizure detection and brain mapping visualization. ACM Trans. Multimedia Comput. Commun. 15, 1s 10 (2019), 17 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Muhammad G., Hossain M. S., and N. Kumar. 2021. EEG-Based pathology detection for home health monitoring. IEEE Journal on Selected Areas in Communications 39, 2 (2021), 603–610. DOI: 10.1109/JSAC.2020.3020654Google ScholarGoogle Scholar
  17. [17] Priya R. and Aruna P.. 2012. SVM and neural-network-based diagnosis of diabetic retinopathy. International Journal of Computer Applications 41, 1 (2012), 612.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Rahim S. S., Palade V., Shuttleworth J., and Jayne C.. 2014. Automatic screening and classification of diabetic retinopathy fundus images. In Proceedings of the International Conference on Communications in Computer and Information Science. 113122.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Conde P. P., J. de la Calleja, A. Benitez, and M. A. Medina. 2012. Image-based classification of diabetic retinopathy using machine learning. In Proceedings of the 12th International Conference on Intelligent Systems Design and Applications (ISDA). 826–830.Google ScholarGoogle Scholar
  20. [20] Colomer A., Igual J., and Naranjo V.. 2020. Detection of early signs of diabetic retinopathy based on textural and morphological information in fundus images. Sensors 2020, 20, 1005. DOI: DOI: 10.3390/s20041005Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Hossain M. S., Muhammad G., and Guizani N.. 2020. Explainable AI and mass surveillance system-based healthcare framework to combat COVID-I9 like pandemics. IEEE Network 34, 4 (2020), 126132.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Hossain M. S., M. Al-Hammadi, and G. Muhammad. 2019. Automatic fruit classification using deep learning for industrial applications. IEEE Transactions on Industrial Informatics 15, 2 (2019), 1027–1034.Google ScholarGoogle Scholar
  23. [23] Rajalakshmi R., S. Arulmalar, M. Usha, V. Prathiba, K. S. Kareemuddin, R. M. Anjana, and V. Mohan. 2015. Validation of smartphone based retinal photography for diabetic retinopathy screening. PLoS One 10, 9 (2015).Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Gulshan V.. 2016. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 22 (2016), 24022410.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Szegedy C., C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. 2016. Rethinking the inception architecture for computer vision. In Proc.of IEEE Conference on Computer Vision and Pattern Recognition 2016, 28182826.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Diabetic Retinopathy Detection. https://www.kaggle.com/c/diabetic-retinopathy-detection/data, Last accessed October 30, 2020.Google ScholarGoogle Scholar
  27. [27] Messidor-2 DR Grades. https://www.kaggle.com/google-brain/messidor2-dr-grades, Last accessed October 30, 2020.Google ScholarGoogle Scholar
  28. [28] Gargey R. and Leng T.. 2017. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 124, 7 (2017), 962969.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Li Y.-H., N.-N. Yeh, S.-J. Chen, and Y.-C. Chung. 2019. Computer-assisted diagnosis for diabetic retinopathy based on fundus images using deep convolutional neural network. Mobile Information Systems 2019, Article ID 6142839. https://doi.org/10.1155/2019/6142839Google ScholarGoogle Scholar
  30. [30] Hossain M. S.. 2017. Cloud-supported cyber–physical localization framework for patients monitoring. IEEE Systems Journal 11, 1 (2017), 118127.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Tymchenko B., Marchenko P., and Spodarets D.. 2020. Deep learning approach to diabetic retinopathy detection. arXiv:2003.02261 [cs.LG], 2020.Google ScholarGoogle Scholar
  32. [32] Hagos M. T. and Kant S.. 2019. Transfer learning based detection of diabetic retinopathy from small dataset. CoRR, abs/1905.07203.Google ScholarGoogle Scholar
  33. [33] Sarki R., Michalska S., K. Ahmed, H. Wang, and Y. Zhang. 2019. Convolutional neural networks for mild diabetic retinopathy detection: An experimental study. bioRxiv, 2019.Google ScholarGoogle Scholar
  34. [34] Tseng V. S., C.-L. Chen, C.-M. Liang, M.-C. Tai, J.-T. Liu, P.-Y. Wu, M.-S. Deng, Y.-W. Lee, T.-Y. Huang, and Y.-H. Chen. 2020. Leveraging multimodal deep learning architecture with retina lesion information to detect diabetic retinopathy. Translational Vision Science & Technology (TVST) 9, 2 (2020), 41. https://doi.org/10.1167/tvst.9.2.41Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Hacisoftaoglu R. E., Karakaya M., and Sallam A. B.. 2020. Deep learning frameworks for diabetic retinopathy detection with smartphone-based retinal imaging systems. Pattern Recognition Letters 135, (2020), 409417.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Mahiba C. and Jayachandran A.. 2019. Severity analysis of diabetic retinopathy in retinal images using hybrid structure descriptor and modified CNNs. Measurement 135 (2019), 762767.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Mateen M., Wen J., Nasrullah N., Sun S., and Hayat S.. 2020. Exudate detection for diabetic retinopathy using pretrained convolutional neural networks. Complexity 2020.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Chetoui M. and Akhloufi M. A.. 2020. Explainable diabetic retinopathy using efficientNET. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2020. DOI: DOI: 10.1109/EMBC44109.2020.9175664Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Kind A. and Azzopardi G.. 2019. An explainable AI-based computer aided detection system for diabetic retinopathy using retinal fundus images. CAIP (1) 2019, 457468.Google ScholarGoogle Scholar
  40. [40] Stolte S. and Fang R.. 2020. A survey on medical image analysis in diabetic retinopathy. Med Image Anal 64 101742.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Deng J., W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. International Conference on Computer Vision and Pattern Recognition (CVPR), 248255.Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Szegedy Christian, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–9.Google ScholarGoogle Scholar
  43. [43] Zhou B., Khosla A., Lapedriza A., Oliva A., and Torralba A.. 2016. Learning deep features for discriminative localization.In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, 29212929. DOI: DOI: 10.1109/CVPR.2016.319Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Selvaraju R. R., M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. 2019. Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision 128, 2 (2019), 336359.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Lundberg S. and Lee S. I.. 2017. A unified approach to interpreting model predictions. In Proceedings of Neural Information Processing Systems (NIPS). 47684777. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Kenstler B.. Cyclical Learning Rates Implementation. https://github.com/bckenstler/CLR.Google ScholarGoogle Scholar

Index Terms

  1. An Explainable Deep Learning Ensemble Model for Robust Diagnosis of Diabetic Retinopathy Grading

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3s
        October 2021
        324 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3492435
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 26 October 2021
        • Accepted: 1 June 2021
        • Revised: 1 May 2021
        • Received: 1 December 2020
        Published in tomm Volume 17, Issue 3s

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!