Abstract
In this article, we share our experiences in designing and developing a suite of deep neural network–(DNN) based COVID-19 case detection and recognition framework. Existing pathological tests such as RT-PCR-based pathogen RNA detection from nasal swabbing seem to display low detection rates during the early stages of virus contraction. Moreover, the reliance on a few overburdened laboratories based around an epicenter capable of supplying large numbers of RT-PCR tests makes this testing method non-scalable when the rate of infections is high. Similarly, finding an effective drug or vaccine with which to combat COVID-19 requires a long time and many clinical trials. The development of pathological COVID-19 tests is hindered by shortages in the supply chain of chemical reagents necessary for testing on a large scale. This diminishes the speed of diagnosis and the ability to filter out COVID-19 positive patients from uninfected patients on a national level. Existing research has shown that DNN has been successful in identifying COVID-19 from radiological media such as CT scans and X-ray images, audio media such as cough sounds, optical coherence tomography to identify conjunctivitis and pink eye symptoms on the ocular surface, body temperature measurement using smartphone fingerprint sensors or thermal cameras, the use of live facial detection to identify safe social distancing practices from camera images, and face mask detection from camera images. We also investigate the utility of federated learning in diagnosis cases where private data can be trained via edge learning. These point-of-care modalities can be integrated with DNN-based RT-PCR laboratory test results to assimilate multiple modalities of COVID-19 detection and thereby provide more dimensions of diagnosis. Finally, we will present our initial test results, which are encouraging.
- M. S. Hossain, S. U. Amin, M. Alsulaiman, and G. Muhammad. 2019. Applying deep learning for epilepsy seizure detection and brain mapping visualization. ACM Trans. Multimed. Comput. Commun. Appl 15, 1 (2019), 1–17. DOI:10.1145/3241056 Google Scholar
Digital Library
- M. A. Rahman and M. S. Hossain. 2021. An internet of medical things-enabled edge computing framework for tackling COVID-19. IEEE IoT J. DOI:10.1109/JIOT.2021.3051080Google Scholar
- T. Phan. 2020. Novel coronavirus: From discovery to clinical diagnostics. Infect. Genet. Evol. 79 (2020), 104211. DOI:10.1016/j.meegid.2020.104211Google Scholar
Cross Ref
- M. Shen et al. 2020. Recent advances and perspectives of nucleic acid detection for coronavirus. J. Pharm. Anal. 10, 2 (2020), 97--101. DOI:10.1016/j.jpha.2020.02.010Google Scholar
Cross Ref
- M. Shorfuzzaman and M. S. Hossain. 2021. MetaCOVID: A Siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients. Pattern Recognition 113 (2021), 107700. DOI:10.1016/j.patcog.2020.107700Google Scholar
Cross Ref
- M. Shorfuzzaman, M. S. Hossain, and M. F. Alhamid. 2021. Towards the sustainable development of smart cities through mass video surveillance: A response to the COVID-19 pandemic. Sustainable Cities and Society 64 (2021), 102582. DOI:10.1016/j.scs.2020.102582Google Scholar
Cross Ref
- C. Biscayart, P. Angeleri, S. Lloveras, T. do S. S. Chaves, P. Schlagenhauf, and A. J. Rodríguez-Morales. 2020. The next big threat to global health? 2019 novel coronavirus (2019-nCoV): What advice can we give to travellers? Interim recommendations January 2020, from the Latin-American society for Travel Medicine (SLAMVI). Travel Med. Infect. Dis. 33 (2020), 17–20. DOI:10.1016/j.tmaid.2020.101567Google Scholar
Cross Ref
- M. Hoffmann et al. 2020. SARS-CoV-2cell entry depends on ACE2 and TMPRSS2 and is blocked by a 1clinically-proven protease inhibitor. Cell (2020), 1–10. DOI:10.1016/j.cell.2020.02.052Google Scholar
- X. Liu and X.-J. Wang. 2020. Potential inhibitors against 2019-ncov coronavirus m protease from clinically approved medicines. J. Genet. Genom. (2020). DOI:10.1016/j.jgg.2020.02.001Google Scholar
- X. Li, M. Geng, Y. Peng, L. Meng, and S. Lu. 2020. Molecular immune pathogenesis and diagnosis of COVID-19. J. Pharm. Anal. (2020). DOI:10.1016/j.jpha.2020.03.001Google Scholar
- J. Chen. 2020. Pathogenicity and transmissibility of 2019-nCoV—A quick overview and comparison with other emerging viruses. Microbes Infect. 2019—2021. DOI:10.1016/j.micinf.2020.01.004Google Scholar
- A. K. Cordes and A. Heim. 2020. Rapid random access detection of the novel SARS-coronavirus-2 (SARS-CoV-2, previously 2019-nCoV) using an open access protocol for the panther fusion. J. Clin. Virol 125 (2020), 104305. DOI:10.1016/j.jcv.2020.104305Google Scholar
Cross Ref
- H. Shi et al. 2020. Radiological findings from 81 patients with COVID-19 pneumonia in wuhan, china: A descriptive study. Lancet. Infect. Dis. 3099, 20 (2020), 1–10.Google Scholar
- R. Lu et al. 2020. Genomic characterisation and epidemiology of 2019 novel coronavirus: Implications for virus origins and receptor binding. Lancet 395, 10224 (2020), 565–574. DOI:10.1016/S0140-6736(20)30251-8Google Scholar
Cross Ref
- J. A. Al-tawfiq, A. H. Al-homoud, and Z. A. Memish. 2020. Remdesivir as a possible therapeutic option for the COVID-19. Travel Med. Infect. Dis. (March 2020), 101615. DOI:10.1016/j.tmaid.2020.101615Google Scholar
- A. Lopez-Rincon, A. Tonda, L. Mendoza-Maldonado, E. Claassen, J. Garssen, and A. D. Kraneveld. 2020. Accurate identification of SARS-CoV-2 from viral genome sequences using deep learning. bioRxiv, 2020.03.13.990242Google Scholar
- C. Huang, P. Lee, and P. Hsueh. 2020. Arguments in favor of remdesivir for treating SARS-CoV-2 infections. Int. J. Antimicrob. Agents (2020) 105933. DOI:10.1016/j.ijantimicag.2020.105933Google Scholar
- G. S. Randhawa, M. P. M. Soltysiak, H. El Roz, C. P. E. de Souza, K. A. Hill, and L. Kari. 2020. Machine learning using intrinsic genomic signatures for rapid classification of novel pathogens: COVID-19 case study. bioRxiv. 2020.02.03.932350. DOI:10.1101/2020.02.03.932350Google Scholar
- K. C. Santosh. 2020. AI-driven tools for coronavirus outbreak: Need of active learning and cross-population train/test models on multitudinal/multimodal data. J. Med. Syst. 44, 5 (2020), 93. DOI:10.1007/s10916-020-01562-1Google Scholar
Digital Library
- H. Hou et al. 2020. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. RSNA Radiol. (2020).Google Scholar
- M. S. Hossain and G. Muhammad. 2019. Emotion recognition using deep learning approach from audio-visual emotional big data. Inf. Fus. 49 (2019), 69–78.Google Scholar
Cross Ref
- Y. Song et al. 2020. Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images. MedRxiv. 2020.02.23.20026930. DOI:10.1101/2020.02.23.20026930Google Scholar
- S. K. Sundararajan and S. P. D. 2020. Detection of conjunctivitis with deep learning algorithm in medical image processing. In Proceedings of the 3rd International Conference on the Internet of Things in Social, Mobile, Analytics and Cloud (I-SMAC’20). 714–717. DOI:10.1109/i-smac47947.2019.9032705Google Scholar
- M. A. Rahman et al. 2020. Secure and provenance enhanced Internet of health things framework: A blockchain managed federated learning approach. IEEE Access 8 (2020), 205071--205087. DOI:10.1109/ACCESS.2020.3037474Google Scholar
Cross Ref
- G. Muhammad, M. S. Hossain, and N. Kumar. 2021. EEG-Based pathology detection for home health monitoring. IEEE J. Sel. Areas Commun. 39, 2 (2021), 603--610, DOI:10.1109/JSAC.2020.3020654Google Scholar
Cross Ref
- G. A. Kaissis, M. R. Makowski, D. Rückert, and R. F. Braren. 2020. Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell 2, 6 (2020), 305–311. DOI:10.1038/s42256-020-0186-1Google Scholar
Cross Ref
- N. D. Lane, P. Georgiev, and L. Qendro. 2015. DeepEar: Robust smartphone audio sensing in unconstrained acoustic environments using deep learning. In Proceedings of the 2015 ACM International Joint Conference on Pervasive Ubiquitous Computing (UbiComp’15), 283–294. DOI:10.1145/2750858.2804262 Google Scholar
Digital Library
- R. S. Hande and P. S. Deshpande. 2017. An integrated computerized cough analysis by using wavelet for pneumonia diagnosis. Int. J. Innov. Sci. Res. Technol 2, 9 (2017), 283–287.Google Scholar
- M. S. Hossain and G. Muhammad. 2020. Deep learning based pathology detection for smart connected healthcares. IEEE Network. 34, 6 (2020), 120--125.Google Scholar
Digital Library
- M. S. Hossain, G. Muhammad, and N. Guizani. 2020. Explainable ai and mass surveillance system-based healthcare framework to combat COVID-I9 like pandemics. IEEE Netw. 34, 4 (2020), 126–132. DOI:10.1109/MNET.011.2000458Google Scholar
Cross Ref
- C. Pham. 2016. MobiCough: Real-time cough detection and monitoring using low-cost mobile devices. In Proceedings of the 8th Asian Conference on Intelligent Information and Database Systems (ACIIDS’16). 300–309.Google Scholar
Cross Ref
- P. Porter et al. 2019. A prospective multicentre study testing the diagnostic accuracy of an automated cough sound centred analytic system for the identification of common respiratory disorders in children. Respir. Res 20, 1 (2019), 1–10. DOI:10.1186/s12931-019-1046-6Google Scholar
Cross Ref
- F. Guo et al. 2018. Yanbao: A mobile app using the measurement of clinical parameters for glaucoma screening. IEEE Access. 6 (2018), 77414–77428. DOI:10.1109/ACCESS.2018.2882946Google Scholar
Cross Ref
- X. Sun et al. 2020. The infection evidence of SARS-COV-2 in ocular surface: A single-center cross-sectional study. MedRxiv. 8197033356. DOI:10.1101/2020.02.26.20027938Google Scholar
- H. Zhang et al. 2020. Corona virus international public health emergencies: Implications for radiology management. Acad. Radiol. (2020), 1–5. DOI:10.1016/j.acra.2020.02.003Google Scholar
- Y.-H. Xu et al. 2020. Clinical and computed tomographic imaging features of novel coronavirus pneumonia caused by SARS-CoV-2. J. Infect. (2020). DOI:10.1016/j.jinf.2020.02.017Google Scholar
- J. A. Al-tawfiq and Z. A. Memish. 2020. Diagnosis of SARS-CoV-2 infection based on CT scan vs. RT-PCR: Reflecting on experience from MERS-CoV. J. Hosp. Infect. (2020). DOI:10.1016/j.jhin.2020.03.001Google Scholar
- L. Wang and A. Wong. 2020. COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images. arXiv:2003.09871v4. Retrieved from https://arxiv.org/abs/2003.09871v4.Google Scholar
- C. Lin et al. 2020. Asymptomatic novel coronavirus pneumonia patient outside WuHan: The value of CT images in the course of the disease. Clin. Imag. (2020), 153057. DOI:10.1016/j.clinimag.2020.02.008Google Scholar
- S. Kooraki, M. Hosseiny, L. Myers, and A. Gholamrezanezhad. 2020. Coronavirus outbreak: What the department of radiology should know. J. Am. Coll. Radiol. (2020), 1–5. DOI:10.1016/j.jacr.2020.02.008Google Scholar
- H. S. Maghdid, K. Z. Ghafoor, A. S. Sadiq, K. Curran, and K. Rabie. 2020. A novel ai-enabled framework to diagnose coronavirus COVID 19 using smartphone embedded sensors: Design study. arXiv:2003.07434. Retrieved from https://arxiv.org/abs/2003.07434.Google Scholar
- H. A. Rothan and S. N. Byrareddy. 2020. The epidemiology and pathogenesis of coronavirus disease (COVID-19) outbreak. J. Autoimmun. PG-102433. (2020), 102433. DOI:https://doi.org/10.1016/j.jaut.2020.102433.Google Scholar
- M. A. Rahman et al. 2019. Blockchain and IoT-based cognitive edge framework for sharing economy services in a smart city. IEEE Access. 7 (2019), 18611–18621.Google Scholar
Cross Ref
- M. A. Rahman et al. 2018. Blockchain-based mobile edge computing framework for secure therapy applications. IEEE Access. 6 (2018), 72469–72478. DOI:10.1109/ACCESS.2018.2881246Google Scholar
Cross Ref
- Y. Qian et al. 2020. Blockchain-based privacy-aware content caching in cognitive internet of vehicles. IEEE Netw. 34, 2 (2020), 46–51, DOI:10.1109/MNET.001.1900161Google Scholar
Cross Ref
- M. A. Rahman et al. 2019. A blockchain-based non-invasive cyber-physical occupational therapy framework: BCI perspective. IEEE Access 7 (2019), 34874–34884. DOI:10.1109/ACCESS.2019.2903024Google Scholar
Cross Ref
- A. Alimadadi, S. Aryal, I. Manandhar, P. B. Munroe, B. Joe, and X. Cheng. 2020. Artificial intelligence and machine learning to fight COVID-19. Physiol. Genom. (2020), 1–9. DOI:10.1152/physiolgenomics.00029.2020Google Scholar
- Y. Abdulsalam and M. S. Hossain. 2020. COVID-19 networking demand: An auction-based mechanism for automated selection of edge computing services. IEEE Transactions on Network Science and Engineering. DOI:10.1109/TNSE.2020.3026637Google Scholar
- Q. Lin et al. 2019. A conceptual model of the outbreak of novel coronavirus (2019-nCoV) in Wuhan, China, with human reaction and holiday effects. Int. J. Infect. Dis. (Feb. 2019), 1–9. DOI:10.1016/j.ijid.2020.02.058Google Scholar
- A. Wilder-smith, C. J. Chiew, and V. J. Lee. 2020. Personal View Can we contain the COVID-19 outbreak with the same measures as for SARS ? Lancet Infect. Dis. 3099 (2020), 20. DOI:10.1016/S1473-3099(20)30129-8Google Scholar
- Kaggle. COVID-19 Image Dataset. Retrieved from https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia/download.Google Scholar
- H. Nishiura, N. M. Linton, and A. R. Akhmetzhanov. 2020. Serial interval of novel coronavirus (COVID-19) infections. Int. J. Infect. Dis. 93 (2020), 284–286. DOI:10.1016/j.ijid.2020.02.060Google Scholar
Cross Ref
- J. P. Cohen, P. Morrison, and L. Dao. 2020. COVID-19 Image data collection. arXiv:2003.11597. Retrieved from https://arxiv.org/abs/2003.11597.Google Scholar
- O. Gozes, M. Frid-Adar, H. Greenspan, P. D. Browning, A. Bernheim, and E. Siegel. 2020. Rapid ai development cycle for the coronavirus (COVID-19) pandemic: initial results for automated detection & patient monitoring using deep learning CT image analysis. arXiv:2003.05037. Retrieved from https://arxiv.org/abs/2003.05037.Google Scholar
- J. Zhao, Y. Zhang, X. He, and P. Xie. 2020. COVID-CT-Dataset: A CT Scan dataset about COVID-19. arXiv:2003.13865. Retrieved from https://arxiv.org/abs/2003.13865.Google Scholar
- A. Narin, C. Kaya, and Z. Pamuk. 2020. Automatic detection of coronavirus disease (COVID-19) using x-ray images and deep convolutional neural networks. arXiv:2003.10849. Retrieved from https://arxiv.org/abs/2003.10849.Google Scholar
- J. Bullock, Alexandra, Luccioni, K. H. Pham, C. S. N. Lam, and M. Luengo-Oroz. 2020. Mapping the landscape of artificial intelligence applications against COVID-19. arXiv:2003.11336. Retrieved from https://arxiv.org/abs/2003.11336.Google Scholar
- J. M. Shuai Wang and Bo Kang. 2020. A deep learning algorithm using CT images to screen for corona virus disease (COVID-19). MedRxiv. DOI:10.1101/2020.02.14.20023028Google Scholar
- L. O. Hall, R. Paul, D. B. Goldgof, and G. M. GOLDGOF. 2020. Finding covid-19 from chest X-rays using deep learning on a small dataset. arXiv:2004.02060. Retrieved from https://arxiv.org/abs/2004.02060.Google Scholar
- B. D. Killeen et al. 2020. A county-level dataset for informing the united states’ response to COVID-19. arXiv:2004.00756. Retrieved from https://arxiv.org/abs/2004.00756.Google Scholar
- P. Afshar, S. Heidarian, F. Naderkhani, A. Oikonomou, K. N. Plataniotis, and A. Mohammadi. 2020. COVID-CAPS: A capsule network-based framework for identification of COVID-19 cases from X-ray Images. arXiv:2004.02696. Retrieved from https://arxiv.org/abs/2004.02696.Google Scholar
- S. Chaganti et al. 2020. Quantification of tomographic patterns associated with COVID-19 from chest CT. 5. arXiv:2004.01279. Retrieved from https://arxiv.org/abs/2004.01279.Google Scholar
- M. A. Rahman, M. S. Hossain, N. A. Alrajeh, and N. Guizani. 2020. B5G and explainable deep learning assisted healthcare vertical at the edge: COVID-I9 perspective. IEEE Netw. 34, 4 (2020), 98–105. DOI:10.1109/MNET.011.2000353Google Scholar
Cross Ref
- M. A. Rahman and M. S. Hossain. 2021. An Internet of medical things-enabled edge computing framework for tackling COVID-19. IEEE Internet of Things J. (2021) 1--8. DOI:10.1109/JIOT.2021.3051080Google Scholar
- M. A. Rahman, M. S. Hossain, N. Alrajeh, and F. Alsolami. 2020. Adversarial examples – security threats to COVID-19 deep learning systems in medical iot devices. IEEE IoT J. DOI:10.1109/JIOT.2020.3013710Google Scholar
- F. Shan et al. 2020. Lung infection quantification of COVID-19 in CT images with deep learning. arXiv:2003.04655. Retrieved from https://arxiv.org/abs/2003.04655.Google Scholar
- M. Farooq and A. Hafeez. 2020. COVID-ResNet: A deep learning framework for screening of COVID19 from radiographs. arXiv:2003.14395. Retrieved from https://arxiv.org/abs/2003.14395.Google Scholar
- P. Kumar and S. Kumari. 2020. Detection of coronavirus disease (COVID-19) based on deep features. DOI:10.20944/preprints202003.0300.v1Google Scholar
- B. Ghoshal and A. Tucker. 2020. Estimating uncertainty and interpretability in deep learning for coronavirus (COVID-19). Detection (2020), 1–14.Google Scholar
- M. Loey, F. Smarandache, and N. E. M. Khalifa. 2020. Within the lack of COVID-19 benchmark dataset : A novel GAN with deep transfer learning for corona- virus detection in chest x-ray images (unpublished).Google Scholar
- Y. Ye et al. 2020. α-Satellite: An ai-driven system and benchmark datasets for hierarchical community-level risk assessment to help combat COVID-19. arXiv:2003.12232. Retrieved from https://arxiv.org/abs/2003.12232.Google Scholar
- N. E. M. Khalifa and M. H. N. Taha, Detection of Coronavirus (COVID-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest x-ray dataset. arXiv:2004.01184. Retrieved from https://arxiv.org/abs/2004.01184.Google Scholar
- Y. Liang and P. Xie. 2020. Identifying radiological findings related to COVID-19 from medical literature. arXiv:2004.01862. Retrieved from https://arxiv.org/abs/2004.01862.Google Scholar
- A. Abbas, M. M. Abdelsamea, and M. M. Gaber. 2020. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. medRxiv. DOI:10.1101/2020.03.30.20047456Google Scholar
- R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. 2020. Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis 128, 2 (2020), 336–359. DOI:10.1007/s11263-019-01228-7Google Scholar
Digital Library
- F. Song et al. 2020. Emerging 2019 novel coronavirus (2019-NCoV) pneumonia. Radiology 295, 1 (2019), 210–217. DOI:10.1148/radiol.2020200274Google Scholar
Cross Ref
- Q. Guo, J. Ye, Y. Hu, G. Zhang, X. Li, and H. Li. 2020. MultiPAD: A multivariant partition-based method for audio adversarial examples detection. IEEE Access. 8 (2020), 63368–63380. DOI:10.1109/ACCESS.2020.2985231Google Scholar
Cross Ref
- R. Taori, A. Kamsetty, B. Chu, and N. Vemuri. 2019. Targeted adversarial examples for black box audio systems. In Proceedings of the 2019 IEEE Symposium on Security and Privacy Workshops (SPW’19). 15–20. DOI:10.1109/SPW.2019.00016Google Scholar
- W. Zhang. 2019. Generating adversarial examples in one shot with image-to-image translation GAN. IEEE Access. 7 (2019), 151103–151119. DOI:10.1109/ACCESS.2019.2946461Google Scholar
Cross Ref
- M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter. 2019. A general framework for adversarial examples with objectives. ACM Trans. Priv. Secur 22, 3 (2019). DOI:10.1145/3317611 Google Scholar
Digital Library
Index Terms
A Multimodal, Multimedia Point-of-Care Deep Learning Framework for COVID-19 Diagnosis
Recommendations
Diagnosis of COVID-19 from Multimodal Imaging Data Using Optimized Deep Learning Techniques
AbstractCOVID-19 had a global impact, claiming many lives and disrupting healthcare systems even in many developed countries. Various mutations of the severe acute respiratory syndrome coronavirus-2, continue to be an impediment to early detection of this ...
A deep learning-based framework for detecting COVID-19 patients using chest X-rays
AbstractSevere Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) has caused outbreaks of new coronavirus disease (COVID-19) around the world. Rapid and accurate detection of COVID-19 coronavirus is an important step in limiting the spread of the COVID-...
Diagnosis of Covid-19 using Deep Learning
IC3-2022: Proceedings of the 2022 Fourteenth International Conference on Contemporary ComputingIn the global health disaster of the Coronavirus infection-2019 (Covid-19) pandemic, the health sector is avidly seeking new technologies and strategies to detect and manage the spread of the Coronavirus outbreak. Artificial Intelligence (AI) is ...






Comments