skip to main content
research-article

BMIF: Privacy-preserving Blockchain-based Medical Image Fusion

Published:24 January 2023Publication History
Skip Abstract Section

Abstract

Medical image fusion generates a fused image containing multiple features extracted from different source images, and it is of great help in clinical analysis and diagnosis. However, training a deep learning model for image fusion usually requires enormous computing power, especially for large volumes of medical data. Meanwhile, the privacy of images is also a critical issue. In this article, we propose a privacy-preserving blockchain-based medical image fusion (BMIF) framework. First, to ensure fusion performance, we design a new medical image fusion model based on convolutional neural network and Inception network and integrate the proposed model into the consensus process of blockchain. Next, to save computing power of blockchain, we design a consensus mechanism by requesting consensus nodes to train the fusion model instead of calculating useless hash values in traditional blockchain. Then, to protect data privacy, we further present an efficient homomorphic encryption to realize the training of fusion model on encrypted medical data. Finally, we conduct theoretical analysis and extensive experiments on public datasets to evaluate the feasibility and the performance of our proposed BMIF. The results exhibit that BMIF is efficient and secure, and our medical image fusion network performs better than state-of-the-art approaches.

REFERENCES

  1. [1] Aslantas V. and Bendes E.. 2015. A new image quality metric for image fusion: The sum of the correlations of differences. AEU - Int. J. Electron. Commun. 69, 12 (2015), 18901896.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Ball Marshall, Rosen Alon, Sabin Manuel, and Vasudevan Prashant Nalini. 2017. Proofs of Useful Work. Cryptology ePrint Archive, Report 2017/203. Retrieved from https://eprint.iacr.org/2017/203.Google ScholarGoogle Scholar
  3. [3] Bui Tu, Cooper Daniel, Collomosse John, Bell Mark, Green Alex, Sheridan John, Higgins Jez, Das Arindra, Keller Jared Robert, and Thereaux Olivier. 2020. Tamper-proofing video with hierarchical attention autoencoder hashing on blockchain. IEEE Trans. Multim. 22, 11 (2020), 28582872.Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Canetti Ran. 2001. Universally composable security: A new paradigm for cryptographic protocols. In IEEE Symposium on Foundations of Computer Science (SFCS). 136145.Google ScholarGoogle Scholar
  5. [5] Chenli Changhao, Li Boyang, Shi Yiyu, and Jung Taeho. 2019. Energy-recycling blockchain with proof-of-deep-learning. In IEEE International Conference on Blockchain and Cryptocurrency (ICBC). 1923.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Cheon Jung Hee, Kim Andrey, Kim Miran, and Song Yongsoo. 2017. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Application of Cryptology and Information Security (ASIACRYPT). 409437.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Chou Edward, Beal Josh, Levy Daniel, Yeung Serena, Haque Albert, and Fei-Fei Li. 2018. Faster CryptoNets: Leveraging sparsity for real-world encrypted inference. ArXiv: abs/1811.09953 (2018).Google ScholarGoogle Scholar
  8. [8] Cunha Arthur L., Zhou Jianping, and Do Minh N.. 2006. The nonsubsampled contourlet transform: Theory, design, and applications. IEEE Trans. Image Process. 15, 10 (2006), 30893101.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Dinh Tien Tuan Anh, Liu Rui, Zhang Meihui, Chen Gang, Ooi Beng Chin, and Wang Ji. 2018. Untangling blockchain: A data processing view of blockchain systems. IEEE Trans. Knowl. Data Eng. 30, 7 (2018), 13661385.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Eskicioglu A. M. and Fisher P. S.. 1995. Image quality measures and their performance. IEEE Trans. Commun. 43, 12 (1995), 29592965.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Fan Fanda, Huang Yunyou, Wang Lei, Xiong Xingwang, Jiang Zihan, Zhang Zhifei, and Zhan J.. 2019. A semantic-based medical image fusion approach. ArXiv: abs/1906.00225 (2019).Google ScholarGoogle Scholar
  12. [12] Ghimire Sarala, Choi Jae Young, and Lee Bumshik. 2020. Using blockchain for improved video integrity verification. IEEE Trans. Multim. 22, 1 (2020), 108121.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Haddadpour Mozhdeh, Daneshvar Sabalan, and Seyedarabi Hadi. 2017. PET and MRI image fusion based on combination of 2-D Hilbert transform and IHS method. Biomed. J. 40, 4 (2017), 219225.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Haghighat Mohammad and Razian Masoud Amirkabiri. 2014. Fast-FMI: Non-reference image fusion metric. In IEEE International Conference on Application of Information and Communication Technologies (AICT). 13.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Haithem Hermessi, Olfa Mourali, and Ezzeddine Zagrouba. 2018. Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput. Applic. 30 (2018), 20292045.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Hesamifard Ehsan, Takabi Hassan, and Ghasemi Mehdi. 2017. CryptoDL: Deep neural networks over encrypted data. arXiv e-prints (2017), arXiv:1711.05189.Google ScholarGoogle Scholar
  17. [17] Himanshi, Bhateja Vikrant, Krishn Abhinav, and Sahu Akanksha. 2016. Medical image fusion in curvelet domain employing PCA and maximum selection rule. In 2nd International Conference on Computer and Communication Technologies. 19.Google ScholarGoogle Scholar
  18. [18] Hou Ruichao, Zhou Dongming, Nie Rencan, Liu Dong, Xiong Lei, Guo Yanbu, and Yu Chuanbo. 2020. VIF-Net: An unsupervised framework for infrared and visible image fusion. IEEE Trans. Computat. Imag. 6 (2020), 640651.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] James Alex Pappachen and Dasarathy Belur V.. 2014. Medical image fusion: A survey of the state of the art. Inf. Fusion 19 (2014), 419.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Jiang Linzhi, Xu Chunxiang, Wang Xiaofang, Luo Bo, and Wang Huaqun. 2020. Secure outsourcing SIFT: Efficient and privacy-preserving image feature extraction in the encrypted domain. IEEE Trans. Depend. Secure Comput. 17, 1 (2020), 179193.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Johnson Keith A. and Becker J. Alex. [n.d.]. The Whole Brain Atlas. Retrieved from http://www.med.harvard.edu/AANLIB/home.html.Google ScholarGoogle Scholar
  22. [22] Jung Hyungjoo, Kim Youngjung, Jang Hyunsung, Ha Namkoo, and Sohn Kwanghoon. 2020. Unsupervised deep image fusion with structure tensor representations. IEEE Trans. Image Process. 29 (2020), 38453858.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Kumar B. K. Shreyamsha2015. Image fusion based on pixel significance using cross bilateral filter. Sig., Image Vid. Process 9, 5 (2015), 11931204.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Li Boyang, Chenli Changhao, Xu Xiaowei, Jung Taeho, and Shi Yiyu. 2019. Exploiting computation power of blockchain for biomedical image segmentation. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 28022811.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Li Hui and Wu Xiao-Jun. 2019. DenseFuse: A fusion approach to infrared and visible images. IEEE Trans. Image Process. 28, 5 (2019), 26142623.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Li Hui, Wu Xiao-Jun, and Kittler Josef. 2018. Infrared and visible image fusion using a deep learning framework. In International Conference on Pattern Recognition (ICPR). 27052710.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Li Weisheng, Du Jiao, Zhao Zhengmin, and Long Jinyi. 2019. Fusion of medical sensors using adaptive cloud model in local Laplacian pyramid domain. IEEE Trans. Biomed. Eng. 66, 4 (2019), 11721183.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Li Yifei, Wang Yiwen, Wan Ji, Ren Youzhi, and Li Yafei. 2021. Privacy protection for medical image management based on blockchain. In International Conference on Database Systems for Advanced Applications (DASFAA) Workshops. 414428.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Liang Haoran, Wu Jun, Zheng Xi, Zhang Mengshi, Li Jianhua, and Jolfaei Alireza. 2020. Fog-based secure service discovery for internet of multimedia things: A cross-blockchain approach. ACM Trans. Multim. Comput., Commun. Applic. 16, 3s (2020).Google ScholarGoogle Scholar
  30. [30] Liu Xingbin, Mei Wenbo, and Du Huiqian. 2017. Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion. Neurocomputing 235 (2017), 131139.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Liu Yu, Chen Xun, Peng Hu, and Wang Zengfu. 2017. Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36 (2017), 191207.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Liu Yu, Chen Xun, Ward Rabab K., and Wang Z. Jane. 2016. Image fusion with convolutional sparse representation. IEEE Sig. Process. Lett. 23, 12 (2016), 18821886.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Ma Jiayi, Chen Chen, Li Chang, and Huang Jun. 2016. Infrared and visible image fusion via gradient transfer and total variation minimization. Inf. Fusion 31 (2016), 100109.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Ma Jiayi, Xu Han, Jiang Junjun, Mei Xiaoguang, and Zhang Xiao-Ping. 2020. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion. IEEE Trans. Image Process. 29 (2020), 49804995.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Ma Kede, Zeng Kai, and Wang Zhou. 2015. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 24, 11 (2015), 33453356.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Madanala Sindhuja and Rani K. Jhansi. 2016. PCA-DWT based medical image fusion using non sub-sampled contourlet transform. In International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES). 10891094.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] pyfhel. [n.d.]. Pyfhel: PYthon For Homomorphic Encryption Libraries. Retrieved from https://github.com/ibarrond/Pyfhel.Google ScholarGoogle Scholar
  38. [38] Singh A. K., Anand A., Lv Z., Ko H., and Mohan A.. 2021. A survey on healthcare data: A security perspective. ACM Trans. Multim. Comput., Commun. Applic. 17 (2021).Google ScholarGoogle Scholar
  39. [39] Singh Amit Kumar, Anand A., Lv Zhihan, Ko Hoon, and Mohan Anand. 2021. A survey on healthcare data: A security perspective. ACM Trans. Multim. Comput., Commun. Applic. 17, 2s (2021).Google ScholarGoogle Scholar
  40. [40] Singh Sneha, Gupta Deep, Anand R. S., and Kumar Vinod. 2015. Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network. Inf. Fusion 18 (2015), 91101.Google ScholarGoogle Scholar
  41. [41] TCIA. [n.d.]. The Cancer Imaging Archive (TCIA). Retrieved from https://www.cancerimagingarchive.net/.Google ScholarGoogle Scholar
  42. [42] Wu Hongjiao, Dwivedi Ashutosh Dhar, and Srivastava Gautam. 2021. Security and privacy of patient information in medical systems based on blockchain technology. ACM Trans. Multim. Comput., Commun. Applic. 17, 2s (2021).Google ScholarGoogle Scholar
  43. [43] Xin Jin, Nie Rencan, Zhou Dongming, Wang Quan, and He Kangjian. 2016. Multifocus color image fusion based on NSST and PCNN. J. Sensors 2016 (2016), 112.Google ScholarGoogle Scholar
  44. [44] Zhang Chao, Hu Haojin, Tai Yonghang, Yun Lijun, and Zhang Jun. 2021. Trustworthy image fusion with deep learning for wireless applications. Wirel. Commun. Mob. Comput. 2021 (2021).Google ScholarGoogle Scholar

Index Terms

  1. BMIF: Privacy-preserving Blockchain-based Medical Image Fusion

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 1s
        February 2023
        504 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3572859
        • Editor:
        • Abdulmotaleb El Saddik
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 24 January 2023
        • Online AM: 20 April 2022
        • Accepted: 10 April 2022
        • Revised: 5 April 2022
        • Received: 25 June 2021
        Published in tomm Volume 19, Issue 1s

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!