Abstract
Medical image fusion generates a fused image containing multiple features extracted from different source images, and it is of great help in clinical analysis and diagnosis. However, training a deep learning model for image fusion usually requires enormous computing power, especially for large volumes of medical data. Meanwhile, the privacy of images is also a critical issue. In this article, we propose a privacy-preserving blockchain-based medical image fusion (BMIF) framework. First, to ensure fusion performance, we design a new medical image fusion model based on convolutional neural network and Inception network and integrate the proposed model into the consensus process of blockchain. Next, to save computing power of blockchain, we design a consensus mechanism by requesting consensus nodes to train the fusion model instead of calculating useless hash values in traditional blockchain. Then, to protect data privacy, we further present an efficient homomorphic encryption to realize the training of fusion model on encrypted medical data. Finally, we conduct theoretical analysis and extensive experiments on public datasets to evaluate the feasibility and the performance of our proposed BMIF. The results exhibit that BMIF is efficient and secure, and our medical image fusion network performs better than state-of-the-art approaches.
- [1] . 2015. A new image quality metric for image fusion: The sum of the correlations of differences. AEU - Int. J. Electron. Commun. 69, 12 (2015), 1890–1896.Google Scholar
Cross Ref
- [2] . 2017. Proofs of Useful Work. Cryptology ePrint Archive, Report 2017/203. Retrieved from https://eprint.iacr.org/2017/203.Google Scholar
- [3] . 2020. Tamper-proofing video with hierarchical attention autoencoder hashing on blockchain. IEEE Trans. Multim. 22, 11 (2020), 2858–2872.Google Scholar
Cross Ref
- [4] . 2001. Universally composable security: A new paradigm for cryptographic protocols. In IEEE Symposium on Foundations of Computer Science (SFCS). 136–145.Google Scholar
- [5] . 2019. Energy-recycling blockchain with proof-of-deep-learning. In IEEE International Conference on Blockchain and Cryptocurrency (ICBC). 19–23.Google Scholar
Cross Ref
- [6] . 2017. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Application of Cryptology and Information Security (ASIACRYPT). 409–437.Google Scholar
Cross Ref
- [7] . 2018. Faster CryptoNets: Leveraging sparsity for real-world encrypted inference. ArXiv: abs/1811.09953 (2018).Google Scholar
- [8] . 2006. The nonsubsampled contourlet transform: Theory, design, and applications. IEEE Trans. Image Process. 15, 10 (2006), 3089–3101.Google Scholar
Digital Library
- [9] . 2018. Untangling blockchain: A data processing view of blockchain systems. IEEE Trans. Knowl. Data Eng. 30, 7 (2018), 1366–1385.Google Scholar
Cross Ref
- [10] . 1995. Image quality measures and their performance. IEEE Trans. Commun. 43, 12 (1995), 2959–2965.Google Scholar
Cross Ref
- [11] . 2019. A semantic-based medical image fusion approach. ArXiv: abs/1906.00225 (2019).Google Scholar
- [12] . 2020. Using blockchain for improved video integrity verification. IEEE Trans. Multim. 22, 1 (2020), 108–121.Google Scholar
Digital Library
- [13] . 2017. PET and MRI image fusion based on combination of 2-D Hilbert transform and IHS method. Biomed. J. 40, 4 (2017), 219–225.Google Scholar
Cross Ref
- [14] . 2014. Fast-FMI: Non-reference image fusion metric. In IEEE International Conference on Application of Information and Communication Technologies (AICT). 1–3.Google Scholar
Cross Ref
- [15] . 2018. Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput. Applic. 30 (2018), 2029–2045.Google Scholar
Digital Library
- [16] . 2017. CryptoDL: Deep neural networks over encrypted data. arXiv e-prints (2017), arXiv:1711.05189.Google Scholar
- [17] . 2016. Medical image fusion in curvelet domain employing PCA and maximum selection rule. In 2nd International Conference on Computer and Communication Technologies. 1–9.Google Scholar
- [18] . 2020. VIF-Net: An unsupervised framework for infrared and visible image fusion. IEEE Trans. Computat. Imag. 6 (2020), 640–651.Google Scholar
Cross Ref
- [19] . 2014. Medical image fusion: A survey of the state of the art. Inf. Fusion 19 (2014), 4–19.Google Scholar
Digital Library
- [20] . 2020. Secure outsourcing SIFT: Efficient and privacy-preserving image feature extraction in the encrypted domain. IEEE Trans. Depend. Secure Comput. 17, 1 (2020), 179–193.Google Scholar
Cross Ref
- [21] . [n.d.]. The Whole Brain Atlas. Retrieved from http://www.med.harvard.edu/AANLIB/home.html.Google Scholar
- [22] . 2020. Unsupervised deep image fusion with structure tensor representations. IEEE Trans. Image Process. 29 (2020), 3845–3858.Google Scholar
Digital Library
- [23] 2015. Image fusion based on pixel significance using cross bilateral filter. Sig., Image Vid. Process 9, 5 (2015), 1193–1204.Google Scholar
Cross Ref
- [24] . 2019. Exploiting computation power of blockchain for biomedical image segmentation. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2802–2811.Google Scholar
Cross Ref
- [25] . 2019. DenseFuse: A fusion approach to infrared and visible images. IEEE Trans. Image Process. 28, 5 (2019), 2614–2623.Google Scholar
Digital Library
- [26] . 2018. Infrared and visible image fusion using a deep learning framework. In International Conference on Pattern Recognition (ICPR). 2705–2710.Google Scholar
Cross Ref
- [27] . 2019. Fusion of medical sensors using adaptive cloud model in local Laplacian pyramid domain. IEEE Trans. Biomed. Eng. 66, 4 (2019), 1172–1183.Google Scholar
Cross Ref
- [28] . 2021. Privacy protection for medical image management based on blockchain. In International Conference on Database Systems for Advanced Applications (DASFAA) Workshops. 414–428.Google Scholar
Digital Library
- [29] . 2020. Fog-based secure service discovery for internet of multimedia things: A cross-blockchain approach. ACM Trans. Multim. Comput., Commun. Applic. 16, 3s (2020).Google Scholar
- [30] . 2017. Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion. Neurocomputing 235 (2017), 131–139.Google Scholar
Digital Library
- [31] . 2017. Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36 (2017), 191–207.Google Scholar
Digital Library
- [32] . 2016. Image fusion with convolutional sparse representation. IEEE Sig. Process. Lett. 23, 12 (2016), 1882–1886.Google Scholar
Cross Ref
- [33] . 2016. Infrared and visible image fusion via gradient transfer and total variation minimization. Inf. Fusion 31 (2016), 100–109.Google Scholar
Digital Library
- [34] . 2020. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion. IEEE Trans. Image Process. 29 (2020), 4980–4995.Google Scholar
Digital Library
- [35] . 2015. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 24, 11 (2015), 3345–3356.Google Scholar
Digital Library
- [36] . 2016. PCA-DWT based medical image fusion using non sub-sampled contourlet transform. In International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES). 1089–1094.Google Scholar
Cross Ref
- [37] . [n.d.]. Pyfhel: PYthon For Homomorphic Encryption Libraries. Retrieved from https://github.com/ibarrond/Pyfhel.Google Scholar
- [38] . 2021. A survey on healthcare data: A security perspective. ACM Trans. Multim. Comput., Commun. Applic. 17 (2021).Google Scholar
- [39] . 2021. A survey on healthcare data: A security perspective. ACM Trans. Multim. Comput., Commun. Applic. 17, 2s (2021).Google Scholar
- [40] . 2015. Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network. Inf. Fusion 18 (2015), 91–101.Google Scholar
- [41] . [n.d.]. The Cancer Imaging Archive (TCIA). Retrieved from https://www.cancerimagingarchive.net/.Google Scholar
- [42] . 2021. Security and privacy of patient information in medical systems based on blockchain technology. ACM Trans. Multim. Comput., Commun. Applic. 17, 2s (2021).Google Scholar
- [43] . 2016. Multifocus color image fusion based on NSST and PCNN. J. Sensors 2016 (2016), 1–12.Google Scholar
- [44] . 2021. Trustworthy image fusion with deep learning for wireless applications. Wirel. Commun. Mob. Comput. 2021 (2021).Google Scholar
Index Terms
BMIF: Privacy-preserving Blockchain-based Medical Image Fusion
Recommendations
Medical Image Fusion in the NSST Domain with AR-Improved PA-PCNN
IMIP 2022: 2022 4th International Conference on Intelligent Medicine and Image ProcessingTo present a medical image fusion method based on the non-subsampled shearlet transform (NSST) and improved parametric adaptive pulse-coupled neural network (PA-PCNN) for CT and MRI, which makes the fused images clearer and contains more information. ...
Deep learning methods for medical image fusion: A review
AbstractThe image fusion methods based on deep learning has become a research hotspot in the field of computer vision in recent years. This paper reviews these methods from five aspects: Firstly, the principle and advantages of image fusion ...
Graphical abstractDisplay Omitted
Highlights- Summarizing various deep learning models of medical image fusion.
- Discussing ...
Objective evaluation of noisy multimodal medical image fusion using Daubechies complex wavelet transform
ICVGIP '12: Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image ProcessingMedical image fusion needs proper attention as images obtained from medical instruments are of poor contrast and corrupted by blur and noise due to imperfection of image capturing devices. Thus, objective evaluation of medical image fusion techniques ...






Comments