Abstract
In recent years, many model intellectual property (IP) proof methods for IP protection have been proposed, such as model watermarking and model fingerprinting. However, with the increasing number of models transmitted and deployed on the Internet, quickly finding the suspect model among thousands of models on model-sharing platforms such as GitHub is in great demand, which concurrently triggers the new security problem of model copy detection for IP protection. As an important part of the model IP protection system, the model copy detection task has not received enough attention. Due to the high computational complexity, both model watermarking and model fingerprinting lack the capability to efficiently find suspected infringing models among tens of millions of models. In this article, inspired by the hash-based image retrieval methods, we introduce a novel model copy detection mechanism: perceptual hashing for convolutional neural networks (CNNs). The proposed perceptual hashing algorithm can convert the weights of CNNs to fixed-length binary hash codes so that the lightly modified version has the similar hash code as the original model. By comparing the similarity of a pair of hash codes between a query model and a test model in the model library, similar versions of a query model can be retrieved efficiently. To the best of our knowledge, this is the first perceptual hashing algorithm for deep neural network models. Specifically, we first select the important model weights based on the model compression theory, then calculate the normal test statistics (NTS) on the segments of important weights, and finally encode the NTS features into hash codes. The experiment performed on a model library containing 3,565 models indicates that our perceptual hashing scheme has a superior copy detection performance.
- [1] . 2020. Fractal coding-based robust and alignment-free fingerprint image hashing. IEEE Trans. Inf. Forens. Secur. 15, 1 (2020), 2587–2601. Google Scholar
Digital Library
- [2] . 2018. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In Proceedings of the USENIX Security Symposium (USENIX Security’18). 1615–1631.Google Scholar
- [3] . 2017. NTIRE 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Google Scholar
Cross Ref
- [4] . 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In British Machine Vision Conference. BMVA Press, 135.1–135.10. Google Scholar
Cross Ref
- [5] . 2020. What is the state of neural network pruning? Proceedings of Machine Learning and Systems 2 (2020), 129–146.Google Scholar
- [6] . 2015. Weight uncertainty in neural networks. In International Conference on Machine Learning, Vol. 37. 1613–1622.Google Scholar
- [7] . 2019. IPGuard: Protecting the intellectual property of deep neural networks via fingerprinting the classification boundary. arXiv:1910.12903 Retrieved from https://arxiv.org/abs/1910.129.03. Google Scholar
Digital Library
- [8] . 2017. Towards evaluating the robustness of neural networks. In Proceedings of the IEEE Symposium on Security and Privacy. IEEE, 39–57. Google Scholar
Cross Ref
- [9] . 2020. Neural network compression using higher-order statistics and auxiliary reconstruction losses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 716–717. Google Scholar
Cross Ref
- [10] . 2012. Multi-column deep neural networks for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3642–3649.Google Scholar
- [11] . 2019. DeepSigns: An end-to-end watermarking framework for ownership protection of deep neural networks. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems. 485–497.Google Scholar
- [12] . 2019. An improved perceptual hash algorithm based on U-Net for the authentication of high-resolution remote sensing image. Appl. Sci. 9, 15 (2019), 2972. Google Scholar
Cross Ref
- [13] . 2019. From selective deep convolutional features to compact binary representations for image retrieval. ACM Trans. Multimedia Comput. Commun. Appl. 15, 2 (2019), 1–22. Google Scholar
Digital Library
- [14] . 2006. An introduction to ROC analysis. Pattern Recogn. Lett. 27, 8 (
June 2006), 861–874. Google ScholarDigital Library
- [15] . 2020. Multi-dimensional pruning: A unified framework for model compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1508–1517. Google Scholar
Cross Ref
- [16] . 2018. Watermarking deep neural networks for embedded systems. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design. IEEE, 1–8. Google Scholar
Digital Library
- [17] . 2015. Learning both weights and connections for efficient neural network. In Neural Information Processing Systems. 1135–1143.Google Scholar
- [18] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778. Google Scholar
Cross Ref
- [19] . 2015. Learning visual semantic relationships for efficient visual retrieval. IEEE Trans. Big Data. 1, 4 (2015), 152–161. Google Scholar
Cross Ref
- [20] . 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4700–4708. Google Scholar
Cross Ref
- [21] . 2010. Perceptual image hashing based on virtual watermark detection. IEEE Trans. Image Process. 19, 4 (2010), 981–994. Google Scholar
Digital Library
- [22] . 2020. Universal litmus patterns: Revealing backdoor attacks in CNNs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 301–310. Google Scholar
Cross Ref
- [23] . 2009. Learning Multiple Layers of Features from Tiny Images.
Technical Report .Google Scholar - [24] . 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324. Google Scholar
Cross Ref
- [25] . 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google Scholar
Cross Ref
- [26] . 2022. Evaluating the robustness of trigger set-based watermarks embedded in deep neural networks. IEEE Trans. Depend. Sec. Comput. (2022), 1–15. Google Scholar
Cross Ref
- [27] . 2021. ModelDiff: Testing-based DNN similarity comparison for model reuse detection. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2021). Association for Computing Machinery, New York, NY, 139–151. Google Scholar
Digital Library
- [28] . 2021. Efficient hashing method using 2D-2D PCA for image copy detection. IEEE Trans. Knowl. Data Eng. (2021), 1–1. Google Scholar
Digital Library
- [29] . 2019. Efficient image hashing with geometric invariant vector distance for copy detection. ACM Trans. Multimedia Comput. Commun. Appl. 15, 4 (2019), 1–22. Google Scholar
Digital Library
- [30] . 2019. Deep neural network fingerprinting by conferrable adversarial examples. arXiv:1912.00888. Retrieved from https://arxiv.org/abs/1912.00888.Google Scholar
- [31] . 2021. Fine-grained visual computing based on deep learning. ACM Trans. Multimidia Comput. Commun. Appl. 17, 1s (2021), 1–19. Google Scholar
Digital Library
- [32] . 2018. A self-adaptive deep learning-based system for anomaly detection in 5G networks. IEEE Access 6, 1 (2018), 7700–7712. Google Scholar
Cross Ref
- [33] . 2022. Scenario-aware recurrent transformer for goal-directed video captioning. ACM Trans. Multim. Comput. Commun. Appl. 18, 4 (2022), 104:1–104:17.Google Scholar
Digital Library
- [34] . 2014. The stanford CoreNLP natural language processing toolkit. In Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 55–60. Google Scholar
Cross Ref
- [35] . 2019. A novel liver image classification method using perceptual hash-based convolutional neural network. Arab. J. Sci. Eng. 44, 4 (2019), 3173–3182.Google Scholar
Cross Ref
- [36] . 2011. Power comparisons of shapiro-wilk, kolmogorov-smirnov, lilliefors and anderson-darling tests. J. Stat. Model. Analyt. 2, 1 (2011), 21–33.Google Scholar
- [37] . 2015. Mlaas: Machine learning as a service. In Proceedings of the IEEE International Conference on Machine Learning and Applications. IEEE, 896–902. Google Scholar
Cross Ref
- [38] . 1992. RFC1321: The MD5 Message-Digest Algorithm.Google Scholar
- [39] . 2015. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 3 (2015), 211–252. Google Scholar
Digital Library
- [40] . 2018. MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4510–4520. Google Scholar
Cross Ref
- [41] . 1965. An analysis of variance test for normality (complete samples). Biometrika. 52, 3/4 (1965), 591–611. Google Scholar
Cross Ref
- [42] . 2022. Shuffle-invariant network for action recognition in videos. ACM Trans. Multimedia Comput. Commun. Appl. 18, 3 (2022), 1–18. Google Scholar
Digital Library
- [43] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https://arxiv.org/abs/1409.1556.Google Scholar
- [44] . 2013. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD ’13). Association for Computing Machinery, New York, NY, 785–796. Google Scholar
Digital Library
- [45] . 2011. The german traffic sign recognition benchmark: A multi-class classification competition. In Proceedings of the International Joint Conference on Neural Networks. 1453–1460.Google Scholar
Cross Ref
- [46] . 2006. Robust and secure image hashing. IEEE Trans. Inf. Forens. Secur. 1, 2 (2006), 215–230. Google Scholar
Digital Library
- [47] . 2019. Robust image hashing with tensor decomposition. IEEE Trans. Knowl. Data Eng. 31, 3 (2019), 549–560. Google Scholar
Digital Library
- [48] . 2022. UAV-satellite view synthesis for cross-view geo-localization. IEEE Trans. Circ. Syst. Video Technol. 32, 7 (2022), 4804–4815.Google Scholar
Digital Library
- [49] . 2021. SemanticHash: Hash coding via semantics-guided label prototype learning. IEEE Trans. Artif. Intell. 2, 1 (2021), 42–57. Google Scholar
Cross Ref
- [50] . 2017. Embedding watermarks into deep neural networks. In Proceedings of the ACM International Conference on Multimedia Retrieval. 269–277. Google Scholar
Digital Library
- [51] . 2019. MIASec: Enabling data indistinguishability against membership inference attacks in MLaaS. IEEE Trans. Sust. Comput. 5, 3 (2019), 365–376. Google Scholar
Cross Ref
- [52] . 2009. A novel image hash algorithm resistant to print-scan. Sign. Process. 89, 12 (2009), 2415–2424.
Special Section: Visual Information Analysis for Security .Google ScholarDigital Library
- [53] . 2021. Equivariant adversarial network for image-to-image translation. ACM Trans. Multimedia Comput. Commun. Appl. 17, 2s (2021), 1–14. Google Scholar
Digital Library
- [54] . 2021. Exploring structure consistency for deep model watermarking. arXiv:2108.02360. Retrieved from https://arxiv.org/abs/2108.02360.Google Scholar
- [55] . 2020. Model watermarking for image processing networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 12805–12812. Google Scholar
Cross Ref
- [56] . 2021. Deep model intellectual property protection via deep watermarking. IEEE Trans. Pattern Anal. Mach. Intell. 44, 8 (2021), 4005–4020. Google Scholar
Digital Library
- [57] . 2020. Passport-aware normalization for deep model protection. Neural Inf. Process. Syst. 33 (2020).Google Scholar
- [58] . 2018. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the Asia Conference on Computer and Communications Security. 159–172. Google Scholar
Digital Library
- [59] . 2022. Progressive meta-learning with curriculum. IEEE Trans. Circ. Syst. Video Technol. 32, 9 (2022), 5916–5930. Google Scholar
Cross Ref
- [60] . 2020. AFA: Adversarial fingerprinting authentication for deep neural networks. Comput. Commun. 150, 1 (2020), 488–497. Google Scholar
Digital Library
- [61] . 2016. Effective and efficient global context verification for image copy detection. IEEE Trans. Inf. Forens. Secur. 12, 1 (2016), 48–63. Google Scholar
Digital Library
- [62] . 2022. Step by step: A hierarchical framework for multi-hop knowledge graph reasoning with reinforcement learning. Knowl. Bas. Syst. 248, 1 (2022), 108843. Google Scholar
Digital Library
- [63] . 2016. Deep hashing network for efficient similarity retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30. Google Scholar
Cross Ref
Index Terms
Perceptual Hashing of Deep Convolutional Neural Networks for Model Copy Detection
Recommendations
Theory of deep convolutional neural networks: Downsampling
AbstractEstablishing a solid theoretical foundation for structured deep neural networks is greatly desired due to the successful applications of deep learning in various practical domains. This paper aims at an approximation theory of deep ...
Deep Convolutional Neural Networks for pedestrian detection
Pedestrian detection is a popular research topic due to its paramount importance for a number of applications, especially in the fields of automotive, surveillance and robotics. Despite the significant improvements, pedestrian detection is still an open ...
Deep Convolutional Neural Networks for Large-scale Speech Tasks
Convolutional Neural Networks (CNNs) are an alternative type of neural network that can be used to reduce spectral variations and model spectral correlations which exist in signals. Since speech signals exhibit both of these properties, we hypothesize ...






Comments