skip to main content
research-article

Differentially Private Tensor Train Deep Computation for Internet of Multimedia Things

Authors Info & Claims
Published:31 December 2020Publication History
Skip Abstract Section

Abstract

The significant growth of the Internet of Things (IoT) takes a key and active role in healthcare, smart homes, smart manufacturing, and wearable gadgets. Due to complexness and difficulty in processing multimedia data, the IoT based scheme, namely Internet of Multimedia Things (IoMT) exists that is specialized for services and applications based on multimedia data. However, IoMT generated data are facing major processing and privacy issues. Therefore, tensor-based deep computation models proved a better platform to process IoMT generated data. A differentially private deep computation method working in the tensor space can attest to its efficacy for IoMT. Nevertheless, the deep computation model comprises a multitude of parameters; thus, it requires large units of memory and expensive computing units with higher performance levels, which hinders its performance for IoMT. Motivated by this, therefore, the paper proposes a deep private tensor train autoencoder (dPTTAE) technique to deal with IoMT generated data. Notably, the compression of weight tensors to manageable tensor train format is achieved through Tensor Train (TT) network. Moreover, TT format parameters are trained through higher-order back-propagation and gradient descent. We applied dPTTAE on three representative datasets. Comprehensive experimental evaluations and theoretical analysis show that dPTTAE enhances training time efficiency, and greatly improve memory utilization efficiency, attesting its potential for IoMT.

References

  1. Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 308--318.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Rakesh Agrawal and Ramakrishnan Srikant. 2000. Privacy-preserving data mining. In ACM Sigmod Record, Vol. 29. ACM, 439--450.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Sheeraz A. Alvi, Bilal Afzal, Ghalib A. Shah, Luigi Atzori, and Waqar Mahmood. 2015. Internet of multimedia things: Vision and challenges. Ad Hoc Networks 33 (2015), 87--111.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Asra Aslam and Edward Curry. 2018. Towards a generalized approach for deep neural network based event processing for the internet of multimedia things. IEEE Access 6 (2018), 25573--25587.Google ScholarGoogle ScholarCross RefCross Ref
  5. Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The internet of things: A survey. Computer Networks 54, 15 (2010), 2787--2805.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yoshua Bengio, et al. 2009. Learning deep architectures for AI. Foundations and Trends® in Machine Learning 2, 1 (2009), 1--127.Google ScholarGoogle Scholar
  7. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 8 (2013), 1798--1828.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Wenlin Chen, James Wilson, Stephen Tyree, Kilian Weinberger, and Yixin Chen. 2015. Compressing neural networks with the hashing trick. In International Conference on Machine Learning. 2285--2294.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Xue-Wen Chen and Xiaotong Lin. 2014. Big data deep learning: Challenges and perspectives. IEEE Access 2 (2014), 514--525.Google ScholarGoogle ScholarCross RefCross Ref
  10. Andrzej Cichocki. 2014. Era of big data processing: A new approach via tensor networks and tensor decompositions. arXiv preprint arXiv:1403.2048 (2014).Google ScholarGoogle Scholar
  11. Adam Coates, Andrew Ng, and Honglak Lee. 2011. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. 215--223.Google ScholarGoogle Scholar
  12. Cynthia Dwork. 2008. Differential privacy: A survey of results. In International Conference on Theory and Applications of Models of Computation. Springer, 1--19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography Conference. Springer, 265--284.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jun Feng, Laurence T. Yang, Xingang Liu, and Ronghao Zhang. 2018. Privacy-preserving tensor analysis and processing models for wireless Internet of Things. IEEE Wireless Communications 25, 6 (2018), 98--103.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jun Feng, Laurence T. Yang, Xin Nie, and Nicholaus J. Gati. 2020. Edge-cloud-aided differentially private tucker decomposition for cyber-physical-social systems. IEEE Internet of Things Journal (2020), DOI:10.1109/JIOT.2020.3004826Google ScholarGoogle Scholar
  16. Jun Feng, Laurence T. Yang, and Ronghao Zhang. 2019. Practical privacy-preserving high-order bi-lanczos in integrated edge-fog-cloud architecture for cyber-physical-social systems. ACM Transactions on Internet Technology 19, 2 (2019), 26.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jun Feng, Laurence T. Yang, Ronghao Zhang, and Benard S. Gavuna. 2020. Privacy preserving tucker train decomposition over blockchain-based encrypted industrial IoT data. IEEE Transactions on Industrial Informatics (2020), DOI:10.1109/TII.2020.2968923Google ScholarGoogle Scholar
  18. Jun Feng, Laurence T. Yang, Ronghao Zhang, Weizhong Qiang, and Jinjun Chen. 2020. Privacy preserving high-order bi-lanczos in cloud-fog computing for industrial applications. IEEE Transactions on Industrial Informatics (2020), DOI:10.1109/TII.2020.2998086Google ScholarGoogle ScholarCross RefCross Ref
  19. Jun Feng, Laurence T. Yang, Qing Zhu, and Kim-Kwang Raymond Choo. 2018. Privacy-preserving tensor decomposition over encrypted data in a federated cloud environment. IEEE Transactions on Dependable and Secure Computing (2018), DOI:10.1109/TDSC.2018.2881452Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Daojing He, Chun Chen, Sammy Chan, Jiajun Bu, and Laurence T. Yang. 2012. Security analysis and improvement of a secure and distributed reprogramming protocol for wireless sensor networks. IEEE Transactions on Industrial Electronics 60, 11 (2012), 5348--5354.Google ScholarGoogle ScholarCross RefCross Ref
  21. Aparna Kumari, Sudeep Tanwar, Sudhanshu Tyagi, Neeraj Kumar, Michele Maasberg, and Kim-Kwang Raymond Choo. 2018. Multimedia big data computing and Internet of Things applications: A taxonomy and process model. Journal of Network and Computer Applications 124 (2018), 169--195.Google ScholarGoogle ScholarCross RefCross Ref
  22. Yehuda Lindell and Benny Pinkas. 2000. Privacy preserving data mining. In Annual International Cryptology Conference. Springer, 36--54.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yang Liu, Yan Liu, and Keith C. C. Chan. 2010. Tensor distance based multilinear locality-preserved maximum information embedding. IEEE Transactions on Neural Networks 21, 11 (2010), 1848--1854.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Changqing Luo, Shengyong Guo, Song Guo, Laurence T. Yang, Geyong Min, and Xia Xie. 2014. Green communication in energy renewable wireless mesh networks: Routing, rate control, and power allocation. IEEE Transactions on Parallel and Distributed Systems 25, 12 (2014), 3211--3220.Google ScholarGoogle ScholarCross RefCross Ref
  25. Frank McSherry and Ratul Mahajan. 2011. Differentially-private network trace analysis. ACM SIGCOMM Computer Communication Review 41, 4 (2011), 123--134.Google ScholarGoogle Scholar
  26. Alexander Novikov, Dmitrii Podoprikhin, Anton Osokin, and Dmitry P. Vetrov. 2015. Tensorizing neural networks. In Advances in Neural Information Processing Systems. 442--450.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Eric K. Patterson, Sabri Gurbuz, Zekeriya Tufekci, and John N. Gowdy. 2002. CUAVE: A new audio-visual database for multimodal human-computer interface research. In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 2. IEEE, II--2017.Google ScholarGoogle Scholar
  28. NhatHai Phan, Yue Wang, Xintao Wu, and Dejing Dou. 2016. Differential privacy preservation for deep auto-encoders: An application of human behavior prediction. In Thirtieth AAAI Conference on Artificial Intelligence.Google ScholarGoogle Scholar
  29. Shalli Rani, Syed Hassan Ahmed, Rajneesh Talwar, Jyoteesh Malhotra, and Houbing Song. 2017. IoMT: A reliable cross layer protocol for internet of multimedia things. IEEE Internet of Things Journal 4, 3 (2017), 832--839.Google ScholarGoogle ScholarCross RefCross Ref
  30. Reza Shokri and Vitaly Shmatikov. 2015. Privacy-preserving deep learning. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS’15). ACM, New York, NY, USA, 1310--1321. DOI:https://doi.org/10.1145/2810103.2813687Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jun Zhang, Zhenjie Zhang, Xiaokui Xiao, Yin Yang, and Marianne Winslett. 2012. Functional mechanism: Regression analysis under differential privacy. Proceedings of the VLDB Endowment 5, 11 (2012), 1364--1375.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Qingchen Zhang, Man Lin, Laurence T. Yang, Zhikui Chen, and Peng Li. 2017. Energy-efficient scheduling for real-time systems based on deep Q-learning model. IEEE Transactions on Sustainable Computing 4, 1 (2017), 132--141.Google ScholarGoogle ScholarCross RefCross Ref
  33. Qingchen Zhang, Laurence T. Yang, Zhikui Chen, and Peng Li. 2018. A tensor-train deep computation model for industry informatics big data feature learning. IEEE Transactions on Industrial Informatics 14, 7 (2018), 3197--3204.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Differentially Private Tensor Train Deep Computation for Internet of Multimedia Things

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!