skip to main content
research-article
Open Access

Enabling Live Video Analytics with a Scalable and Privacy-Aware Framework

Published:15 June 2018Publication History
Skip Abstract Section

Abstract

We show how to build the components of a privacy-aware, live video analytics ecosystem from the bottom up, starting with OpenFace, our new open-source face recognition system that approaches state-of-the-art accuracy. Integrating OpenFace with interframe tracking, we build RTFace, a mechanism for denaturing video streams that selectively blurs faces according to specified policies at full frame rates. This enables privacy management for live video analytics while providing a secure approach for handling retrospective policy exceptions. Finally, we present a scalable, privacy-aware architecture for large camera networks using RTFace and show how it can be an enabler for a vibrant ecosystem and marketplace of privacy-aware video streams and analytics services.

References

  1. Paarijaat Aditya, Rijurekha Sen, Peter Druschel, Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele, Bobby Bhattacharjee, and Tong Tong Wu. 2016. I-pic: A platform for privacy-compliant image capture. In Proceedings of ACM MobiSys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. US Energy Information Administration. 2017. Electric power monthly. Retrieved from https://www.eia.gov/electricity/monthly/epm_table_grapher.php.Google ScholarGoogle Scholar
  3. Timo Ahonen, Abdenour Hadid, and Matti Pietikäinen. 2006. Face description with local binary patterns: Application to face recognition. IEEE Transactions on PAMI 28, 12 (December 2006), 2037--2041. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Amazon. 2017. EC2 instance types. Retrieved from https://aws.amazon.com/ec2/instance-types.Google ScholarGoogle Scholar
  5. Amazon. 2017. EC2 pricing. Retrieved from https://aws.amazon.com/ec2/pricing/on-demand/.Google ScholarGoogle Scholar
  6. Amazon. 2017. EC2 reserved instances pricing. Retrieved from https://aws.amazon.com/ec2/pricing/reserved-instances/pricing.Google ScholarGoogle Scholar
  7. Amazon. 2017. EC2 spot instances pricing. Retrieved from https://aws.amazon.com/ec2/spot/pricing/.Google ScholarGoogle Scholar
  8. Amazon. 2017. S3 pricing. Retrieved from https://aws.amazon.com/s3/pricing/.Google ScholarGoogle Scholar
  9. Amazon. 2017. Seagate 1TB IronWolf NAS internal hard drive. Retrieved from https://www.amazon.com/Seagate-IronWolf-3-5-Inch-Internal-ST1000VN002/dp/B01LOOJ8TE.Google ScholarGoogle Scholar
  10. Amazon. 2017. Survelliance camera prices. Retrieved from https://www.amazon.com/Reolink-Security-Megapixels-2560x1440-Optical/dp/B016UCNP08.Google ScholarGoogle Scholar
  11. David Barrett. 2013. One surveillance camera for every 11 people in britain, says CCTV survey. Daily Telegraph (July 10, 2013).Google ScholarGoogle Scholar
  12. Peter N. Belhumeur, João P. Hespanha, and David J. Kriegman. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on PAMI 19, 7 (July 1997), 711--720. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cheng Bo, Guobin Shen, Jie Liu, Xiang-Yang Li, YongGuang Zhang, and Feng Zhao. 2014. Privacy tag: Privacy concern expressed and respected. In Proceedings of ACM SenSys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Tiffany Yu-Han Chen, Lenin Ravindranath, Shuo Deng, Paramvir Bahl, and Hari Balakrishnan. 2015. Glimpse: Continuous, real-time object recognition on mobile devices. In Proceedings of ACM SenSys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Ronan Collobert, Koray Kavukcuoglu, and Clément Farabet. 2011. Torch7: A Matlab-like environment for machine learning. In BigLearn, NIPS Workshop.Google ScholarGoogle Scholar
  16. Stealth Communications. 2017. Dedicated gigabit Internet access for businesses in New York City. Retrieved from https://stealth.net/services/fiber/nyc/gigabit/dedicated.Google ScholarGoogle Scholar
  17. Western Digital Corporation. 2017. My Cloud Pro Series PR4100. Retrieved from https://www.wdc.com/products/network-attached-storage/my-cloud-pr4100.html.Google ScholarGoogle Scholar
  18. Martin Danelljan, Gustav Häger, Fahad Khan, and Michael Felsberg. 2014. Accurate scale estimation for robust visual tracking. In Proceedings of the British Machine Vision Conference.Google ScholarGoogle ScholarCross RefCross Ref
  19. Anupam Das, Martin Degeling, Xiaoyou Wang, Junjue Wang, Norman Sadeh, and Mahadev Satyanarayanan. 2017. Assisting users in a world full of cameras: A privacy-aware infrastructure for computer vision applications. In Proceedings of IEEE CVPR Workshops.Google ScholarGoogle ScholarCross RefCross Ref
  20. Nigel Davies, Nina Taft, Mahadev Satyanarayanan, Sarah Clinch, and Brandon Amos. 2016. Privacy mediators: Helping IoT cross the chasm. In Proceedings of ACM HotMobile 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Debate.org. 2017. Are video surveillance cameras in public places a good idea? Retrieved from http://debate.org.Google ScholarGoogle Scholar
  22. Dell. 2017. PowerEdge R430 Server Intel ® Xeon ® E5-2683 v4. Retrieved from https://tinyurl.com/ybxajmop.Google ScholarGoogle Scholar
  23. Ralph Gross, Latanya Sweeney, Fernando De la Torre, and Simon Baker. 2006. Model-based face de-identification. In Proceedings of IEEE CVPR Workshop. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Kiryong Ha and Mahadev Satyanarayanan. 2015. Openstack++ for Cloudlet Deployment. Technical Report CMU-CS-15-123. School of Computer Science, Carnegie Mellon University.Google ScholarGoogle Scholar
  25. YouTube Help. 2017. Recommended upload encoding settings. Retrieved from https://support.google.com/youtube/answer/1722171?hl=en.Google ScholarGoogle Scholar
  26. Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07-49. University of Massachusetts, Amherst.Google ScholarGoogle Scholar
  27. Labeled Faces in the Wild. 2017. LFW results. Retrieved from http://vis-www.cs.umass.edu/lfw/results.htm.Google ScholarGoogle Scholar
  28. Intel. 2016. Milestone leverages Intel processors with Intel quick sync video to create breakthrough capabilities for video surveillance and monitoring. Retrieved from http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/milestone-software-quick-sync-video-surveillance-monitoring-white-paper.pdf.Google ScholarGoogle Scholar
  29. Rabia Jafri and Hamid Arabnia. 2009. A survey of face recognition techniques. Journal of Information Processing Systems 5, 2 (2009), 41--68.Google ScholarGoogle ScholarCross RefCross Ref
  30. Suman Jana, Arvind Narayanan, and Vitaly Shmatikov. 2013. A scanner darkly: Protecting user privacy from perceptual applications. In IEEE Symposium on Security and Privacy. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Tony S. Jebara. 1995. 3D Pose Estimation and Normalization for Face Recognition. Ph.D. Dissertation. McGill University.Google ScholarGoogle Scholar
  32. Michal Kampf, Israel Nachson, and Harvey Babkoff. 2002. A serial test of the laterality of familiar face recognition. Brain and Cognition 50, 1 (2002), 35--50.Google ScholarGoogle ScholarCross RefCross Ref
  33. Takeo Kanade. 1973. Picture Processing System by Computer Complex and Recognition of Human Faces. Ph.D. Dissertation. Kyoto University.Google ScholarGoogle Scholar
  34. Minyoung Kim, Sanjiv Kumar, Vladimir Pavlovic, and Henry Rowley. 2008. Face tracking and recognition with visual constraints in real-world videos. In Proceedings of IEEE CVPR.Google ScholarGoogle Scholar
  35. Davis E. King. 2009. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research 10 (2009), 1755--1758. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Suleyman Serdar Kozat, Ramarathnam Venkatesan, and Mehmet Kivanç Mihçak. 2004. Robust perceptual image hashing via matrix invariants. In Proceedings of IEEE ICIP. 3443--3446.Google ScholarGoogle ScholarCross RefCross Ref
  37. Vishal Monga and Brian L. Evans. 2006. Perceptual image hashing via feature points: Performance evaluation and tradeoffs. IEEE Transactions on Image Processing 15, 11 (2006), 3452--3465. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Pardis Emami Naeini, Sruti Bhagavatula, Hana Habib, Martin Degeling, Lujo Bauer, Lorrie Cranor, and Norman Sadeh. 2017. Privacy expectations and preferences in an IoT world. In Proceedings of SOUPS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Netflix. 2017. Internet connection speed recommendations. Retrieved from https://help.netflix.com/en/node/306.Google ScholarGoogle Scholar
  40. Elaine Newton, Latanya Sweeney, and Bradley Malin. 2005. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering 17, 2 (2005), 232--243. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Hong-Wei Ng and Stefan Winkler. 2014. A data-driven approach to cleaning large face datasets. In Proceedings of IEEE ICIP. 343--347.Google ScholarGoogle ScholarCross RefCross Ref
  42. Meike Ramon, Stephanie Caharel, and Bruno Rossion. 2011. The speed of recognition of personally familiar faces. Perception 40, 4 (2011), 437--449.Google ScholarGoogle ScholarCross RefCross Ref
  43. Nisarg Raval, Animesh Srivastava, Ali Razeen, Kiron Lebeck, Ashwin Machanavajjhala, and Landon P. Cox. 2016. What you mark is what apps see. In Proceedings of ACM MobiSys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jeremy Schiff, Marci Meingast, Deirdre K. Mulligan, Shankar Sastry, and Ken Goldberg. 2007. Respectful cameras: Detecting visual markers in real-time to address privacy concerns. In Proceedings of IEEE/RSJ Intelligent Robots and Systems.Google ScholarGoogle ScholarCross RefCross Ref
  46. Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of IEEE CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  47. Andrew Senior, Sharath Pankanti, Arun Hampapur, Lisa Brown, Ying-Li Tian, Ahmet Ekin, Jonathan Connell, Chiao Fe Shu, and Max Lu. 2005. Enabling video privacy through computer vision. IEEE Security 8 Privacy 3 (2005), 50--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Pieter Simoens, Yu Xiao, Padmanablan Pillai, Zhuo Chen, Kiryong Ha, and Mahadev Satyanarayanan. 2013. Scalable crowd-sourcing of video from mobile devices. In Proceedings of ACM MobiSys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of IEEE CVPR.Google ScholarGoogle ScholarCross RefCross Ref
  50. Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. 2014. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of IEEE CVPR. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Carnegie Mellon University Personalized Privacy Assistant Team. 2017. Personalized Privacy Assistant Project. Retrieved from http://privacyassistant.org/.Google ScholarGoogle Scholar
  52. Seagate Technology. 2014. Video surveillance trends report. Retrieved from http://www.seagate.com/files/www-content/solutions-content/surveillance-security-video-analytics/en-us/docs/video-surveillance-trends-report.pdf.Google ScholarGoogle Scholar
  53. Matthew Turk and Alex Pentland. 1991. Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 1 (1991), 71--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Junjue Wang, Brandon Amos, Anupam Das, Padmanabhan Pillai, Norman Sadeh, and Mahadev Satyanarayanan. 2017. A scalable and privacy-aware IoT service for live video analytics. In Proceedings of the 8th ACM on Multimedia Systems Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. 2014. Learning face representation from scratch. Arxiv Preprint Arxiv:1411.7923 (2014).Google ScholarGoogle Scholar
  56. YouTube and Tubefilter. 2017. Hours of video uploaded to YouTube every minute as of July 2015. Retrieved from https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute.Google ScholarGoogle Scholar

Index Terms

  1. Enabling Live Video Analytics with a Scalable and Privacy-Aware Framework

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 3s
            Special Section on Delay-Sensitive Video Computing in the Cloud and Special Section on Extended MMSys-NOSSDAV Best Papers
            June 2018
            317 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/3233173
            Issue’s Table of Contents

            Copyright © 2018 Owner/Author

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 15 June 2018
            • Accepted: 1 April 2018
            • Revised: 1 February 2018
            • Received: 1 September 2017
            Published in tomm Volume 14, Issue 3s

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!