Abstract
We show how to build the components of a privacy-aware, live video analytics ecosystem from the bottom up, starting with OpenFace, our new open-source face recognition system that approaches state-of-the-art accuracy. Integrating OpenFace with interframe tracking, we build RTFace, a mechanism for denaturing video streams that selectively blurs faces according to specified policies at full frame rates. This enables privacy management for live video analytics while providing a secure approach for handling retrospective policy exceptions. Finally, we present a scalable, privacy-aware architecture for large camera networks using RTFace and show how it can be an enabler for a vibrant ecosystem and marketplace of privacy-aware video streams and analytics services.
- Paarijaat Aditya, Rijurekha Sen, Peter Druschel, Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele, Bobby Bhattacharjee, and Tong Tong Wu. 2016. I-pic: A platform for privacy-compliant image capture. In Proceedings of ACM MobiSys. Google Scholar
Digital Library
- US Energy Information Administration. 2017. Electric power monthly. Retrieved from https://www.eia.gov/electricity/monthly/epm_table_grapher.php.Google Scholar
- Timo Ahonen, Abdenour Hadid, and Matti Pietikäinen. 2006. Face description with local binary patterns: Application to face recognition. IEEE Transactions on PAMI 28, 12 (December 2006), 2037--2041. Google Scholar
Digital Library
- Amazon. 2017. EC2 instance types. Retrieved from https://aws.amazon.com/ec2/instance-types.Google Scholar
- Amazon. 2017. EC2 pricing. Retrieved from https://aws.amazon.com/ec2/pricing/on-demand/.Google Scholar
- Amazon. 2017. EC2 reserved instances pricing. Retrieved from https://aws.amazon.com/ec2/pricing/reserved-instances/pricing.Google Scholar
- Amazon. 2017. EC2 spot instances pricing. Retrieved from https://aws.amazon.com/ec2/spot/pricing/.Google Scholar
- Amazon. 2017. S3 pricing. Retrieved from https://aws.amazon.com/s3/pricing/.Google Scholar
- Amazon. 2017. Seagate 1TB IronWolf NAS internal hard drive. Retrieved from https://www.amazon.com/Seagate-IronWolf-3-5-Inch-Internal-ST1000VN002/dp/B01LOOJ8TE.Google Scholar
- Amazon. 2017. Survelliance camera prices. Retrieved from https://www.amazon.com/Reolink-Security-Megapixels-2560x1440-Optical/dp/B016UCNP08.Google Scholar
- David Barrett. 2013. One surveillance camera for every 11 people in britain, says CCTV survey. Daily Telegraph (July 10, 2013).Google Scholar
- Peter N. Belhumeur, João P. Hespanha, and David J. Kriegman. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on PAMI 19, 7 (July 1997), 711--720. Google Scholar
Digital Library
- Cheng Bo, Guobin Shen, Jie Liu, Xiang-Yang Li, YongGuang Zhang, and Feng Zhao. 2014. Privacy tag: Privacy concern expressed and respected. In Proceedings of ACM SenSys. Google Scholar
Digital Library
- Tiffany Yu-Han Chen, Lenin Ravindranath, Shuo Deng, Paramvir Bahl, and Hari Balakrishnan. 2015. Glimpse: Continuous, real-time object recognition on mobile devices. In Proceedings of ACM SenSys. Google Scholar
Digital Library
- Ronan Collobert, Koray Kavukcuoglu, and Clément Farabet. 2011. Torch7: A Matlab-like environment for machine learning. In BigLearn, NIPS Workshop.Google Scholar
- Stealth Communications. 2017. Dedicated gigabit Internet access for businesses in New York City. Retrieved from https://stealth.net/services/fiber/nyc/gigabit/dedicated.Google Scholar
- Western Digital Corporation. 2017. My Cloud Pro Series PR4100. Retrieved from https://www.wdc.com/products/network-attached-storage/my-cloud-pr4100.html.Google Scholar
- Martin Danelljan, Gustav Häger, Fahad Khan, and Michael Felsberg. 2014. Accurate scale estimation for robust visual tracking. In Proceedings of the British Machine Vision Conference.Google Scholar
Cross Ref
- Anupam Das, Martin Degeling, Xiaoyou Wang, Junjue Wang, Norman Sadeh, and Mahadev Satyanarayanan. 2017. Assisting users in a world full of cameras: A privacy-aware infrastructure for computer vision applications. In Proceedings of IEEE CVPR Workshops.Google Scholar
Cross Ref
- Nigel Davies, Nina Taft, Mahadev Satyanarayanan, Sarah Clinch, and Brandon Amos. 2016. Privacy mediators: Helping IoT cross the chasm. In Proceedings of ACM HotMobile 2016. Google Scholar
Digital Library
- Debate.org. 2017. Are video surveillance cameras in public places a good idea? Retrieved from http://debate.org.Google Scholar
- Dell. 2017. PowerEdge R430 Server Intel ® Xeon ® E5-2683 v4. Retrieved from https://tinyurl.com/ybxajmop.Google Scholar
- Ralph Gross, Latanya Sweeney, Fernando De la Torre, and Simon Baker. 2006. Model-based face de-identification. In Proceedings of IEEE CVPR Workshop. Google Scholar
Digital Library
- Kiryong Ha and Mahadev Satyanarayanan. 2015. Openstack++ for Cloudlet Deployment. Technical Report CMU-CS-15-123. School of Computer Science, Carnegie Mellon University.Google Scholar
- YouTube Help. 2017. Recommended upload encoding settings. Retrieved from https://support.google.com/youtube/answer/1722171?hl=en.Google Scholar
- Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07-49. University of Massachusetts, Amherst.Google Scholar
- Labeled Faces in the Wild. 2017. LFW results. Retrieved from http://vis-www.cs.umass.edu/lfw/results.htm.Google Scholar
- Intel. 2016. Milestone leverages Intel processors with Intel quick sync video to create breakthrough capabilities for video surveillance and monitoring. Retrieved from http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/milestone-software-quick-sync-video-surveillance-monitoring-white-paper.pdf.Google Scholar
- Rabia Jafri and Hamid Arabnia. 2009. A survey of face recognition techniques. Journal of Information Processing Systems 5, 2 (2009), 41--68.Google Scholar
Cross Ref
- Suman Jana, Arvind Narayanan, and Vitaly Shmatikov. 2013. A scanner darkly: Protecting user privacy from perceptual applications. In IEEE Symposium on Security and Privacy. Google Scholar
Digital Library
- Tony S. Jebara. 1995. 3D Pose Estimation and Normalization for Face Recognition. Ph.D. Dissertation. McGill University.Google Scholar
- Michal Kampf, Israel Nachson, and Harvey Babkoff. 2002. A serial test of the laterality of familiar face recognition. Brain and Cognition 50, 1 (2002), 35--50.Google Scholar
Cross Ref
- Takeo Kanade. 1973. Picture Processing System by Computer Complex and Recognition of Human Faces. Ph.D. Dissertation. Kyoto University.Google Scholar
- Minyoung Kim, Sanjiv Kumar, Vladimir Pavlovic, and Henry Rowley. 2008. Face tracking and recognition with visual constraints in real-world videos. In Proceedings of IEEE CVPR.Google Scholar
- Davis E. King. 2009. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research 10 (2009), 1755--1758. Google Scholar
Digital Library
- Suleyman Serdar Kozat, Ramarathnam Venkatesan, and Mehmet Kivanç Mihçak. 2004. Robust perceptual image hashing via matrix invariants. In Proceedings of IEEE ICIP. 3443--3446.Google Scholar
Cross Ref
- Vishal Monga and Brian L. Evans. 2006. Perceptual image hashing via feature points: Performance evaluation and tradeoffs. IEEE Transactions on Image Processing 15, 11 (2006), 3452--3465. Google Scholar
Digital Library
- Pardis Emami Naeini, Sruti Bhagavatula, Hana Habib, Martin Degeling, Lujo Bauer, Lorrie Cranor, and Norman Sadeh. 2017. Privacy expectations and preferences in an IoT world. In Proceedings of SOUPS. Google Scholar
Digital Library
- Netflix. 2017. Internet connection speed recommendations. Retrieved from https://help.netflix.com/en/node/306.Google Scholar
- Elaine Newton, Latanya Sweeney, and Bradley Malin. 2005. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering 17, 2 (2005), 232--243. Google Scholar
Digital Library
- Hong-Wei Ng and Stefan Winkler. 2014. A data-driven approach to cleaning large face datasets. In Proceedings of IEEE ICIP. 343--347.Google Scholar
Cross Ref
- Meike Ramon, Stephanie Caharel, and Bruno Rossion. 2011. The speed of recognition of personally familiar faces. Perception 40, 4 (2011), 437--449.Google Scholar
Cross Ref
- Nisarg Raval, Animesh Srivastava, Ali Razeen, Kiron Lebeck, Ashwin Machanavajjhala, and Landon P. Cox. 2016. What you mark is what apps see. In Proceedings of ACM MobiSys. Google Scholar
Digital Library
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211--252. Google Scholar
Digital Library
- Jeremy Schiff, Marci Meingast, Deirdre K. Mulligan, Shankar Sastry, and Ken Goldberg. 2007. Respectful cameras: Detecting visual markers in real-time to address privacy concerns. In Proceedings of IEEE/RSJ Intelligent Robots and Systems.Google Scholar
Cross Ref
- Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of IEEE CVPR.Google Scholar
Cross Ref
- Andrew Senior, Sharath Pankanti, Arun Hampapur, Lisa Brown, Ying-Li Tian, Ahmet Ekin, Jonathan Connell, Chiao Fe Shu, and Max Lu. 2005. Enabling video privacy through computer vision. IEEE Security 8 Privacy 3 (2005), 50--57. Google Scholar
Digital Library
- Pieter Simoens, Yu Xiao, Padmanablan Pillai, Zhuo Chen, Kiryong Ha, and Mahadev Satyanarayanan. 2013. Scalable crowd-sourcing of video from mobile devices. In Proceedings of ACM MobiSys. Google Scholar
Digital Library
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of IEEE CVPR.Google Scholar
Cross Ref
- Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. 2014. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of IEEE CVPR. Google Scholar
Digital Library
- Carnegie Mellon University Personalized Privacy Assistant Team. 2017. Personalized Privacy Assistant Project. Retrieved from http://privacyassistant.org/.Google Scholar
- Seagate Technology. 2014. Video surveillance trends report. Retrieved from http://www.seagate.com/files/www-content/solutions-content/surveillance-security-video-analytics/en-us/docs/video-surveillance-trends-report.pdf.Google Scholar
- Matthew Turk and Alex Pentland. 1991. Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 1 (1991), 71--86. Google Scholar
Digital Library
- Junjue Wang, Brandon Amos, Anupam Das, Padmanabhan Pillai, Norman Sadeh, and Mahadev Satyanarayanan. 2017. A scalable and privacy-aware IoT service for live video analytics. In Proceedings of the 8th ACM on Multimedia Systems Conference. Google Scholar
Digital Library
- Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z Li. 2014. Learning face representation from scratch. Arxiv Preprint Arxiv:1411.7923 (2014).Google Scholar
- YouTube and Tubefilter. 2017. Hours of video uploaded to YouTube every minute as of July 2015. Retrieved from https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute.Google Scholar
Index Terms
Enabling Live Video Analytics with a Scalable and Privacy-Aware Framework
Recommendations
A Scalable and Privacy-Aware IoT Service for Live Video Analytics
MMSys'17: Proceedings of the 8th ACM on Multimedia Systems ConferenceWe present OpenFace, our new open-source face recognition system that approaches state-of-the-art accuracy. Integrating OpenFace with inter-frame tracking, we build RTFace, a mechanism for denaturing video streams that selectively blurs faces according ...
Privacy preserving security using biometrics in cloud computing
Cloud computing and the efficient storage provide new paradigms and approaches designed at efficiently utilization of resources through computation and many alternatives to guarantee the privacy preservation of individual user. It also ensures the ...
Towards efficient privacy-preserving face recognition in the cloud
Highlights- We propose a randomness-based privacy-preserving face recognition scheme.
- We ...
AbstractFace recognition (FR) has become increasingly significant in many computer vision applications. However, with the rapid deployment of FR, the privacy of face images has been a growing concern, especially when FR is performed in cloud ...






Comments