skip to main content
research-article

PrivacyPrimer: Towards Privacy-Preserving Episodic Memory Support For Older Adults

Authors Info & Claims
Published:18 October 2021Publication History
Skip Abstract Section

Abstract

Built-in pervasive cameras have become an integral part of mobile/wearable devices and enabled a wide range of ubiquitous applications with their ability to be "always-on". In particular, life-logging has been identified as a means to enhance the quality of life of older adults by allowing them to reminisce about their own life experiences. However, the sensitive images captured by the cameras threaten individuals' right to have private social lives and raise concerns about privacy and security in the physical world. This threat gets worse when image recognition technologies can link images to people, scenes, and objects, hence, implicitly and unexpectedly reveal more sensitive information such as social connections. In this paper, we first examine life-log images obtained from 54 older adults to extract (a) the artifacts or visual cues, and (b) the context of the image that influences an older life-logger's ability to recall the life events associated with a life-log image. We call these artifacts and contextual cues "stimuli". Using the set of stimuli extracted, we then propose a set of obfuscation strategies that naturally balances the trade-off between reminiscability and privacy (revealing social ties) while selectively obfuscating parts of the images. More specifically, our platform yields privacy-utility tradeoff by compromising, on average, modest 13.4% reminiscability scores while significantly improving privacy guarantees -- around 40% error in cloud estimation.

References

  1. Tousif Ahmed, Apu Kapadia, Venkatesh Potluri, and Manohar Swaminathan. 2018. Up to a Limit? Privacy Concerns of Bystanders and Their Willingness to Share Additional Information with Visually Impaired Users of Assistive Technologies. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 2, 3 (2018), 1--27.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Rawan Alharbi, Mariam Tolba, Lucia C Petito, Josiah Hester, and Nabil Alshurafa. 2019. To Mask or Not to Mask?: Balancing Privacy with Visual Confirmation Utility in Activity-Oriented Wearable Cameras. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 3, 3 (2019), 72.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Yoann Baveye, Romain Cohendet, Matthieu Perreira Da Silva, and Patrick Le Callet. 2016. Deep learning for image memorability prediction: The emotional bias. In Proceedings of the 24th ACM international conference on Multimedia. ACM, 491--495.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Michael F Bergeron, Sara Landset, Franck Tarpin-Bernard, Curtis B Ashford, Taghi M Khoshgoftaar, and J Wesson Ashford. 2019. Episodic-Memory Performance in Machine Learning Modeling for Predicting Cognitive Health Status Classification. Journal of Alzheimer's Disease, Vol. 70, 1 (2019), 277--286.Google ScholarGoogle ScholarCross RefCross Ref
  5. Valérie Camos and Sophie Portrat. 2015. The impact of cognitive load on delayed recall. Psychonomic bulletin & review, Vol. 22, 4 (2015), 1029--1034.Google ScholarGoogle Scholar
  6. Tiffany E Chow and Jesse Rissman. 2017. Neurocognitive mechanisms of real-world autobiographical memory retrieval: insights from studies using wearable camera technology. Annals of the New York Academy of Sciences, Vol. 1396, 1 (2017), 202--221.Google ScholarGoogle ScholarCross RefCross Ref
  7. Soumyadeb Chowdhury, Md. Sadek Ferdous, and Joemon M. Jose. 2016. Bystander Privacy in Lifelogging. In BCS HCI.Google ScholarGoogle Scholar
  8. Dwork Cynthia. 2006. Differential privacy. Automata, languages and programming (2006), 1--12.Google ScholarGoogle Scholar
  9. Mariella Dimiccoli, Juan Mar'in, and Edison Thomaz. 2018. Mitigating Bystander Privacy Concerns in Egocentric Activity Recognition with Deep Learning and Intentional Image Degradation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, 4 (2018), 132.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Aiden R Doherty, Katalin Pauly-Takacs, Niamh Caprani, Cathal Gurrin, Chris JA Moulin, Noel E O'Connor, and Alan F Smeaton. 2012. Experiences of aiding autobiographical memory using the SenseCam. Human-Computer Interaction, Vol. 27, 1--2 (2012), 151--174.Google ScholarGoogle ScholarCross RefCross Ref
  11. Aaron Duane and Cathal Gurrin. 2018. Lifelog Exploration Prototype in Virtual Reality. In MMM.Google ScholarGoogle Scholar
  12. Cynthia Dwork. 2008. Differential privacy: A survey of results. In International conference on theory and applications of models of computation. Springer, 1--19.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Passant Elagroudy, Mohamed Khamis, Florian Mathis, Diana Irmscher, Andreas Bulling, and Albrecht Schmidt. 2019. Can Privacy-Aware Lifelogs Alter Our Memories?. In CHI Extended Abstracts.Google ScholarGoogle Scholar
  14. Md Sadek Ferdous, Soumyadeb Chowdhury, and Joemon M Jose. 2017. Analysing privacy in visual lifelogging. Pervasive and Mobile Computing, Vol. 40 (2017), 430--449.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Olga Gelonch, Mireia Ribera, Núria Codern-Bové, S'ilvia Ramos, Maria Quintana, Gloria Chico, Noem'i Cerulla, Paula Lafarga, Petia Radeva, and Maite Garolera. 2019. Acceptability of a lifelogging wearable camera in older adults with mild cognitive impairment: a mixed-method study. BMC geriatrics, Vol. 19, 1 (2019), 110.Google ScholarGoogle Scholar
  16. Arushi Goel, Keng Teck Ma, and Cheston Tan. 2019. An End-to-End Network for Generating Social Relationship Graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 11186--11195.Google ScholarGoogle ScholarCross RefCross Ref
  17. Cathal Gurrin, Rami Albatal, Hideo Joho, and Kaori Ishii. 2014a. A privacy by design approach to lifelogging.Google ScholarGoogle Scholar
  18. Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014b. LifeLogging: Personal Big Data. Foundations and Trends in Information Retrieval, Vol. 8, 1 (2014), 1--125. https://doi.org/10.1561/1500000033Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. Harvey, M.and Langheinrich and G. Ward. 2016. Remembering through lifelogging: A survey of human memory augmentation. Pervasive and Mobile Computing, Vol. 27 (01 2016). https://doi.org/10.1016/j.pmcj.2015.12.002Google ScholarGoogle Scholar
  20. Rakibul Hasan, David Crandall, Mario Fritz, and Apu Kapadia. 2020. Automatically Detecting Bystanders in Photos to Reduce Privacy Risks. In Proceedings of the 2020 IEEE Symposium on Security and Privacy (SP).Google ScholarGoogle ScholarCross RefCross Ref
  21. Rakibul Hasan, Eman Hassan, Yifang Li, Kelly Caine, David J Crandall, Roberto Hoyle, and Apu Kapadia. 2018a. Viewer experience of obscuring scene elements in photos to enhance privacy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 47.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Rakibul Hasan, Eman T. Hassan, Yifang Li, Kelly Caine, David J. Crandall, Roberto Hoyle, and Apu Kapadia. 2018b. Viewer Experience of Obscuring Scene Elements in Photos to Enhance Privacy. In CHI.Google ScholarGoogle Scholar
  23. Rakibul Hasan, Yifang Li, Eman Hassan, Kelly Caine, David J Crandall, Roberto Hoyle, and Apu Kapadia. 2019. Can privacy be satisfying? on improving viewer satisfaction for privacy-enhanced photos using aesthetic transforms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Roberto Hoyle, Robert Templeman, Denise Anthony, David Crandall, and Apu Kapadia. 2015. Sensitive lifelogs: A privacy analysis of photos from wearable cameras. In Proceedings of the 33rd Annual ACM conference on human factors in computing systems. ACM, 1645--1648.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Roberto Hoyle, Robert Templeman, Steven Armes, Denise Anthony, David Crandall, and Apu Kapadia. 2014. Privacy behaviors of lifeloggers using wearable cameras. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 571--582.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Panagiotis Ilia, Iasonas Polakis, Elias Athanasopoulos, Federico Maggi, and Sotiris Ioannidis. 2015. Face/Off: Preventing Privacy Leakage From Photos in Social Networks. In ACM Conference on Computer and Communications Security.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Phillip Isola, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2011. What makes an image memorable?. In CVPR 2011. IEEE, 145--152.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Peggy L St Jacques, Christopher Olm, and Daniel L Schacter. 2013. Neural mechanisms of reactivation-induced updating that enhance and distort memory. Proceedings of the National Academy of Sciences, Vol. 110, 49 (2013), 19671--19678.Google ScholarGoogle ScholarCross RefCross Ref
  29. Kasthuri Jayarajah, Archan Misra, Xiao-Wen Ruan, and Ee-Peng Lim. 2015. Event detection: Exploiting socio-physical interactions in physical spaces. In 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 508--513.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Soon-Gyo Jung, Jisun An, Haewoon Kwak, Joni Salminen, and Bernard Jim Jansen. 2018. Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. In Twelfth International AAAI Conference on Web and Social Media.Google ScholarGoogle Scholar
  31. Thivya Kandappu, Arik Friedman, Roksana Boreli, and Vijay Sivaraman. 2014. PrivacyCanary: Privacy-aware recommenders with adaptive input obfuscation. In 2014 IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems. IEEE, 453--462.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Thivya Kandappu, Archan Misra, and Randy Tandriansyah. 2017. Collaboration Trumps Homophily in Urban Mobile Crowdsourcing. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 902--915.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Aditya Khosla, Atish Das Sarma, and Raffay Hamid. 2014. What makes an image popular?. In Proceedings of the 23rd international conference on World wide web. 867--876.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Aditya Khosla, Akhil S. Raju, Antonio Torralba, and Aude Oliva. 2015. Understanding and Predicting Image Memorability at a Large Scale. In International Conference on Computer Vision (ICCV).Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Marion Koelle, Swamy Ananthanarayan, Simon Czupalla, Wilko Heuten, and Susanne Boll. 2018. Your smart glasses' camera bothers me! exploring opt-in and opt-out gestures for privacy mediation. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction. 473--481.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Marc Langheinrich. 2001. Privacy by Design - Principles of Privacy-Aware Ubiquitous Systems. In UbiComp.Google ScholarGoogle Scholar
  37. Hady W Lauw, Ee-Peng Lim, Hweehwa Pang, and Teck-Tim Tan. 2010. STEvent: Spatio-temporal event model for social network discovery. ACM Transactions on Information Systems (TOIS), Vol. 28, 3 (2010), 15.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Matthew L Lee and Anind K Dey. 2008. Lifelogging memory appliance for people with episodic memory impairment. In Proceedings of the 10th international conference on Ubiquitous computing. ACM, 44--53.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Junnan Li, Yongkang Wong, Qi Zhao, and Mohan S Kankanhalli. 2017c. Dual-glance model for deciphering social relationships. In Proceedings of the IEEE International Conference on Computer Vision. 2650--2659.Google ScholarGoogle ScholarCross RefCross Ref
  40. Yifang Li, Nishant Vishwamitra, Bart P Knijnenburg, Hongxin Hu, and Kelly Caine. 2017a. Effectiveness and users' experience of obfuscation as a privacy-enhancing technology for sharing photos. Proceedings of the ACM on Human-Computer Interaction, Vol. 1, CSCW (2017), 1--24.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yifang Li, Nishant Vishwamitra, Bart P. Knijnenburg, Hongxin Hu, and Kelly Caine. 2017b. Effectiveness and Users' Experience of Obfuscation as a Privacy-Enhancing Technology for Sharing Photos. PACMHCI, Vol. 1 (2017), 67:1--67:24.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Catherine Loveday and Martin A. Conway. 2011. Using SenseCam with an amnesic patient: accessing inaccessible everyday memories. Memory, Vol. 19 7 (2011), 697--704.Google ScholarGoogle ScholarCross RefCross Ref
  43. Francesco Marini, Jerry Scott, Adam R Aron, and Edward F Ester. 2017. Task-irrelevant distractors in the delay period interfere selectively with visual short-term memory for spatial locations. Attention, Perception, & Psychophysics, Vol. 79, 5 (2017), 1384--1392.Google ScholarGoogle ScholarCross RefCross Ref
  44. Sascha RA Meyer, Jos FM De Jonghe, Ben Schmand, and Rudolf WHM Ponds. 2019. Visual associations to retrieve episodic memory across healthy elderly, mild cognitive impairment, and patients with Alzheimer's disease. Aging, Neuropsychology, and Cognition, Vol. 26, 3 (2019), 447--462.Google ScholarGoogle ScholarCross RefCross Ref
  45. F Milton, Nils Muhlert, CR Butler, A Benattayallah, and Adam Z Zeman. 2011. The neural correlates of everyday recognition memory. Brain and Cognition, Vol. 76, 3 (2011), 369--381.Google ScholarGoogle ScholarCross RefCross Ref
  46. Christiane SH Oedekoven, James L Keidel, Sam C Berens, and Chris M Bird. 2017. Reinstatement of memory representations for lifelike events over the course of a week. Scientific Reports, Vol. 7, 1 (2017), 1--12.Google ScholarGoogle ScholarCross RefCross Ref
  47. Tribhuvanesh Orekondy, Mario Fritz, and Bernt Schiele. 2018. Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarGoogle ScholarCross RefCross Ref
  48. Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2017. Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images. In ICCV.Google ScholarGoogle Scholar
  49. K. O'hara, Mischa Moussavian Tuffield, and Nigel Shadbolt. 2008. Lifelogging: Privacy and empowerment with memories for life. Identity in the Information Society, Vol. 1 (2008), 155--172.Google ScholarGoogle ScholarCross RefCross Ref
  50. Michael S Ryoo, Brandon Rothrock, Charles Fleming, and Hyun Jong Yang. 2017. Privacy-preserving human activity recognition from extreme low resolution. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. 4255--4262.Google ScholarGoogle ScholarCross RefCross Ref
  51. A. J. Sellen, A. Fogg, M. Aitken, S. Hodges, R. Carsten, and K. Wood. 2007. Do Life-logging Technologies Support Memory for the Past?: An Experimental Study Using Sensecam. In CHI '07 (San Jose, California, USA). 81--90. https://doi.org/10.1145/1240624.1240636Google ScholarGoogle Scholar
  52. Zhiqi Shen, Shaojing Fan, Yongkang Wong, Tian-Tsong Ng, and Mohan Kankanhalli. 2019. Human-imperceptible privacy protection against machines. In Proceedings of the 27th ACM International Conference on Multimedia. 1119--1128.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. A.R. Silva, M.S. Pinho, L. Macedo, and C.J.A. Moulin. 2018. A critical review of the effects of wearable cameras on memory. Neuropsychological Rehabilitation, Vol. 28, 1 (2018), 117--141.Google ScholarGoogle ScholarCross RefCross Ref
  54. R Nathan Spreng, Raymond A Mar, and Alice SN Kim. 2009. The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: a quantitative meta-analysis. Journal of cognitive neuroscience, Vol. 21, 3 (2009), 489--510.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Eleftherios Spyromitros-Xioufis, Georgios Petkos, Symeon Papadopoulos, Rob Heyman, and Yiannis Kompatsiaris. 2016. Perceived versus actual predictability of personal information in social networks. In International Conference on Internet Science. Springer, 133--147.Google ScholarGoogle ScholarCross RefCross Ref
  56. Peggy L St. Jacques, Martin A Conway, Matthew W Lowder, and Roberto Cabeza. 2011. Watching my mind unfold versus yours: An fMRI study using a novel camera technology to examine neural differences in self-projection of self versus other perspectives. Journal of Cognitive Neuroscience, Vol. 23, 6 (2011), 1275--1284.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Julian Steil, Marion Koelle, Wilko Heuten, Susanne Boll, and Andreas Bulling. 2019. Privaceye: privacy-preserving head-mounted eye tracking using egocentric scene image and eye movement features. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications. 1--10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Ashwini Tonge and Cornelia Caragea. 2019. Dynamic deep multi-modal fusion for image privacy prediction. In The World Wide Web Conference. ACM, 1829--1840.Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Igor Vasiljevic, Ayan Chakrabarti, and Gregory Shakhnarovich. 2016. Examining the impact of blur on recognition by convolutional networks. arXiv preprint arXiv:1611.05760 (2016).Google ScholarGoogle Scholar
  60. Hui-Po Wang, Tribhuvanesh Orekondy, and Mario Fritz. 2020. InfoScrub: Towards Attribute Privacy by Targeted Obfuscation. arXiv preprint arXiv:2005.10329 (2020).Google ScholarGoogle Scholar
  61. Zhenyu Wu, Zhangyang Wang, Zhaowen Wang, and Hailin Jin. 2018. Towards privacy-preserving visual recognition via adversarial training: A pilot study. In Proceedings of the European Conference on Computer Vision (ECCV). 606--624.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Qianli Xu, Vigneshwaran Subbaraju, Chee How Cheong, Aijing Wang, Kathleen Kang, Munirah Bashir, Yanhong Dong, Liyuan Li, and Joo Hwee Lim. 2018. Personalized Serious Games for Cognitive Intervention with Lifelog Visual Analytics. In ACM Multimedia.Google ScholarGoogle Scholar
  63. Xin Ye and Jiro Tanaka. 2019. Aiding Episodic Memory in Lifelog System Focusing on User Status. In HCI.Google ScholarGoogle Scholar
  64. Juheon Yi, Sunghyun Choi, and Youngki Lee. 2020. EagleEye: Wearable camera-based person identification in crowded urban spaces. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Wei Liu, and Hongdong Li. 2018. Adversarial spatio-temporal learning for video deblurring. IEEE Transactions on Image Processing, Vol. 28, 1 (2018), 291--301.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Haoti Zhong, Anna Squicciarini, David Miller, and Cornelia Caragea. 2017. A Group-Based Personalized Model for Image Privacy Classification and Labeling. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. 3952--3958. https://doi.org/10.24963/ijcai.2017/552Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. PrivacyPrimer: Towards Privacy-Preserving Episodic Memory Support For Older Adults

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Article Metrics

          • Downloads (Last 12 months)53
          • Downloads (Last 6 weeks)4

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!