skip to main content
research-article

Data-driven Context Detection Leveraging Passively Sensed Nearables for Recognizing Complex Activities of Daily Living

Published:04 January 2021Publication History
Skip Abstract Section

Abstract

Wearable systems have unlocked new sensing paradigms in various applications such as human activity recognition, which can enhance effectiveness of mobile health applications. Current systems using wearables are not capable of understanding their surroundings, which limits their sensing capabilities. For instance, distinguishing certain activities such as attending a meeting or class, which have similar motion patterns but happen in different contexts, is challenging by merely using wearable motion sensors. This article focuses on understanding user's surroundings, i.e., environmental context, to enhance capability of wearables, with focus on detecting complex activities of daily living (ADL). We develop a methodology to automatically detect the context using passively observable information broadcasted by devices in users’ locale. This system does not require specific infrastructure or additional hardware. We develop a pattern extraction algorithm and probabilistic mapping between the context and activities to reduce the set of probable outcomes. The proposed system contains a general ADL classifier working with motion sensors, learns personalized context, and uses that to reduce the search space of activities to those that occur within a certain context. We collected real-world data of complex ADLs and by narrowing the search space with context, we improve average F1-score from 0.72 to 0.80.

References

  1. A. Akbari and R. Jafari. 2019. Transferring activity recognition models for new wearable sensors with deep generative domain adaptation. In Proceedings of the 18th International Conference on Information Processing in Sensor Networks 2019, 85--96.Google ScholarGoogle Scholar
  2. B. Ibrahim, J. McMurray, and R. Jafari. 2018. A wrist-worn strap with an array of electrodes for robust physiological sensing. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’18). 4313--4317.Google ScholarGoogle Scholar
  3. M. Esfahani, M. Iman, and M. A. Nussbaum. 2019. Classifying diverse physical activities using smart garments. Sensors 19, 14 (2019), 3133.Google ScholarGoogle Scholar
  4. P. Bharti, D. De, S. Chellappan, and S. K. Das. 2019. HuMAn: Complex activity recognition with multi-modal multi-positional body sensing. IEEE Trans. Mob. Comput. 18, 4 (2019), 857--870.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. R. Solis, A. Pakbin, A. Akbari, B. J. Mortazavi, and R. Jafari. 2019. A human-centered wearable sensing platform with intelligent automated data annotation capabilities. In Proceedings of the International Conference on Internet of Things Design and Implementation 2019, 255--260.Google ScholarGoogle Scholar
  6. G. Sprint, D. Cook, R. Fritz, and M. Schmitter-Edgecombe. 2016. Detecting health and behavior change by analyzing smart home sensor data. In Proceedings of the 2016 IEEE International Conference on Smart Computing (SMARTCOMP’16). 1--3.Google ScholarGoogle Scholar
  7. P. Bharti, A. Panwar, G. Gopalakrishna, and S. Chellappan. 2017. Watch-dog: Detecting self-harming activities from wrist worn accelerometers. IEEE J. Biomed. Heal. Inf. 22, 3 (2017), 686--696.Google ScholarGoogle Scholar
  8. L. Wang, T. Gu, X. Tao, H. Chen, and J. Lu. 2011. Recognizing multi-user activities using wearable sensors in a smart home. Perv Mob. Comput 7, 3 (2011), 287--298.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. Bulling, U. Blanke, and B. Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv 46, 3 (2014), 33.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Akbari, J. Wu, R. Grimsley, and R. Jafari. 2018. Hierarchical signal segmentation and classification for accurate activity recognition. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 1596--1605.Google ScholarGoogle Scholar
  11. A. Akbari and R. Jafari. 2019. An autoencoder-based approach for recognizing null class in activities of daily living in-the-wild via wearable motion sensors. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’19). 3392--3396.Google ScholarGoogle Scholar
  12. J. C. Krumm, E. J. Horvitz, and R. Hariharan. 2011. Integration of location logs, GPS signals, and spatial resources for identifying user activities, goals, and context. Google Patents, 2011.Google ScholarGoogle Scholar
  13. E. R. Sykes, S. Pentland, and S. Nardi. 2015. Context-aware mobile apps using iBeacons: Towards smarter interactions. In Proceedings of the 25th Annual International Conference on Computer Science and Software Engineering. 120--129.Google ScholarGoogle Scholar
  14. G. W. Musumba and H. O. Nyongesa. 2013. Context awareness in mobile computing: A review. Int. J. Mach. Learn. Appl. 2, 1 (2013), 5.Google ScholarGoogle Scholar
  15. M. Weiser. 1991. The computer for the 21st century IEEE Perv. Comput. 265, 3 (1991), 94--105.Google ScholarGoogle Scholar
  16. C. Perera, A. Zaslavsky, P. Christen, and D. Georgakopoulos. 2014. Context aware computing for the internet of things: A survey. IEEE Commun. Surv. Tutor. 16, 1 (2014), 414--454.Google ScholarGoogle ScholarCross RefCross Ref
  17. L. Herranz, S. Jiang, and R. Xu. 2016. Modeling restaurant context for food recognition. IEEE Trans. Multimed. 19, 2 (2016), 430--440.Google ScholarGoogle Scholar
  18. R. Reichle et al. 2008. A comprehensive context modeling framework for pervasive computing systems. In Proceedings of the IFIP International Conference on Distributed Applications and Interoperable Systems. 281--295.Google ScholarGoogle Scholar
  19. D. De, P. Bharti, S. K. Das, and S. Chellappan. 2015. Multimodal wearable sensing for fine-grained activity recognition in healthcare. IEEE Internet Comput. 19, 5 (2015), 26--35.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Filippoupolitis, W. Oliff, B. Takand, and G. Loukas. 2017. Location-enhanced activity recognition in indoor environments using off the shelf smart watch technology and BLE beacons. Sensors 17, 6 (2017), 1230.Google ScholarGoogle ScholarCross RefCross Ref
  21. L. Aalto, N. Göthlin, J. Korhonen, and T. Ojala. 2004. Bluetooth and WAP push based location-aware mobile advertising system. In Proceedings of the 2nd International Conference on Mobile Systems, Applications, and Services. 49--58.Google ScholarGoogle Scholar
  22. J. Zheng and L. M. Ni. 2013. An unsupervised learning approach to social circles detection in ego bluetooth proximity network. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 721--724.Google ScholarGoogle Scholar
  23. M. Han, J. Bang, C. Nugent, S. McClean, and S. Lee. 2014. A lightweight hierarchical activity recognition framework using smartphone sensors. Sensors 14, 9 (2014), 16181--16195.Google ScholarGoogle ScholarCross RefCross Ref
  24. A. Subramanya, A. Raj, J. A. Bilmes, and D. Fox. 2012. Recognizing activities and spatial context using wearable sensors. arXiv:1206.6869. Retrieved from https://arxiv.org/abs/1206.6869.Google ScholarGoogle Scholar
  25. J. Chon and H. Cha. 2011. Lifemap: A smartphone-based context provider for location-based services. IEEE Perv. Comput. 10, 2 (2011), 58--67.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. N. Bulusu, J. Heidemann, D. Estrin, and others. 2000. GPS-less low-cost outdoor localization for very small devices. IEEE Pers. Commun. 7, 5 (2000), 28--34.Google ScholarGoogle ScholarCross RefCross Ref
  27. T.-B. Nguyen, T. Nguyen, W. Luo, S. Venkatesh, and D. Phung. 2014. Unsupervised inference of significant locations from wifi data for understanding human dynamics. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia. 232--235.Google ScholarGoogle Scholar
  28. A. Alvarez-Alvarez, J. M. Alonso, and G. Trivino. 2013. Human activity recognition in indoor environments by means of fusing information extracted from intensity of WiFi signal and accelerations. Inf. Sci. (N. Y.). 233 (2013), 162--182. https://doi.org/10.1016/j.ins.2013.01.029.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. H. Hong, C. Luo, and M. C. Chan. 2016. Socialprobe: Understanding social interaction through passive wifi monitoring. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 94--103.Google ScholarGoogle Scholar
  30. Z. Chen, Y. Chen, S. Wang, J. Liu, X. Gao, and A. T. Campbell. 2013. Inferring social contextual behavior from bluetooth traces. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication 2013, 267--270.Google ScholarGoogle Scholar
  31. T.-F. Wu, C.-J. Lin, and R. C. Weng. 2004. Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. 5 (August 2004), 975--1005. https://www.jmlr.org/papers/v5/wu04a.html?907d3908.Google ScholarGoogle Scholar
  32. J. Wu, L. Sun, and R. Jafari. 2016. A wearable system for recognizing american sign language in real-time using IMU and surface EMG sensors. IEEE J. Biomed. Heal. Inf. 20, 5 (2016), 1281--1290.Google ScholarGoogle ScholarCross RefCross Ref
  33. J. Wu and R. Jafari. 2018. Orientation independent activity/gesture recognition using wearable motion sensors. IEEE IoT J. 6, 2 (2018), 1427--1437.Google ScholarGoogle Scholar

Index Terms

  1. Data-driven Context Detection Leveraging Passively Sensed Nearables for Recognizing Complex Activities of Daily Living

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!