skip to main content
research-article

Deriving Effective Human Activity Recognition Systems through Objective Task Complexity Assessment

Published: 18 December 2020 Publication History

Abstract

Research in sensor based human activity recognition (HAR) has been a core concern of the mobile and ubiquitous computing community. Sophisticated systems have been developed with the main view on applications of HAR methods in research settings. This work addresses a related yet practically different problem that mainly focuses on users of HAR technology. We acknowledge that practitioners from outside the core HAR research community are motivated to employ HAR methods for practical deployments. Even though standard processing approaches exist, arguably, often times substantial modifications are necessary to derive effective analysis systems. It is not always clear a-priori how challenging a HAR task actually is and what dimensions of an analysis pipeline are crucial for successful automated assessments. In practice this can lead to disappointing results or disproportionate efforts that have to be invested into the optimization of data analysis pipelines, that were supposed to work "out of the box". We present a framework for the objective complexity assessment of HAR tasks that directly supports practitioners' decision making of whether and how to employ HAR for their deployments. We map a HAR task onto a vectorial representation that allows us to analyse the inherent challenges of the task and to draw conclusions through similarity analysis with regards to existing tasks. We validate our complexity assessment framework on 23 HAR datasets and derive a data-driven categorization of human activity recognition. We demonstrate how our objective analysis can be used to inform the deployment of HAR systems in practical scenarios.

References

[1]
Talal Alshammari, Nasser Alshammari, Mohamed Sedky, and Chris Howard. Simadl: simulated activities of daily living dataset. Data, 3(2):11, 2018.
[2]
Ong Chin Ann and Lau Bee Theng. Human activity recognition: a review. In 2014 IEEE international conference on control system, computing and engineering (ICCSCE 2014), pages 389--393. IEEE, 2014.
[3]
Marc Bachlin, Meir Plotnik, Daniel Roggen, Inbal Maidan, Jeffrey M Hausdorff, Nir Giladi, and Gerhard Troster. Wearable assistant for parkinson's disease patients with the freezing of gait symptom. IEEE Transactions on Information Technology in Biomedicine, 14 (2):436--446, 2009.
[4]
Oresti Banos, Jose Antonio Moral-Munoz, Ignacio Diaz-Reyes, Manuel Arroyo-Morales, Miguel Damas, Enrique Herrera-Viedma, Choong Seon Hong, Sungyong Lee, Hector Pomares, Ignacio Rojas, et al. mdurance: a novel mobile health system to support trunk endurance assessment. Sensors, 15(6):13159--13183, 2015.
[5]
Verónica Bolón-Canedo, Laura Moran-Fernandez, and Amparo Alonso-Betanzos. An insight on complexity measures and classification in microarray data. In 2015 International Joint Conference on Neural Networks (IJCNN), pages 1--8. IEEE, 2015.
[6]
Andreas Bulling, U Blanke, and Bernt Schiele. A Tutorial on Human Activity Recognition Using Body-worn Inertial Sensors. ACM Computing Surveys, 46(3), January 2014.
[7]
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[8]
James M. Fisher, Nils Y. Hammerla, Thomas Ploetz, Peter Andras, Lynn Rochester, and Richard W. Walker. Unsupervised home monitoring of parkinson's disease motor symptoms using body-worn accelerometers. Parkinsonism Related Disorders, 33:44 - 50, 2016.
[9]
James M. Fisher, Nils Y. Hammerla, Lynn Rochester, Peter Andras, and Richard W. Walker. Body-worn sensors in parkinson's disease: Evaluating their acceptability to patients. Telemedicine and e-Health, 22(1):63--69, 2016. 26186307.
[10]
Yan Gao, Yang Long, Yu Guan, Anna Basu, Jessica Baggaley, and Thomas Ploetz. Towards reliable, automated general movement assessment for perinatal stroke screening in infants using wearable accelerometers. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 3(1), March 2019.
[11]
Alan Godfrey, AK Bourke, GM Olaighin, P Van De Ven, and J Nelson. Activity classification using a single chest mounted tri-axial accelerometer. Medical engineering & physics, 33(9):1127--1135, 2011.
[12]
Yu Guan and Thomas Plötz. Ensembles of deep lstm learners for activity recognition using wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2):1--28, 2017.
[13]
Nils Y Hammerla, Shane Halloran, and Thomas Plötz. Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 1533-1540, 2016.
[14]
Nils Y. Hammerla, Reuben Kirkham, Peter Andras, and Thomas Ploetz. On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution. In Proceedings of the 2013 International Symposium on Wearable Computers, ISWC '13, page 65--68, New York, NY, USA, 2013. Association for Computing Machinery.
[15]
Nils Yannick Hammerla, James Fisher, Peter Andras, Lynn Rochester, Richard Walker, and Thomas Plötz. Pd disease state assessment in naturalistic environments using deep learning. In Twenty-Ninth AAAI conference on artificial intelligence, 2015.
[16]
Harish Haresamudram, David V Anderson, and Thomas Plötz. On the role of features in human activity recognition. In Proceedings of the 23rd International Symposium on Wearable Computers, pages 78-88, 2019.
[17]
T. K. Ho and M. Basu. Complexity measures of supervised classification problems. IEEE Trans. Pattern Anal. Mach. Intell., 24 (3):289--300, March 2002.
[18]
Tin Kam Ho and Mitra Basu. Measuring the complexity of classification problems. In Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, volume 2, pages 43--47. IEEE, 2000.
[19]
Jesse Hoey, Thomas Plötz, Dan Jackson, Andrew Monk, Cuong Pham, and Patrick Olivier. Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive and Mobile Computing, 7(3):299--318, 2011.
[20]
Hyeokhyen Kwon, Gregory D Abowd, and Thomas Plötz. Adding structural characteristics to distribution-based accelerometer representations for activity recognition using wearables. In Proceedings of the 2018 ACM international symposium on wearable computers, pages 72-75, 2018.
[21]
Hyeokhyen Kwon, Gregory D Abowd, and Thomas Plötz. Handling annotation uncertainty in human activity recognition. In Proceedings of the 23rd International Symposium on Wearable Computers, pages 109--117, 2019.
[22]
Hong Li, Gregory D Abowd, and Thomas Plötz. On specialized window lengths and detector based human activity recognition. In Proceedings of the 2018 ACM international symposium on wearable computers, pages 68--71, 2018.
[23]
V-M Mantyla, J Mantyjarvi, T Seppanen, and Esa Tuulari. Hand gesture recognition of a mobile device user. In 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No. 00TH8532), volume 1, pages 281--284. IEEE, 2000.
[24]
Laura Morán-Fernández, Verónica Bolón-Canedo, and Amparo Alonso-Betanzos. Can classification performance be predicted by complexity measures? a study using microarray data. Knowledge and Information Systems, 51(3):1067--1090, 2017.
[25]
Le Nguyen Ngu Nguyen, Daniel Rodríguez-Martín, Andreu Català, Carlos Pérez-López, Albert Samà, and Andrea Cavallaro. Basketball activity recognition using wearable inertial measurement units. In Proceedings of the XVI international conference on Human Computer Interaction, pages 1--6, 2015.
[26]
JT O'Brien, P Gallagher, D Stow, N Hammerla, T Ploetz, M Firbank, C Ladha, K Ladha, D Jackson, Roisin McNaney, et al. A study of wrist-worn activity measurement as a potential real-world biomarker for late-life depression. Psychological medicine, 47(1):93--102, 2017.
[27]
Francisco Javier Ordóñez and Daniel Roggen. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1):115, 2016.
[28]
Thomas Plötz and Yu Guan. Deep learning for human activity recognition in mobile computing. Computer, 51(5):50--59, 2018.
[29]
Thomas Plötz, Nils Y. Hammerla, Agata Rozga, Andrea Reavis, Nathan Call, and Gregory D. Abowd. Automatic assessment of problem behavior in individuals with developmental disabilities. page 391--400, 2012.
[30]
K. Polat. Freezing of gait (fog) detection using logistic regression in parkinson's disease from acceleration signals. In 2019 Scientific Meeting on Electrical-Electronics Biomedical Engineering and Computer Science (EBBT), pages 1--4, 2019.
[31]
Attila Reiss and Didier Stricker. Introducing a new benchmarked dataset for activity monitoring. In 2012 16th International Symposium on Wearable Computers, pages 108--109. IEEE, 2012.
[32]
Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, et al. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh international conference on networked sensing systems (INSS), pages 233--240. IEEE, 2010.
[33]
T Scott Saponas, Brian R Meyers, and AJ Bernheim Brush. Human-activity recognition (har) everywhere: Embedded, mobile, desktop, home & cloud. In Pervasive 2011 Workshop: Frontiers in Activity Recognition using Pervasive Sensing (IWFAR), Microsoft Research, 2011.
[34]
Muhammad Harris Shoaib. Human activity recognition using heterogeneous sensors. In UbiComp 2013, 2013.
[35]
T. Stiefmeier, D. Roggen, G. Ogris, P. Lukowicz, and G. Tröster. Wearable activity tracking in car manufacturing. IEEE Pervasive Computing, 7(2):42--50, 2008.
[36]
Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, SenSys '15, page 127--140, New York, NY, USA, 2015. Association for Computing Machinery.
[37]
Roberto L Shinmoto Torres, Damith C Ranasinghe, Qinfeng Shi, and Alanson P Sample. Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. In 2013 IEEE International Conference on RFID (RFID), pages 191--198. IEEE, 2013.
[38]
Vladimir Vapnik, Esther Levin, and Yann Le Cun. Measuring the vc-dimension of a learning machine. Neural computation, 6(5):851--876, 1994.
[39]
Kun Wang, Jun He, and Lei Zhang. Sequential weakly labeled multi-activity recognition and location on wearable sensors using recurrent attention network, 2020.
[40]
Anjana Wijekoon, Nirmalie Wiratunga, and Kay Cooper. Mex: Multi-modal exercises dataset for human activity recognition. arXiv preprint [Web Link], 2019.
[41]
Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37--52, 1987.
[42]
Xuewen Yao, Thomas Plötz, Mckensey Johnson, and K de Barbaro. Automated detection of infant holding using wearable sensing: Implications for developmental science and intervention. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 3(2):1--17, 2019.
[43]
Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors. IEEE, November 2014.
[44]
Mi Zhang and Alexander A. Sawchuk. Usc-had: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In ACM International Conference on Ubiquitous Computing (Ubicomp) Workshop on Situation, Activity and Goal Awareness (SAGAware), Pittsburgh, Pennsylvania, USA, September 2012.

Cited By

View all
  • (2024)Maintenance Required: Updating and Extending Bootstrapped Human Activity Recognition Systems for Smart Homes2024 International Conference on Activity and Behavior Computing (ABC)10.1109/ABC61795.2024.10651685(1-13)Online publication date: 29-May-2024
  • (2023)The Lifespan of Human Activity Recognition Systems for Smart HomesSensors10.3390/s2318772923:18(7729)Online publication date: 7-Sep-2023
  • (2023)If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops56833.2023.10150267(565-570)Online publication date: 13-Mar-2023
  • Show More Cited By

Index Terms

  1. Deriving Effective Human Activity Recognition Systems through Objective Task Complexity Assessment

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 4, Issue 4
      December 2020
      1356 pages
      EISSN:2474-9567
      DOI:10.1145/3444864
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 18 December 2020
      Published in IMWUT Volume 4, Issue 4

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. data complexity
      2. human activity recognition
      3. pattern recognition

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)75
      • Downloads (Last 6 weeks)3
      Reflects downloads up to 28 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Maintenance Required: Updating and Extending Bootstrapped Human Activity Recognition Systems for Smart Homes2024 International Conference on Activity and Behavior Computing (ABC)10.1109/ABC61795.2024.10651685(1-13)Online publication date: 29-May-2024
      • (2023)The Lifespan of Human Activity Recognition Systems for Smart HomesSensors10.3390/s2318772923:18(7729)Online publication date: 7-Sep-2023
      • (2023)If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops56833.2023.10150267(565-570)Online publication date: 13-Mar-2023
      • (2023)Enhancing human activity recognition using features reduction in IoT edge and Azure cloudDecision Analytics Journal10.1016/j.dajour.2023.1002828(100282)Online publication date: Oct-2023
      • (2022)Predicting Performance Improvement of Human Activity Recognition Model by Additional Data CollectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503196:3(1-33)Online publication date: 7-Sep-2022
      • (2022)Assessing the State of Self-Supervised Human Activity Recognition Using WearablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35502996:3(1-47)Online publication date: 7-Sep-2022
      • (2022)Bootstrapping Human Activity Recognition Systems for Smart Homes from ScratchProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35502946:3(1-27)Online publication date: 7-Sep-2022
      • (2022)Stepping Into the Next Decade of Ubiquitous and Pervasive Computing: UbiComp and ISWC 2021IEEE Pervasive Computing10.1109/MPRV.2022.316006321:2(87-99)Online publication date: 1-Apr-2022
      • (2022)Anwendung von Human Activity Recognition im Unternehmenskontext – Ein Konzept für die Zukunft?Smart Services10.1007/978-3-658-37344-3_15(459-478)Online publication date: 2-Jul-2022
      • (2021)Video and Image Complexity in Human Action RecognitionProgress in Artificial Intelligence and Pattern Recognition10.1007/978-3-030-89691-1_34(349-359)Online publication date: 4-Nov-2021

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media