Abstract
Annotating activities of daily living (ADL) is vital for developing machine learning models for activity recognition. In addition, it is critical for self-reporting purposes such as in assisted living where the users are asked to log their ADLs. However, data annotation becomes extremely challenging in real-world data collection scenarios, where the users have to provide annotations and labels on their own. Methods such as self-reports that rely on users’ memory and compliance are prone to human errors and become burdensome since they increase users’ cognitive load. In this article, we propose a light yet effective context-aware change point detection algorithm that is implemented and run on a smartwatch for facilitating data annotation for high-level ADLs. The proposed system detects the moments of transition from one to another activity and prompts the users to annotate their data. We leverage freely available Bluetooth low energy (BLE) information broadcasted by various devices to detect changes in environmental context. This contextual information is combined with a motion-based change point detection algorithm, which utilizes data from wearable motion sensors, to reduce the false positives and enhance the system's accuracy. Through real-world experiments, we show that the proposed system improves the quality and quantity of labels collected from users by reducing human errors while eliminating users’ cognitive load and facilitating the data annotation process.
- M. Sandberg, J. Kristensson, P. Midlöv, C. Fagerström, and U. Jakobsson. 2012. Prevalence and predictors of healthcare utilization among older people (60+): Focusing on ADL dependency and risk of depression. Archives of Gerontology and Geriatrics 54, 3 (2012), e349--e363.Google Scholar
Cross Ref
- D. Wang, J. Zheng, M. Kurosawa, Y. Inaba, and N. Kato. 2009. Changes in activities of daily living (ADL) among elderly Chinese by marital status, living arrangement, and availability of healthcare over a 3-year period. Environmental Health and Preventive Medicine 14, 2 (2009), 128.Google Scholar
Cross Ref
- P. Bharti, D. De, S. Chellappan, and S. K. Das. 2019. HuMAn: Complex activity recognition with multi-modal multi-positional body sensing. IEEE Transactions on Mobile Computing 18, 4 (2019), 857--870.Google Scholar
Digital Library
- L. G. Clift, J. Lepley, H. Hagras, and A. F. Clark. 2018. Autonomous computational intelligence-based behaviour recognition in security and surveillance. In Proceedings of Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies II (2018), Vol. 10802. 108020L.Google Scholar
Cross Ref
- K. Davis et al. 2016. Activity recognition based on inertial sensors for ambient assisted living. In Proceedings of the 2016 19th International Conference on Information Fusion (Fusion). 371--378.Google Scholar
- A. Akbari, J. Wu, R. Grimsley, and R. Jafari. 2018. Hierarchical signal segmentation and classification for accurate activity recognition. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 1596--1605.Google Scholar
- A. Akbari and R. Jafari. 2019. Transferring activity recognition models for new wearable sensors with deep generative domain adaptation. In Proceedings of the 18th International Conference on Information Processing in Sensor Networks. 85--96.Google Scholar
- F. Ordóñez and D. Roggen. 2016. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.Google Scholar
Cross Ref
- B. Hagsten, O. Svensson, and A. Gardulf. 2006. Health-related quality of life and self-reported ability concerning ADL and IADL after hip fracture: A randomized trial. Acta Orthopaedica 77, 1 (2006), 114--119.Google Scholar
Cross Ref
- S. Aminikhanghahi, T. Wang, and D. J. Cook. 2018. Real-time change point detection with application to smart home time series data. IEEE Transactions on Knowledge and Data Engineering 31 (2018), 1010--1023.Google Scholar
Cross Ref
- R. Solis, A. Pakbin, A. Akbari, B. J. Mortazavi, and R. Jafari. 2019. A human-centered wearable sensing platform with intelligent automated data annotation capabilities. In Proceedings of the International Conference on Internet of Things Design and Implementation. 255--260.Google Scholar
- A. Akbari and R. Jafari. 2020. Personalizing activity recognition models with quantifying different types of uncertainty using wearable sensors. In IEEE Transactions on Biomedical Engineering 67, 9 (2020), 2530--2541. DOI:10.1109/TBME.2019.2963816Google Scholar
Cross Ref
- S. Aminikhanghahi, M. Schmitter-Edgecombe, and D. J. Cook. 2019. Context-aware delivery of ecological momentary assessment. IEEE Journal of Biomedical and Health Informatics 24, 4 (2019), 1206--1214.Google Scholar
Cross Ref
- T. Patterson et al. 2016. Sensor-based change detection for timely solicitation of user engagement. IEEE Transactions on Mobile Computing 16, 10 (2016), 2889--2900.Google Scholar
Digital Library
- Y. Kawahara and M. Sugiyama. 2012. Sequential change-point detection based on direct density-ratio estimation. Statistical Analysis and Data Mining: The ASA Data Science Journal 5, 2 (2012), 114--127.Google Scholar
Digital Library
- T. Patterson, S. McClean, C. Nugent, S. Zhang, L. Galway, and I. Cleland. 2014. Online change detection for timely solicitation of user interaction. In Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence. 116--123.Google Scholar
- T. R. Bennett, H. C. Massey, J. Wu, S. A. Hasnain, and R. Jafari. 2016. MotionSynthesis toolset (MoST): An open source tool and data set for human motion data synthesis and validation. IEEE Sensors Journal 16, 13 (2016), 5365--5375.Google Scholar
Cross Ref
- D. Roggen et al. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 2010 7th International Conference on Networked Sensing Systems (INSS’2010). 233--240.Google Scholar
Cross Ref
- H. Gjoreski et al. 2018. The university of Sussex-Huawei locomotion and transportation dataset for multimodal analytics with mobile devices. IEEE Access 6 (2018), 42592--42604.Google Scholar
Digital Library
- T. Diethe, N. Twomey, and P. A. Flach. 2016. Active transfer learning for activity recognition. In Proceedings of the European Symposium on Artificial Neural Networks (ESANN’16).Google Scholar
- S. Sabato and T. Hess. 2016. Interactive algorithms: From pool to stream. In Proceedings of the Conference on Learning Theory (2016). 1419--1439.Google Scholar
- A. Akbari, R. Solis Castilla, R. Jafari, and B. J. Mortazavi. 2020. Using intelligent personal annotations to improve human activity recognition for movements in natural environments. IEEE Journal of Biomedical and Health Informatics (JBHI) 24, 9 (2020), 2639--2650. DOI:10.1109/JBHI.2020.2966151Google Scholar
Cross Ref
- T. Sztyler and H. Stuckenschmidt. 2017. Online personalization of cross-subjects based activity recognition models on wearable devices. In Proceedings of the 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom’17). 180--189.Google Scholar
- I. Cleland et al. 2014. Evaluation of prompted annotation of activity data recorded from a smart phone. Sensors 14, 9 (2014), 15861--15879.Google Scholar
Cross Ref
- S. Reddy, M. Mun, J. Burke, D. Estrin, M. Hansen, and M. Srivastava. 2010. Using mobile phones to determine transportation modes. ACM Transactions on Sensor Networks (TOSN) 6, 2 (2010).Google Scholar
Digital Library
- J. Muñoz-Marí, F. Bovolo, L. Gómez-Chova, L. Bruzzone, and G. Camp-Valls. 2010. Semisupervised one-class support vector machines for classification of remote sensing data. IEEE Transactions on Geoscience and Remote Sensing 48, 8 (2010), 3188--3197.Google Scholar
Cross Ref
- S. Aminikhanghahi and D. J. Cook. 2017. A survey of methods for time series change point detection. Knowledge and Information Systems 51, 2 (2017), 339--367.Google Scholar
Digital Library
- D. R. Jeske, V. M. De Oca, W. Bischoff, and M. Marvasti. 2009. Cusum techniques for timeslot sequences with applications to network surveillance. Computational Statistics 8 Data Analysis 53, 12 (2009), 4332--4344.Google Scholar
- X. Song, M. Wu, C. Jermaine, and S. Ranka. 2007. Statistical change detection for multi-dimensional data. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2007). 667--676.Google Scholar
Digital Library
- L. I. Kuncheva and W. J. Faithfull. 2013. PCA feature extraction for change detection in multidimensional unlabeled data. IEEE Transactions on Neural Networks and Learning Systems 25, 1 (2013), 69--80.Google Scholar
Cross Ref
- Y. Kawahara and M. Sugiyama. 2009. Change-point detection in time-series data by direct density-ratio estimation. In Proceedings of the 2009 SIAM International Conference on Data Mining (2009). 389--400.Google Scholar
- S. Liu, M. Yamada, N. Collier, and M. Sugiyama. 2013. Change-point detection in time-series data by relative density-ratio estimation. Neural Networks 43 (2013), 72--83.Google Scholar
Digital Library
- A. A. Qahtan, B. Alharbi, S. Wang, and X. Zhang. 2015. A PCA-based change detection framework for multidimensional data streams: Change detection in multidimensional data streams. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015). 935--944.Google Scholar
- J. Wu and R. Jafari. 2018. Orientation independent activity/gesture recognition using wearable motion sensors. IEEE Internet of Things Journal 6, 2 (2018), 1427--1437.Google Scholar
Cross Ref
- E. R. Sykes, S. Pentland, and S. Nardi. 2015. Context-aware mobile apps using iBeacons: Towards smarter interactions. In Proceedings of the 25th Annual International Conference on Computer Science and Software Engineering (2015). 120--129.Google Scholar
- M. Levandowsky and D. Winter. 1971. Distance between sets. Nature 234, 5323 (1971), 34--35.Google Scholar
- Polar M600 | Sport smart watch for fitness | Polar Global. Retrieved on March 2019 from https://www.polar.com/blog/polar-m600-android-wear-2-0-sports-smartwatch/.Google Scholar
Index Terms
Facilitating Human Activity Data Annotation via Context-Aware Change Detection on Smartwatches
Recommendations
Data-driven Context Detection Leveraging Passively Sensed Nearables for Recognizing Complex Activities of Daily Living
Wearable systems have unlocked new sensing paradigms in various applications such as human activity recognition, which can enhance effectiveness of mobile health applications. Current systems using wearables are not capable of understanding their ...
Annotating smart environment sensor data for activity learning
Smart Environments: Technology to Support HealthcareThe pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of ...
Wearable Sensing Framework for Human Activity Monitoring
WearSys '15: Proceedings of the 2015 workshop on Wearable Systems and ApplicationsWearable computation is getting integrated into our daily life day by day. In this work, we propose a generic framework to continuously monitor users' daily activities. The framework proposes light computation tasks on the wearable device to reduce the ...






Comments