Abstract
During the past decade, human activity recognition (HAR) using wearable sensors has become a new research hot spot due to its extensive use in various application domains such as healthcare, fitness, smart homes, and eldercare. Deep neural networks, especially convolutional neural networks (CNNs), have gained a lot of attention in HAR scenario. Despite exceptional performance, CNNs with heavy overhead is not the best option for HAR task due to the limitation of computing resource on embedded devices. As far as we know, there are many invalid filters in CNN that contribute very little to output. Simply pruning these invalid filters could effectively accelerate CNNs, but it inevitably hurts performance. In this article, we first propose a novel CNN for HAR that uses filter activation. In comparison with filter pruning that is motivated for efficient consideration, filter activation aims to activate these invalid filters from an accuracy boosting perspective. We perform extensive experiments on several public HAR datasets, namely, UCI-HAR (UCI), OPPORTUNITY (OPPO), UniMiB-SHAR (Uni), PAMAP2 (PAM2), WISDM (WIS), and USC-HAD (USC), which show the superiority of the proposed method against existing state-of-the-art (SOTA) approaches. Ablation studies are conducted to analyze its internal mechanism. Finally, the inference speed and power consumption are evaluated on an embedded Raspberry Pi Model 3 B plus platform.
- [1] . 2019. An ultra-low energy human activity recognition accelerator for wearable health applications. ACM Trans. Embed. Comput. Syst. 18, 5s (2019), 1–22.Google Scholar
Digital Library
- [2] . 2021. Two-stream convolution augmented transformer for human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence. 286–293.Google Scholar
Cross Ref
- [3] . 2021. XGest: Enabling cross-label gesture recognition with RF signals. ACM Trans. Sensor Netw. 17, 4 (2021), 1–23.Google Scholar
Digital Library
- [4] . 2021. Facilitating human activity data annotation via context-aware change detection on smartwatches. ACM Trans. Embed. Comput. Syst. 20, 2 (2021), 1–20.Google Scholar
Digital Library
- [5] . 2019. A fog-based application for human activity recognition using personal smart devices. ACM Trans. Internet Technol. 19, 2 (2019), 1–20.Google Scholar
Digital Library
- [6] . 2022. Channel-Equalization-HAR: A light-weight convolutional neural network for wearable sensor based human activity recognition. IEEE Trans. Mob. Comput. Early access.
DOI: Google ScholarCross Ref
- [7] . 2022. Real-time human activity recognition using conditionally parametrized convolutions on mobile and wearable devices. IEEE Sensors J. 22, 6 (2022), 5889–5901.Google Scholar
- [8] . 2021. Semisupervised human activity recognition with radar micro-Doppler signatures. IEEE Trans. Geosci. Rem. Sens. 60 (2021), 1–12.Google Scholar
- [9] . 2021. Shallow convolutional neural networks for human activity recognition using wearable sensors. IEEE Trans. Instrum. Measur. 70 (2021), 1–11.Google Scholar
- [10] . 2015. Deep learning. Nature 521, 7553 (2015), 436–444.Google Scholar
Digital Library
- [11] . 2021. A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Trans. Neural Netw. Learn. Syst. Early access.
DOI: Google ScholarCross Ref
- [12] . 2018. Recent advances in convolutional neural networks. Pattern Recog. 77 (2018), 354–377.Google Scholar
Digital Library
- [13] . 2018. RF-based fall monitoring using convolutional neural networks. Proc. ACM Interact., Mob., Wear. Ubiq. Technol. 2, 3 (2018), 1–24.Google Scholar
Digital Library
- [14] . 2018. Understanding and improving recurrent networks for human activity recognition by continuous attention. In Proceedings of the ACM International Symposium on Wearable Computers. 56–63.Google Scholar
Digital Library
- [15] . 2016. Human activity recognition with smartphone sensors using deep learning neural networks. Exp. Syst. Applic. 59 (2016), 235–244.Google Scholar
Digital Library
- [16] . 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of the 24th International Joint Conference on Artificial Intelligence.Google Scholar
Digital Library
- [17] . 2016. Deep learning for human activity recognition: A resource efficient implementation on low-power devices. In Proceedings of the IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, 71–76.Google Scholar
Cross Ref
- [18] . 2015. Human activity recognition using wearable sensors by deep convolutional neural networks. In Proceedings of the 23rd ACM International Conference on Multimedia. 1307–1310.Google Scholar
Digital Library
- [19] . 2020. Interpretable and accurate convolutional neural networks for human activity recognition. IEEE Trans. Industr. Inform. 16, 11 (2020), 7190–7198.Google Scholar
Cross Ref
- [20] . 2018. Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl. Soft Comput. 62 (2018), 915–922.Google Scholar
Cross Ref
- [21] . 2019. Adaptive fusion and category-level dictionary learning model for multiview human action recognition. IEEE Internet Things J. 6, 6 (2019), 9280–9293.Google Scholar
Cross Ref
- [22] . 2016. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.Google Scholar
Cross Ref
- [23] . 2016. Pruning filters for efficient convNets. arXiv preprint arXiv:1608.08710 (2016).Google Scholar
- [24] . 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. arXiv preprint arXiv:1510.00149 (2015).Google Scholar
- [25] . 2018. Exploring linear relationship in feature map subspace for convNets compression. arXiv preprint arXiv:1803.05729 (2018).Google Scholar
- [26] . 2018. Optimizing multi-sensor deployment via ensemble pruning for wearable activity recognition. Inf. Fusion 41 (2018), 68–79.Google Scholar
Digital Library
- [27] . 2021. Pruning by explaining: A novel criterion for deep neural network pruning. Pattern Recog. 115 (2021), 107899.Google Scholar
Cross Ref
- [28] . 2018. A novel ensemble ELM for human activity recognition using smartphone sensors. IEEE Trans. Industr. Inform. 15, 5 (2018), 2691–2699.Google Scholar
Cross Ref
- [29] . 2018. SCSP: Spectral clustering filter pruning with soft self-adaption manners. arXiv preprint arXiv:1806.05320 (2018).Google Scholar
- [30] . 2020. Filter grafting for deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6599–6607.Google Scholar
Cross Ref
- [31] . 2018. On attention models for human activity recognition. In Proceedings of the ACM International Symposium on Wearable Computers. 100–103.Google Scholar
Digital Library
- [32] . 2019. Attention-based convolutional neural network for weakly labeled human activities’ recognition with wearable sensors. IEEE Sensors J. 19, 17 (2019), 7598–7604.Google Scholar
Cross Ref
- [33] . 2019. AttnSense: Multi-level attention mechanism for multimodal human activity recognition. In Proceedings of the International Joint Conferences on Artificial Intelligence. 3109–3115.Google Scholar
Cross Ref
- [34] . 2018. Deep mutual learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4320–4328.Google Scholar
Cross Ref
- [35] . 2021. A systematic review of smartphone-based human activity recognition methods for health research. Nat. Partn. J.: Digit. Med. 4, 1 (2021), 1–15.Google Scholar
- [36] . 2017. Ensembles of deep LSTM learners for activity recognition using wearables. Proc. ACM Interact., Mob., Wear. Ubiq 1, 2 (2017), 1–28.Google Scholar
Digital Library
- [37] . 2012. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In Proceedings of the International Workshop on Ambient Assisted Living. Springer, 216–223.Google Scholar
Digital Library
- [38] . 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 7th International Conference on Networked Sensing Systems (INSS). IEEE, 233–240.Google Scholar
Cross Ref
- [39] . 2017. UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7, 10 (2017), 1101.Google Scholar
Cross Ref
- [40] . 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the 16th International Symposium on Wearable Computers. IEEE, 108–109.Google Scholar
Digital Library
- [41] . 2011. Design considerations for the WISDM smart phone-based sensor mining architecture. In Proceedings of the 5th International Workshop on Knowledge Discovery from Sensor Data. 25–33.Google Scholar
Digital Library
- [42] . 2012. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the ACM Conference on Ubiquitous Computing. 1036–1043.Google Scholar
Digital Library
- [43] . 2021. Attention induced multi-head convolutional neural network for human activity recognition. Appl. Soft Comput. 110 (2021), 107671.Google Scholar
Digital Library
- [44] . 2020. The layer-wise training convolutional neural networks using local loss for sensor-based human activity recognition. IEEE Sensors J. 20, 13 (2020), 7265–7274.Google Scholar
Cross Ref
- [45] . 2021. Attend and discriminate: Beyond the state-of-the-art for human activity recognition using wearable sensors. Proc. ACM Interact., Mob., Wear. Ubiq 5, 1 (2021), 1–22.Google Scholar
Digital Library
- [46] . 2021. Sequential weakly labeled multiactivity localization and recognition on wearable sensors using recurrent attention networks. IEEE Trans. Hum.-Mach. Syst. 51, 4 (2021), 355–364.Google Scholar
Cross Ref
- [47] . 2021. Latent independent excitation for generalizable sensor-based cross-person activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI, 11921–11929.Google Scholar
Cross Ref
- [48] . 2022. Multi-ResAtt: Multilevel residual network with attention for human activity recognition using wearable sensors. IEEE Trans. Industr. Inform. Early access.
DOI: Google ScholarCross Ref
- [49] . 2021. Learning disentangled representation for mixed-reality human activity recognition with a single IMU sensor. IEEE Trans. Instrum. Measur. 70 (2021), 1–14.Google Scholar
- [50] . 2021. NOHAR-NOvelty discrete data stream for human activity recognition based on smartphones with inertial sensors. Exp. Syst. Applic. 166 (2021), 114093.Google Scholar
Cross Ref
- [51] . 2020. A comparison of unidirectional and bidirectional LSTM networks for human activity recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 1–6.Google Scholar
Cross Ref
- [52] . 2019. Gait activity authentication using LSTM neural networks with smartphone sensors. In Proceedings of the 15th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN). IEEE, 456–461.Google Scholar
Cross Ref
- [53] . 2022. CapsGaNet: Deep neural network based on capsule and GRU for human activity recognition. IEEE Syst. J. Early access.
DOI: Google ScholarCross Ref
- [54] . 2021. Meta-HAR: Federated representation learning for human activity recognition. In Proceedings of the International World Wide Web Conferences (WWW). ACM, 912–922.Google Scholar
Digital Library
- [55] . 2020. Human activity recognition based on dynamic active learning. IEEE J. Biomed. Health Inform. 25, 4 (2020), 922–934.Google Scholar
Cross Ref
- [56] . 2020. Deep ConvLSTM with self-attention for human activity decoding using wearable sensors. IEEE Sensors J. 21, 6 (2020), 8575–8582.Google Scholar
Cross Ref
- [57] . 2021. Time series change point detection with self-supervised contrastive predictive coding. In Proceedings of the Web Conference. ACM, 3124–3135.Google Scholar
Digital Library
Index Terms
Deep Ensemble Learning for Human Activity Recognition Using Wearable Sensors via Filter Activation
Recommendations
Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
MM '15: Proceedings of the 23rd ACM international conference on MultimediaHuman physical activity recognition based on wearable sensors has applications relevant to our daily life such as healthcare. How to achieve high recognition accuracy with low computational cost is an important issue in the ubiquitous computing. Rather ...
Human activity recognition with smartphone sensors using deep learning neural networks
This paper proposes a deep convolutional neural network for HAR using smartphone sensors.Experiments show that the proposed method derives relevant and more complex features.The method achieved an almost perfect classification on moving activities.It ...
Human activity recognition by combining external features with accelerometer sensor data using deep learning network model
AbstractVarious Human Activities are classified through time-series data generated by the sensors of wearable devices. Many real-time scenarios such as Healthcare Surveillance, Smart Cities and Intelligent surveillance etc. are based upon Human Activity ...






Comments