Abstract
Photoplethysmography (PPG) sensors allow for non-invasive and comfortable heart rate (HR) monitoring, suitable for compact wrist-worn devices. Unfortunately, motion artifacts (MAs) severely impact the monitoring accuracy, causing high variability in the skin-to-sensor interface. Several data fusion techniques have been introduced to cope with this problem, based on combining PPG signals with inertial sensor data. Until now, both commercial and reasearch solutions are computationally efficient but not very robust, or strongly dependent on hand-tuned parameters, which leads to poor generalization performance. In this work, we tackle these limitations by proposing a computationally lightweight yet robust deep learning-based approach for PPG-based HR estimation. Specifically, we derive a diverse set of Temporal Convolutional Networks for HR estimation, leveraging Neural Architecture Search. Moreover, we also introduce ActPPG, an adaptive algorithm that selects among multiple HR estimators depending on the amount of MAs, to improve energy efficiency. We validate our approaches on two benchmark datasets, achieving as low as 3.84 beats per minute of Mean Absolute Error on PPG-Dalia, which outperforms the previous state of the art. Moreover, we deploy our models on a low-power commercial microcontroller (STM32L4), obtaining a rich set of Pareto optimal solutions in the complexity vs. accuracy space.
- [1] 2020. Neurosky. Retrieved from http://neurosky.com/wp-content/uploads/2016/06/TOF-side-by-side-competitor-comparison.pdf.Google Scholar
- [2] . 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. Retrieved from https://www.tensorflow.org/.Google Scholar
- [3] . 2018. A review of wrist-worn wearable: Sensors, models, and challenges. J. Sens. 2018 (2018).Google Scholar
- [4] . [n.d.]. Apple Watch Series. Retrieved from https://www.apple.com/lae/watch/.Google Scholar
- [5] . 2020. Robust de-noising technique for accurate heart rate estimation using wrist-type PPG signals. IEEE Sens. J. 20, 14 (2020), 7980–7987.Google Scholar
- [6] . 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271. Retrieved from https://arxiv.org/abs/1803.01271.Google Scholar
- [7] . [n.d.]. Respiban Professional 2019. Retrieved from https://biosignalsplux.com/products/wearables/respiban-pro.html.Google Scholar
- [8] . 2019. CorNET: Deep learning framework for PPG-based heart rate estimation and biometric identification in ambulant environment. IEEE Trans. Biomed. Circ. Syst. 13, 2 (2019), 282–291.Google Scholar
- [9] . 2019. Heart rate estimation from wrist-worn photoplethysmography: A review. IEEE Sens. J. 19, 16 (2019), 6560–6570. Google Scholar
Cross Ref
- [10] . 2020. An ensemble of hyperdimensional classifiers: Hardware-friendly short-latency seizure detection with automatic iEEG electrode selection. IEEE J. Biomed. Health Inf. 25, 4 (2020), 935–946.Google Scholar
- [11] . 2020. Hyperdimensional computing with local binary patterns: One-shot learning for seizure onset detection and identification of ictogenic brain regions from short-time iEEG recordings. IEEE Trans. Biomed. Eng. 67, 2 (2020), 601–613.Google Scholar
Cross Ref
- [12] . 2018. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv:1812.00332. Retrieved from https://arxiv.org/abs/1812.00332.Google Scholar
- [13] . 2018. A review on wearable photoplethysmography sensors and their potential future applications in health care. Int. J. Biosens. Bioelectr. 4, 4 (2018), 195–202.Google Scholar
- [14] . 2021. DeepHeart: A deep learning approach for accurate heart rate estimation from PPG signals. ACM Trans. Sen. Netw. 17, 2, Article
14 (Jan. 2021), 18 pages. Google ScholarDigital Library
- [15] . 2020. Deep learning for heart rate estimation from reflectance photoplethysmography with acceleration power spectrum and acceleration intensity. IEEE Access 8 (2020), 63390–63402.Google Scholar
- [16] . 2018. Finite state machine framework for instantaneous heart rate validation using wearable photoplethysmography during intensive exercise. IEEE J. Biomed. Health Inform. 23, 4 (2018), 1595–1606.Google Scholar
- [17] . 2020. Technical Report: NEMO DNN Quantization for Deployment Model.
arxiv:2004.05930 [cs.LG]. Retrieved from https://arxiv.org/abs/2004.05930.Google Scholar - [18] . 2020. Energy-efficient adaptive machine learning on IoT end-nodes with class-dependent confidence. In Proceedings of the International Conference on Electronics Circuits and Systems (ICECS’20). IEEE, 1–4.Google Scholar
- [19] . [n.d.]. E4 Wristband 2014. Retrieved from https://www.empatica.com/en-eu/research/e4/.Google Scholar
- [20] . [n.d.]. Fitbit Charge 4. Retrieved from https://www.fitbit.com/global/us/products/trackers/charge4.Google Scholar
- [21] . 2016. Evaluating the accuracy of wearable heart rate monitors. In Proceedings of the 2nd International Conference on Advances in Computing, Communication, & Automation (ICACCA’16). IEEE, 1–6.Google Scholar
- [22] . 2018. Morphnet: Fast & simple resource-constrained structure learning of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1586–1595.Google Scholar
Cross Ref
- [23] . 2020. Robust PPG-based ambulatory heart rate tracking algorithm. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 5929–5934.Google Scholar
- [24] . 2020. EEG-TCNet: An accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC’20). 2958–2965. Google Scholar
Digital Library
- [25] . 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167. Retrieved from https://arxiv.org/abs/1502.03167.Google Scholar
- [26] . 2018. Dynamic bit-width reconfiguration for energy-efficient deep learning hardware. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED’18). ACM, New York, NY. Google Scholar
Digital Library
- [27] . 2019. Evaluation of coherence between ECG and PPG derived parameters on heart rate variability and respiration in healthy volunteers with/without controlled breathing. J. Med. Biol. Eng. 39, 5 (2019), 783–795.Google Scholar
- [28] . 2016. Temporal convolutional networks: A unified approach to action segmentation. In Proceedings of the European Conference on Computer Vision. Springer, 47–54.Google Scholar
Cross Ref
- [29] . 2020. Mcunet: Tiny deep learning on iot devices. In Advances in Neural Information Processing Systems, Vol. 33.Google Scholar
- [30] . 2015. Heart rate tracking using wrist-type photoplethysmographic (PPG) signals during physical exercise with simultaneous accelerometry. IEEE Sign. Process. Lett. 23, 2 (2015), 227–231.Google Scholar
- [31] . [n.d.]. STM32L476. Retrieved from https://www.st.com/resource/en/datasheet/stm32l476je.pdf.Google Scholar
- [32] . 2020. Fast and accurate inference on microcontrollers with boosted cooperative convolutional neural networks (BC-Net). IEEE Trans. Circ. Syst. I: Regul. Pap. (2020), 1–12. Google Scholar
Cross Ref
- [33] . 2020. Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research. NPJ Dig. Med. 3, 1 (2020), 1–9.Google Scholar
- [34] . 1998. How to train neural networks. In Neural Networks: Tricks of the Trade. Springer, 373–423.Google Scholar
- [35] . 2015. Big/little deep neural network for ultra low power inference. In Proceedings of the International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS’15). IEEE, 124–132.Google Scholar
Digital Library
- [36] . 2019. Deep PPG: Large-scale heart rate estimation with convolutional neural networks. Sensors 19, 14 (2019), 3079.Google Scholar
Cross Ref
- [37] . 2021. Cloud-edge based lightweight temporal convolutional networks for remaining useful life prediction in IIoT. IEEE IoT J. 8, 16 (2021), 12578–12587.Google Scholar
- [38] . 2021. Robust and energy-efficient PPG-based heart-rate monitoring. In Proceedings of the International Symposium on Circuit and Systems (ISCAS’21). IEEE.Google Scholar
- [39] . 2020. Binary corNET: Accelerator for HR estimation from wrist-PPG. IEEE Trans. Biomed. Circ. Syst. 14, 4 (2020), 715–726.Google Scholar
Cross Ref
- [40] . 2016. A novel time-varying spectral filtering algorithm for reconstruction of motion artifact corrupted heart rate signals during intense physical activities using a wearable photoplethysmogram sensor. Sensors 16, 1 (2016), 10.Google Scholar
- [41] . 2017. Computationally efficient heart rate estimation during physical exercise using photoplethysmographic signals. In Proceedings of the 25th European Signal Processing Conference (EUSIPCO’17). IEEE, 2478–2481.Google Scholar
- [42] . 2009. Adaptive threshold method for the peak detection of photoplethysmographic waveform. Comput. Biol. Med. 39, 12 (2009), 1145–1152.Google Scholar
Digital Library
- [43] . 2019. PPGnet: Deep network for device independent heart rate estimation from photoplethysmogram. In Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 1899–1902.Google Scholar
- [44] . 2015. Human photoplethysmogram: New insight into chaotic characteristics. Chaos Solitons Fract. 77 (2015), 53–63.Google Scholar
- [45] . 2014. Wearable photoplethysmographic sensors-past and present. Electronics 3, 2 (2014), 282–302.Google Scholar
Cross Ref
- [46] . 2019. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2820–2828.Google Scholar
Cross Ref
- [47] . 2017. Accurate heart rate monitoring during physical exercises using PPG. IEEE Trans. Biomed. Eng. 64, 9 (2017), 2016–2024.Google Scholar
Cross Ref
- [48] . 2016. Use of internet of things (IoT) in healthcare: A survey. In Proceedings of the ACM Symposium on Women in Research 2016. 71–76.Google Scholar
Digital Library
- [49] . 2006. Model selection and estimation in regression with grouped variables. J. Roy. Stat. Soc.: Ser. B 68, 1 (2006), 49–67.Google Scholar
Cross Ref
- [50] . 2020. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Trans. Biomed. Circ. Syst. 14, 2 (2020), 244–256.Google Scholar
- [51] . 2019. A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv:1905.04149. Retrieved from https://arxiv.org/abs/1905.04149.Google Scholar
- [52] . 2015. Combining ensemble empirical mode decomposition with spectrum subtraction technique for heart rate monitoring using wrist-type photoplethysmography. Biomed. Sign. Process. Contr. 21 (2015), 119–125.Google Scholar
Cross Ref
- [53] . 2015. Photoplethysmography-based heart rate monitoring in physical activities via joint sparse spectrum reconstruction. IEEE Trans. Biomed. Eng. 62, 8 (2015), 1902–1910.Google Scholar
- [54] . 2014. TROIKA: A general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Trans. Biomed. Eng. 62, 2 (2014), 522–531.Google Scholar
Cross Ref
- [55] . 2020. Heart rate monitoring using sparse spectral curve tracing. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 5347–5352.Google Scholar
Index Terms
Embedding Temporal Convolutional Networks for Energy-efficient PPG-based Heart Rate Monitoring
Recommendations
Integrating mobile and cloud for PPG signal selection to monitor heart rate during intensive physical exercise
MOBILESoft '16: Proceedings of the International Conference on Mobile Software Engineering and SystemsHeart rate monitoring has become increasingly popular in the industry through mobile phones and wearable devices. However, current determination of heart rate through mobile applications suffers from high corruption of signals during intensive physical ...
Toward Hypertension Prediction Based on PPG-Derived HRV Signals: a Feasibility Study
Heart rate variability (HRV) is often used to assess the risk of cardiovascular disease, and data on this can be obtained via electrocardiography (ECG). However, collecting heart rate data via photoplethysmography (PPG) is now a lot easier. We ...
Heart Rate Monitoring Using Capacitive Touchscreen Sensing
BodySys'21: Proceedings of the Workshop on Body-Centric Computing SystemsAccording to World Health Organization (WHO) cardiovascular diseases (CVDs) are the number one cause of global deaths annually. More than 17 million people die each year from CVDs. To diagnose CVDs some clinical tests are required which are invasive and ...






Comments