skip to main content
research-article

Embedding Temporal Convolutional Networks for Energy-efficient PPG-based Heart Rate Monitoring

Authors Info & Claims
Published:03 March 2022Publication History
Skip Abstract Section

Abstract

Photoplethysmography (PPG) sensors allow for non-invasive and comfortable heart rate (HR) monitoring, suitable for compact wrist-worn devices. Unfortunately, motion artifacts (MAs) severely impact the monitoring accuracy, causing high variability in the skin-to-sensor interface. Several data fusion techniques have been introduced to cope with this problem, based on combining PPG signals with inertial sensor data. Until now, both commercial and reasearch solutions are computationally efficient but not very robust, or strongly dependent on hand-tuned parameters, which leads to poor generalization performance. In this work, we tackle these limitations by proposing a computationally lightweight yet robust deep learning-based approach for PPG-based HR estimation. Specifically, we derive a diverse set of Temporal Convolutional Networks for HR estimation, leveraging Neural Architecture Search. Moreover, we also introduce ActPPG, an adaptive algorithm that selects among multiple HR estimators depending on the amount of MAs, to improve energy efficiency. We validate our approaches on two benchmark datasets, achieving as low as 3.84 beats per minute of Mean Absolute Error on PPG-Dalia, which outperforms the previous state of the art. Moreover, we deploy our models on a low-power commercial microcontroller (STM32L4), obtaining a rich set of Pareto optimal solutions in the complexity vs. accuracy space.

REFERENCES

  1. [1] 2020. Neurosky. Retrieved from http://neurosky.com/wp-content/uploads/2016/06/TOF-side-by-side-competitor-comparison.pdf.Google ScholarGoogle Scholar
  2. [2] Abadi Martín, Agarwal Ashish, Barham Paul, Brevdo Eugene, Chen Zhifeng, Citro Craig, Corrado Greg S., Davis Andy, Dean Jeffrey, Devin Matthieu, Ghemawat Sanjay, Goodfellow Ian, Harp Andrew, Irving Geoffrey, Isard Michael, Jia Yangqing, Jozefowicz Rafal, Kaiser Lukasz, Kudlur Manjunath, Levenberg Josh, Mané Dandelion, Monga Rajat, Moore Sherry, Murray Derek, Olah Chris, Schuster Mike, Shlens Jonathon, Steiner Benoit, Sutskever Ilya, Talwar Kunal, Tucker Paul, Vanhoucke Vincent, Vasudevan Vijay, Viégas Fernanda, Vinyals Oriol, Warden Pete, Wattenberg Martin, Wicke Martin, Yu Yuan, and Zheng Xiaoqiang. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. Retrieved from https://www.tensorflow.org/.Google ScholarGoogle Scholar
  3. [3] Al-Eidan Rasha M., Al-Khalifa Hend, and Al-Salman Abdul Malik. 2018. A review of wrist-worn wearable: Sensors, models, and challenges. J. Sens. 2018 (2018).Google ScholarGoogle Scholar
  4. [4] Apple. [n.d.]. Apple Watch Series. Retrieved from https://www.apple.com/lae/watch/.Google ScholarGoogle Scholar
  5. [5] Arunkumar K. R. and Bhaskar M.. 2020. Robust de-noising technique for accurate heart rate estimation using wrist-type PPG signals. IEEE Sens. J. 20, 14 (2020), 79807987.Google ScholarGoogle Scholar
  6. [6] Bai Shaojie, Kolter J. Zico, and Koltun Vladlen. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271. Retrieved from https://arxiv.org/abs/1803.01271.Google ScholarGoogle Scholar
  7. [7] BiosignalsPLUX. [n.d.]. Respiban Professional 2019. Retrieved from https://biosignalsplux.com/products/wearables/respiban-pro.html.Google ScholarGoogle Scholar
  8. [8] Biswas Dwaipayan, Everson Luke, Liu Muqing, Panwar Madhuri, Verhoef Bram-Ernst, Patki Shrishail, Kim Chris H., Acharyya Amit, Hoof Chris Van, Konijnenburg Mario, et al. 2019. CorNET: Deep learning framework for PPG-based heart rate estimation and biometric identification in ambulant environment. IEEE Trans. Biomed. Circ. Syst. 13, 2 (2019), 282291.Google ScholarGoogle Scholar
  9. [9] Biswas D., Simões-Capela N., Hoof C. Van, and Helleputte N. Van. 2019. Heart rate estimation from wrist-worn photoplethysmography: A review. IEEE Sens. J. 19, 16 (2019), 65606570. Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Burrello Alessio, Benatti Simone, Schindler Kaspar Anton, Benini Luca, and Rahimi Abbas. 2020. An ensemble of hyperdimensional classifiers: Hardware-friendly short-latency seizure detection with automatic iEEG electrode selection. IEEE J. Biomed. Health Inf. 25, 4 (2020), 935–946.Google ScholarGoogle Scholar
  11. [11] Burrello Alessio, Schindler Kaspar Anton, Benini Luca, and Rahimi Abbas. 2020. Hyperdimensional computing with local binary patterns: One-shot learning for seizure onset detection and identification of ictogenic brain regions from short-time iEEG recordings. IEEE Trans. Biomed. Eng. 67, 2 (2020), 601613.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Cai Han, Zhu Ligeng, and Han Song. 2018. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv:1812.00332. Retrieved from https://arxiv.org/abs/1812.00332.Google ScholarGoogle Scholar
  13. [13] Castaneda Denisse, Esparza Aibhlin, Ghamari Mohammad, Soltanpur Cinna, and Nazeran Homer. 2018. A review on wearable photoplethysmography sensors and their potential future applications in health care. Int. J. Biosens. Bioelectr. 4, 4 (2018), 195202.Google ScholarGoogle Scholar
  14. [14] Chang Xiangmao, Li Gangkai, Xing Guoliang, Zhu Kun, and Tu Linlin. 2021. DeepHeart: A deep learning approach for accurate heart rate estimation from PPG signals. ACM Trans. Sen. Netw. 17, 2, Article 14 (Jan. 2021), 18 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Chung Heewon, Ko Hoon, Lee Hooseok, and Lee Jinseok. 2020. Deep learning for heart rate estimation from reflectance photoplethysmography with acceleration power spectrum and acceleration intensity. IEEE Access 8 (2020), 6339063402.Google ScholarGoogle Scholar
  16. [16] Chung Heewon, Lee Hooseok, and Lee Jinseok. 2018. Finite state machine framework for instantaneous heart rate validation using wearable photoplethysmography during intensive exercise. IEEE J. Biomed. Health Inform. 23, 4 (2018), 15951606.Google ScholarGoogle Scholar
  17. [17] Conti Francesco. 2020. Technical Report: NEMO DNN Quantization for Deployment Model. arxiv:2004.05930 [cs.LG]. Retrieved from https://arxiv.org/abs/2004.05930.Google ScholarGoogle Scholar
  18. [18] Daghero Francesco, Burrello Alessio, Pagliari Daniele Jahier, Benini Luca, Macii Enrico, and Poncino Massimo. 2020. Energy-efficient adaptive machine learning on IoT end-nodes with class-dependent confidence. In Proceedings of the International Conference on Electronics Circuits and Systems (ICECS’20). IEEE, 14.Google ScholarGoogle Scholar
  19. [19] Empatica. [n.d.]. E4 Wristband 2014. Retrieved from https://www.empatica.com/en-eu/research/e4/.Google ScholarGoogle Scholar
  20. [20] Fitbit. [n.d.]. Fitbit Charge 4. Retrieved from https://www.fitbit.com/global/us/products/trackers/charge4.Google ScholarGoogle Scholar
  21. [21] Ge Z., Prasad P. W. C., Costadopoulos N., Alsadoon Abeer, Singh A. K., and Elchouemi A.. 2016. Evaluating the accuracy of wearable heart rate monitors. In Proceedings of the 2nd International Conference on Advances in Computing, Communication, & Automation (ICACCA’16). IEEE, 16.Google ScholarGoogle Scholar
  22. [22] Gordon Ariel, Eban Elad, Nachum Ofir, Chen Bo, Wu Hao, Yang Tien-Ju, and Choi Edward. 2018. Morphnet: Fast & simple resource-constrained structure learning of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 15861595.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Huang Nicholas and Selvaraj Nandakumar. 2020. Robust PPG-based ambulatory heart rate tracking algorithm. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 59295934.Google ScholarGoogle Scholar
  24. [24] Ingolfsson T. M., Hersche M., Wang X., Kobayashi N., Cavigelli L., and Benini L.. 2020. EEG-TCNet: An accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC’20). 29582965. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ioffe Sergey and Szegedy Christian. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167. Retrieved from https://arxiv.org/abs/1502.03167.Google ScholarGoogle Scholar
  26. [26] Pagliari Daniele Jahier, Macii Enrico, and Poncino Massimo. 2018. Dynamic bit-width reconfiguration for energy-efficient deep learning hardware. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED’18). ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Jan Hao-Yu, Chen Mei-Fen, Fu Tieh-Cheng, Lin Wen-Chen, Tsai Cheng-Lun, and Lin Kang-Ping. 2019. Evaluation of coherence between ECG and PPG derived parameters on heart rate variability and respiration in healthy volunteers with/without controlled breathing. J. Med. Biol. Eng. 39, 5 (2019), 783795.Google ScholarGoogle Scholar
  28. [28] Lea Colin, Vidal Rene, Reiter Austin, and Hager Gregory D.. 2016. Temporal convolutional networks: A unified approach to action segmentation. In Proceedings of the European Conference on Computer Vision. Springer, 4754.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Lin Ji, Chen Wei-Ming, Lin Yujun, Gan Chuang, Han Song, et al. 2020. Mcunet: Tiny deep learning on iot devices. In Advances in Neural Information Processing Systems, Vol. 33.Google ScholarGoogle Scholar
  30. [30] Mashhadi Mahdi Boloursaz, Asadi Ehsan, Eskandari Mohsen, Kiani Shahrzad, and Marvasti Farokh. 2015. Heart rate tracking using wrist-type photoplethysmographic (PPG) signals during physical exercise with simultaneous accelerometry. IEEE Sign. Process. Lett. 23, 2 (2015), 227231.Google ScholarGoogle Scholar
  31. [31] Microelectornics ST. [n.d.]. STM32L476. Retrieved from https://www.st.com/resource/en/datasheet/stm32l476je.pdf.Google ScholarGoogle Scholar
  32. [32] Mocerino L. and Calimera A.. 2020. Fast and accurate inference on microcontrollers with boosted cooperative convolutional neural networks (BC-Net). IEEE Trans. Circ. Syst. I: Regul. Pap. (2020), 112. Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Nelson Benjamin W., Low Carissa A., Jacobson Nicholas, Areán Patricia, Torous John, and Allen Nicholas B.. 2020. Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research. NPJ Dig. Med. 3, 1 (2020), 19.Google ScholarGoogle Scholar
  34. [34] Neuneier Ralph and Zimmermann Hans Georg. 1998. How to train neural networks. In Neural Networks: Tricks of the Trade. Springer, 373423.Google ScholarGoogle Scholar
  35. [35] Park Eunhyeok, Kim Dongyoung, Kim Soobeom, Kim Yong-Deok, Kim Gunhee, Yoon Sungroh, and Yoo Sungjoo. 2015. Big/little deep neural network for ultra low power inference. In Proceedings of the International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS’15). IEEE, 124132.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Reiss Attila, Indlekofer Ina, Schmidt Philip, and Laerhoven Kristof Van. 2019. Deep PPG: Large-scale heart rate estimation with convolutional neural networks. Sensors 19, 14 (2019), 3079.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Ren Lei, Liu Yuxin, Wang Xiaokang, Lü Jinhu, and Deen M Jamal. 2021. Cloud-edge based lightweight temporal convolutional networks for remaining useful life prediction in IIoT. IEEE IoT J. 8, 16 (2021), 12578–12587.Google ScholarGoogle Scholar
  38. [38] Risso Matteo, Burrello Alessio, Pagliari Daniele Jahier, Benatti Simone, Macii Enrico, Benini Luca, and Poncino Massimo. 2021. Robust and energy-efficient PPG-based heart-rate monitoring. In Proceedings of the International Symposium on Circuit and Systems (ISCAS’21). IEEE.Google ScholarGoogle Scholar
  39. [39] Rocha Leandro Giacomini, Biswas Dwaipayan, Verhoef Bram-Ernst, Bampi Sergio, Hoof Chris Van, Konijnenburg Mario, Verhelst Marian, and Helleputte Nick Van. 2020. Binary corNET: Accelerator for HR estimation from wrist-PPG. IEEE Trans. Biomed. Circ. Syst. 14, 4 (2020), 715726.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Salehizadeh Seyed, Dao Duy, Bolkhovsky Jeffrey, Cho Chae, Mendelson Yitzhak, and Chon Ki H.. 2016. A novel time-varying spectral filtering algorithm for reconstruction of motion artifact corrupted heart rate signals during intense physical activities using a wearable photoplethysmogram sensor. Sensors 16, 1 (2016), 10.Google ScholarGoogle Scholar
  41. [41] Schäck Tim, Muma Michael, and Zoubir Abdelhak M.. 2017. Computationally efficient heart rate estimation during physical exercise using photoplethysmographic signals. In Proceedings of the 25th European Signal Processing Conference (EUSIPCO’17). IEEE, 24782481.Google ScholarGoogle Scholar
  42. [42] Shin Hang Sik, Lee Chungkeun, and Lee Myoungho. 2009. Adaptive threshold method for the peak detection of photoplethysmographic waveform. Comput. Biol. Med. 39, 12 (2009), 11451152.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Shyam A., Ravichandran Vignesh, Preejith S. P., Joseph Jayaraj, and Sivaprakasam Mohanasankar. 2019. PPGnet: Deep network for device independent heart rate estimation from photoplethysmogram. In Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 18991902.Google ScholarGoogle Scholar
  44. [44] Sviridova Nina and Sakai Kenshi. 2015. Human photoplethysmogram: New insight into chaotic characteristics. Chaos Solitons Fract. 77 (2015), 5363.Google ScholarGoogle Scholar
  45. [45] Tamura Toshiyo, Maeda Yuka, Sekine Masaki, and Yoshida Masaki. 2014. Wearable photoplethysmographic sensors-past and present. Electronics 3, 2 (2014), 282302.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Tan Mingxing, Chen Bo, Pang Ruoming, Vasudevan Vijay, Sandler Mark, Howard Andrew, and Le Quoc V.. 2019. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 28202828.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Temko Andriy. 2017. Accurate heart rate monitoring during physical exercises using PPG. IEEE Trans. Biomed. Eng. 64, 9 (2017), 20162024.Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Yeole Anjali S. and Kalbande Dhananjay R.. 2016. Use of internet of things (IoT) in healthcare: A survey. In Proceedings of the ACM Symposium on Women in Research 2016. 7176.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Yuan Ming and Lin Yi. 2006. Model selection and estimation in regression with grouped variables. J. Roy. Stat. Soc.: Ser. B 68, 1 (2006), 4967.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Zanghieri Marcello, Benatti Simone, Burrello Alessio, Kartsch Victor, Conti Francesco, and Benini Luca. 2020. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Trans. Biomed. Circ. Syst. 14, 2 (2020), 244–256.Google ScholarGoogle Scholar
  51. [51] Zhang Xiang, Yao Lina, Wang Xianzhi, Monaghan Jessica, Mcalpine David, and Zhang Yu. 2019. A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv:1905.04149. Retrieved from https://arxiv.org/abs/1905.04149.Google ScholarGoogle Scholar
  52. [52] Zhang Yangsong, Liu Benyuan, and Zhang Zhilin. 2015. Combining ensemble empirical mode decomposition with spectrum subtraction technique for heart rate monitoring using wrist-type photoplethysmography. Biomed. Sign. Process. Contr. 21 (2015), 119125.Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Zhang Zhilin. 2015. Photoplethysmography-based heart rate monitoring in physical activities via joint sparse spectrum reconstruction. IEEE Trans. Biomed. Eng. 62, 8 (2015), 19021910.Google ScholarGoogle Scholar
  54. [54] Zhang Zhilin, Pi Zhouyue, and Liu Benyuan. 2014. TROIKA: A general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Trans. Biomed. Eng. 62, 2 (2014), 522531.Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Zhou Menglian and Selvaraj Nandakumar. 2020. Heart rate monitoring using sparse spectral curve tracing. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 53475352.Google ScholarGoogle Scholar

Index Terms

  1. Embedding Temporal Convolutional Networks for Energy-efficient PPG-based Heart Rate Monitoring

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Computing for Healthcare
      ACM Transactions on Computing for Healthcare  Volume 3, Issue 2
      April 2022
      292 pages
      ISSN:2691-1957
      EISSN:2637-8051
      DOI:10.1145/3505188
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 3 March 2022
      • Accepted: 1 September 2021
      • Revised: 1 August 2021
      • Received: 1 March 2021
      Published in health Volume 3, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!