RFWash: a weakly supervised tracking of hand hygiene technique

ABSTRACT

Each year, hundreds of thousands of people contract Healthcare Associated Infections (HAIs). Poor hand hygiene compliance among healthcare workers is thought to be the leading cause of HAIs and methods were developed to measure compliance. Surprisingly, human observation is still considered the gold standard for measuring compliance by World Health Organization (WHO). Moreover, no automated solutions exist for monitoring hand hygiene techniques, such as "how to hand rub" technique by WHO. In this paper, we introduce RFWash; the first radio-based device-free system for monitoring Hand Hygiene (HH) technique. On the technical level, HH gestures are performed back-to-back in a continuous sequence and pose a significant challenge to conventional two-stage gesture detection and recognition approaches. We propose a deep model that can be trained on unsegmented naturally-performed HH gesture sequences. RFWash evaluation demonstrates promising results for tracking HH gestures, achieving gesture error rate of < 8% when trained on 10-second segments, which reduces manual labelling overhead by ≈ 67% compared to fully supervised approach. The work is a step towards practical RF sensing that can reliably operate inside future healthcare facilities.

References

1. 2009. WHO Guidelines on Hand Hygiene in Health Care. Published by World Health Organisation. Retrieved from: whqlibdoc.who.int/publications/009.pdf. (2009).
2. 2016. Healthcare-associated infections: data and statistics. Published by Center for Disease Control and Prevention. (2016).
3. 2020 (accessed July 1, 2020). Ti IWR1443BOOST. "https://www.ti.com/tool/IWR1443BOOST". (2020 (accessed July 1, 2020)).
4. Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. Wigest: A ubiquitous wifi-based gesture recognition system. In 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, 1472--1480.
5. Sari Awwad, Sanjay Tarvade, Massimo Piccardi, and David J Gattas. 2019. The use of privacy-protected computer vision to measure the quality of healthcare worker hand hygiene. International Journal for Quality in Health Care 31, 1 (2019), 36--42.
6. John M Boyce. 2011. Measuring healthcare worker hand hygiene activity: current practices and emerging technologies. Infection control and hospital epidemiology 32, 10 (2011), 1016--1028.
7. Edward Chou, Matthew Tan, Cherry Zou, Michelle Guo, Albert Haque, Arnold Milstein, and Li Fei-Fei. 2018. Privacy-Preserving Action Recognition for Smart Hospitals using Low-Resolution Depth Images. Machine Learning for Health (ML4H) Workshop at NeurIPS 2018 (2018).
8. I. R. Daniels and B. I. Rees. 1999. Handwashing: simple, but effective. Annals of the Royal College of Surgeons of England 81, 2 (Mar 1999), 117--118. https://www.ncbi.nlm.nih.gov/pubmed/10364970 10364970[pmid].
9. Lijie Fan, Tianhong Li, Yuan Yuan, and Dina Katabi. 2020. In-Home Daily-Life Captioning Using Radio Signals. In European Conference on Computer Vision (ECCV) 2020.
10. Piyali Goswami, Sandeep Rao, Sachin Bharadwaj, and Amanda Nguyen. 2019. Real-time multi-gesture recognition using 77 GHz FMCW MIMO single chip radar. In 2019 IEEE International Conference on Consumer Electronics (ICCE). IEEE, 1--4.
11. Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning. ACM, 369--376.
12. Albert Haque, Michelle Guo, Alexandre Alahi, Serena Yeung, Zelun Luo, Alisha Rege, Jeffrey Jopling, Lance Downing, William Beninati, Amit Singh, et al. 2017. Towards vision-based smart hospitals: A system for tracking and monitoring hand hygiene compliance. Proceedings of Machine Learning Research (18--19 Aug 2017).
13. Chengkun Jiang, Junchen Guo, Yuan He, Meng Jin, Shuai Li, and Yunhao Liu. 2020. mmVib: micrometer-level vibration measurement with mmwave radar. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--13.
14. Yen Lee Angela Kwok, Michelle Callard, and Mary-Louise McLaws. 2015. An automated hand hygiene training system improves hand hygiene technique but not compliance. American journal of infection control 43, 8 (2015), 821--825.
15. Yen Lee Angela Kwok, Craig P Juergens, and Mary-Louise McLaws. 2016. Automated hand hygiene auditing with and without an intervention. American journal of infection control 44, 12 (2016), 1475--1480.
16. Hong Li, Shishir Chawla, Richard Li, Sumeet Jain, Gregory D Abowd, Thad Starner, Cheng Zhang, and Thomas Plotz. 2018. Wristwash: towards automatic handwashing assessment using a wrist-worn device. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. 132--139.
17. Yunan Li, Qiguang Miao, Kuan Tian, Yingying Fan, Xin Xu, Rui Li, and Jianfeng Song. 2016. Large-scale gesture recognition with a fusion of rgb-d data based on the c3d model. In 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 25--30.
18. Bingbin Liu, Michelle Guo, Edward Chou, Rishab Mehra, Serena Yeung, N Lance Downing, Francesca Salipur, Jeffrey Jopling, Brandi Campbell, Kayla Deru, et al. 2018. 3D Point Cloud-Based Visual Prediction of ICU Mobility Care Activities. In Machine Learning for Healthcare Conference. 17--29.
19. Hu Liu, Sheng Jin, and Changshui Zhang. 2018. Connectionist temporal classification with maximum entropy regularization. In Advances in Neural Information Processing Systems. 831--841.
20. David Fernández Llorca, Ignacio Parra, Miguel Ángel Sotelo, and Gerard Lacey. 2011. A vision-based system for automatic hand washing quality assessment. Machine Vision and Applications 22, 2 (2011), 219--234.
21. Chris Xiaoxuan Lu, Stefano Rosa, Peijun Zhao, Bing Wang, Changhao Chen, John A. Stankovic, Niki Trigoni, and Andrew Markham. 2020. See through Smoke: Robust Indoor Mapping with Low-Cost MmWave Radar. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services (MobiSys '20). 14--27.
22. Yongsen Ma, Gang Zhou, Shuangquan Wang, Hongyang Zhao, and Woosub Jung. 2018. SignFi: Sign language recognition using WiFi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 23.
23. AR Marra and MB Edmond. 2014. New technologies to monitor healthcare worker hand hygiene. Clinical Microbiology and Infection 20, 1 (2014), 29--33.
24. Maryanne McGuckin and John Govednik. 2015. A review of electronic hand hygiene monitoring: considerations for hospital management in data collection, healthcare worker supervision, and patient perception. Journal of Healthcare Management 60, 5 (2015), 348--361.
25. Pedro Melgarejo, Xinyu Zhang, Parameswaran Ramanathan, and David Chu. 2014. Leveraging directional antenna capabilities for fine-grained gesture recognition. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 541--551.
26. Marco Mercuri, Ilde Rosa Lorato, Yao-Hong Liu, Fokko Wieringa, Chris Van Hoof, and Tom Torfs. 2019. Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor. Nature Electronics 2, 6 (2019), 252--262.
27. Pavlo Molchanov, Xiaodong Yang, Shalini Gupta, Kihwan Kim, Stephen Tyree, and Jan Kautz. 2016. Online detection and classification of dynamic hand gestures with recurrent 3d convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4207--4215.
28. Avishek Patra, Philipp Geuer, Andrea Munari, and Petri Mähönen. 2018. mm-Wave Radar Based Gesture Recognition: Development and Evaluation of a Low-Power, Low-Complexity System. In Proceedings of the 2nd ACM Workshop on Millimeter Wave Networks and Sensing Systems. 51--56.
29. Lisa L Pineles, Daniel J Morgan, Heather M Limper, Stephen G Weber, Kerri A Thom, Eli N Perencevich, Anthony D Harris, and Emily Landon. 2014. Accuracy of a radiofrequency identification (RFID) badge system to monitor hand hygiene behavior during routine clinical activities. American journal of infection control 42, 2 (2014), 144--147.
30. Xingshuai Qiao, Tao Shan, Ran Tao, Xia Bai, and Juan Zhao. 2019. Separation of human micro-Doppler signals based on short-time fractional Fourier transform. IEEE Sensors Journal 19, 24 (2019), 12205--12216.
31. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45, 11 (1997), 2673--2681.
32. Muhammad Shahzad and Shaohu Zhang. 2018. Augmenting User Identification with WiFi Based Gesture Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 134.
33. Karly A Smith, Clément Csech, David Murdoch, and George Shaker. 2018. Gesture recognition using mm-wave sensor for human-car interface. IEEE Sensors Letters 2, 2 (2018), 1--4.
34. Jocelyn A Srigley, Colin D Furness, G Ross Baker, and Michael Gardam. 2014. Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study. BMJ Qual Saf 23, 12 (2014), 974--980.
35. Yonglong Tian, Guang-He Lee, Hao He, Chen-Yu Hsu, and Dina Katabi. 2018. RF-based fall monitoring using convolutional neural networks. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1--24.
36. Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision. 4489--4497.
37. Raghav H Venkatnarayan, Griffin Page, and Muhammad Shahzad. 2018. Multiuser gesture recognition using WiFi. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 401--413.
38. Aditya Virmani and Muhammad Shahzad. 2017. Position and orientation agnostic gesture recognition using wifi. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 252--264.
39. Saiwen Wang, Jie Song, Jaime Lien, Ivan Poupyrev, and Otmar Hilliges. 2016. Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 851--860.
40. Xin Yang, Jian Liu, Yingying Chen, Xiaonan Guo, and Yucheng Xie. 2020. MU-ID: Multi-user Identification Through Gaits Using Millimeter Wave Radios. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 2589--2598.
41. Yinggang Yu, Dong Wang, Run Zhao, and Qian Zhang. 2019. RFID based real-time recognition of ongoing gesture with adversarial learning. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems. 298--310.
42. Shigeng Zhang, Chengwei Yang, Xiaoyan Kui, Jianxin Wang, Xuan Liu, and Song Guo. 2019. ReActor: Real-time and Accurate Contactless Gesture Recognition with RFID. In 2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). IEEE, 1--9.
43. Zhenyuan Zhang, Zengshan Tian, and Mu Zhou. 2018. Latern: Dynamic continuous hand gesture recognition using FMCW radar sensor. IEEE Sensors Journal 18, 8 (2018), 3278--3289.
44. Henry Zhong, Salil S Kanhere, and Chun Tung Chou. 2016. WashInDepth: Lightweight Hand Wash Monitor Using Depth Sensor. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. ACM, 28--37.
45. Y. Zou, J. Xiao, J. Han, K. Wu, Y. Li, and L. M. Ni. 2017. GRfid: A Device-Free RFID-Based Gesture Recognition System. IEEE Transactions on Mobile Computing 16, 2 (Feb 2017), 381--393.

Index Terms

• Human-centered computing

  • Ubiquitous and mobile computing