skip to main content
research-article

The Impact of Walking and Resting on Wrist Motion for Automated Detection of Meals

Published:30 September 2020Publication History
Skip Abstract Section

Abstract

This article considers detecting eating in free-living humans by tracking wrist motion. We are specifically interested in the effect of secondary activities that people conduct while simultaneously eating, such as walking, watching television, or working. These secondary activities cause wrist motions that obfuscate those associated with eating, increasing the difficulty of detecting periods of eating. We collected a large dataset of 4,680 hours of wrist motion from 351 participants during free living. Participants reported secondary activities in 72% of meals. Analysis of wrist motion data revealed that the wrist was resting 12.8% of the time during self-reported meals compared to only 6.8% of the time in a cafeteria dataset, whereas walking motion was found 5.5% of the time during meals in free living compared to 0% in a cafeteria. Augmenting an eating detection classifier to include walking and resting detection improved accuracy from 74% to 77% on our free-living dataset (t[353] = 7.86, p < 0.001). Although eating detection could be improved using more sophisticated machine learning methods or sensor modalities, all approaches would be affected by secondary activities, as they affect the labeling of data itself. Our work suggests that future work should collect detailed ground truth on secondary activities being conducted during eating, as these activities could hold insights into when an eating activity starts or stops in the absence of video-based ground truth.

References

  1. Rawan Alharbi, Tammy Stump, Nilofar Vafaie, Angela Pfammatter, Bonnie Spring, and Nabil Alshurafa. 2018. I can’t be myself: Effects of wearable cameras on the capture of authentic behavior in the wild. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 90.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Nabil Alshurafa, Haik Kalantarian, Mohammad Pourhomayoun, Jason J. Liu, Shruti Sarin, Behnam Shahbazi, and Majid Sarrafzadeh. 2015. Recognition of nutrition intake using time-frequency decomposition in a wearable necklace using a piezoelectric sensor. IEEE Sensors Journal 15, 7 (2015), 3909--3916.Google ScholarGoogle ScholarCross RefCross Ref
  3. Oliver Amft, Martin Kusserow, and Gerhard Troster. 2007. Automatic identification of temporal sequences in chewing sounds. In Proceedings of the 2007 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’07). IEEE, Los Alamitos, CA, 194--201.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Oliver Amft and Gerhard Tröster. 2008. Recognition of dietary activity events using on-body sensors. Artificial Intelligence in Medicine 42, 2 (2008), 121--136.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, and Gregory Abowd. 2017. EarBit: Using wearable sensors to detect eating episodes in unconstrained environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 37.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Shengie Bi, Tao Wang, Nicole Tobias, Josephine Nordrum, Shang Wang, George Halvorsen, Sougata Sen, et al. 2018. Auracle: Detecting eating episodes with an ear-mounted sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 92.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Box Inc.2019. Box: Content Management, Workflow, and Collaboration. Retrieved December 2, 2019 from https://www.box.com/home [Online; accessed 2-December-2019].Google ScholarGoogle Scholar
  8. Keum San Chun, Sarnab Bhattacharya, and Edison Thomaz. 2018. Detecting eating episodes by tracking jawbone movements with a non-contact wearable sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 4.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Guang Deng and L. W. Cahill. 1993. An adaptive Gaussian filter for noise reduction and edge detection. In Proceedings of the 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference. IEEE, Los Alamitos, CA, 1615--1619.Google ScholarGoogle Scholar
  10. Bo Dong, Subir Biswas, Robert Gernhardt, and Janik Schlemminger. 2013. A mobile food intake monitoring system based on breathing signal analysis. In Proceedings of the 8th International Conference on Body Area Networks. 165--168.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Yujie Dong, Jenna Scisco, Mike Wilson, Eric Muth, and Adam Hoover. 2014. Detecting periods of eating during free-living by tracking wrist motion. IEEE Journal of Biomedical and Health Informatics 18, 4 (2014), 1253--1260.Google ScholarGoogle Scholar
  12. Abul Doulah, Xin Yang, Jason Parton, Janine A. Higgins, Megan A. McCrory, and Edward Sazonov. 2018. The importance of field experiments in testing of sensors for dietary assessment and eating behavior monitoring. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’18). IEEE, Los Alamitos, CA, 5759--5762.Google ScholarGoogle Scholar
  13. Muhammad Farooq, Abul Doulah, Jason Parton, Megan A. McCrory, Janine A. Higgins, and Edward Sazonov. 2019. Validation of sensor-based food intake detection by multicamera video observation in an unconstrained environment. Nutrients 11, 3 (2019), 609.Google ScholarGoogle Scholar
  14. Juan M. Fontana, Muhammad Farooq, and Edward Sazonov. 2014. Automatic ingestion monitor: A novel wearable device for monitoring of ingestive behavior. IEEE Transactions on Biomedical Engineering 61, 6 (2014), 1772--1779.Google ScholarGoogle ScholarCross RefCross Ref
  15. Sebastian O. H. Madgwick, Andrew J. L. Harrison, and Ravi Vaidyanathan. 2011. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Proceedings of the 2011 IEEE International Conference on Rehabilitation Robotics. IEEE, Los Alamitos, CA, 1--7.Google ScholarGoogle Scholar
  16. Ryan Mattfeld, Elliot Jesch, and Adam Hoover. 2017. A new dataset for evaluating pedometer performance. In Proceedings of the 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’17). IEEE, Los Alamitos, CA, 865--869.Google ScholarGoogle Scholar
  17. Mark Mirtchouk, Drew Lustig, Alexandra Smith, Ivan Ching, Min Zheng, and Samantha Kleinberg. 2017. Recognizing eating from body-worn sensors: Combining free-living and laboratory data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 85.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Eric Muth, Ryan Mattfeld, and Adam Hoover. 2017. Bites as a unit of measurement. In Advances in the Assessment of Dietary Intake.CRC Press, Boca Raton, FL, 149--161.Google ScholarGoogle Scholar
  19. Eric R. Muth and Adam Hoover. 2017. Designing mobile health technologies for self-monitoring: The bite counter as a case study. In Mobile Health. Springer, 101--120.Google ScholarGoogle Scholar
  20. Raul I. Ramos-Garcia, Eric R. Muth, John N. Gowdy, and Adam W. Hoover. 2015. Improving the recognition of eating gestures using intergesture sequential dependencies. IEEE Journal of Biomedical and Health Informatics 19, 3 (2015), 825--831.Google ScholarGoogle ScholarCross RefCross Ref
  21. James N. Salley, Adam W. Hoover, Michael L. Wilson, and Eric R. Muth. 2016. Comparison between human and bite-based methods of estimating caloric intake. Journal of the Academy of Nutrition and Dietetics 116, 10 (2016), 1568--1577.Google ScholarGoogle Scholar
  22. Edward Sazonov, Stephanie Schuckers, Paulo Lopez-Meyer, Oleksandr Makeyev, Nadezhda Sazonova, Edward L. Melanson, and Michael Neuman. 2008. Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiological Measurement 29, 5 (2008), 525.Google ScholarGoogle ScholarCross RefCross Ref
  23. Giovanni Schiboni and Oliver Amft. 2018. Sparse natural gesture spotting in free living to monitor drinking with wrist-worn inertial sensors. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. ACM, New York, NY, 140--147.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Surya Sharma and Adam Hoover. 2019. A study on linear acceleration of the wrist during free-living. In Proceedings of the 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’19). IEEE, Los Alamitos, CA.Google ScholarGoogle Scholar
  25. Yiru Shen, James Salley, Eric Muth, and Adam Hoover. 2017. Assessing the accuracy of a wrist motion tracking method for counting bites across demographic and food variables. IEEE Journal of Biomedical and Health Informatics 21, 3 (2017), 599--606.Google ScholarGoogle Scholar
  26. Julius Sim and Chris C. Wright. 2005. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy 85, 3 (2005), 257--268.Google ScholarGoogle ScholarCross RefCross Ref
  27. TDK Invensense. 2018. MPU-9150. Retrieved August 31, 2020 from https://www.invensense.com/products/motion-tracking/9-axis/mpu-9150.Google ScholarGoogle Scholar
  28. Edison Thomaz, Irfan Essa, and Gregory D. Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, New York, NY, 1029--1040.Google ScholarGoogle Scholar
  29. Edison Thomaz, Aman Parnami, Jonathan Bidwell, Irfan Essa, and Gregory D. Abowd. 2013. Technological approaches for addressing privacy concerns when recognizing eating behaviors with wearable cameras. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, New York, NY, 739--748.Google ScholarGoogle Scholar
  30. U.S. Department of Labor. 2016. Bureau of Labor Statistics Data for Activity: Eating and Drinking. Retrieved August 31, 2020 from https://data.bls.gov/timeseries/TUU10101AA01013237.Google ScholarGoogle Scholar
  31. Wikipedia Contributors. 2019. Confusion Matrix—Table of Confusion. Retrieved August 6, 2019 from https://en.wikipedia.org/wiki/Confusion_matrix#Table_of_confusion [Online; accessed 6-August-2019].Google ScholarGoogle Scholar
  32. Wikipedia Contributors. 2019. Precision and Recall—Imbalanced Data. Retrieved August 6, 2019 from https://en.wikipedia.org/wiki/Precision_and_recall#Imbalanced_Data [Online; accessed 6-August-2019].Google ScholarGoogle Scholar
  33. Rui Zhang and Oliver Amft. 2017. Monitoring chewing and eating in free-living using smart eyeglasses. IEEE Journal of Biomedical and Health Informatics 22, 1 (2017), 23--32.Google ScholarGoogle Scholar
  34. Shibo Zhang, Rawan Alharbi, Matthew Nicholson, and Nabil Alshurafa. 2017. When generalized eating detection machine learning models fail in the field. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2017 ACM International Symposium on Wearable Computers. ACM, New York, NY, 613--622.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. The Impact of Walking and Resting on Wrist Motion for Automated Detection of Meals

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!