Abstract
This article considers detecting eating in free-living humans by tracking wrist motion. We are specifically interested in the effect of secondary activities that people conduct while simultaneously eating, such as walking, watching television, or working. These secondary activities cause wrist motions that obfuscate those associated with eating, increasing the difficulty of detecting periods of eating. We collected a large dataset of 4,680 hours of wrist motion from 351 participants during free living. Participants reported secondary activities in 72% of meals. Analysis of wrist motion data revealed that the wrist was resting 12.8% of the time during self-reported meals compared to only 6.8% of the time in a cafeteria dataset, whereas walking motion was found 5.5% of the time during meals in free living compared to 0% in a cafeteria. Augmenting an eating detection classifier to include walking and resting detection improved accuracy from 74% to 77% on our free-living dataset (t[353] = 7.86, p < 0.001). Although eating detection could be improved using more sophisticated machine learning methods or sensor modalities, all approaches would be affected by secondary activities, as they affect the labeling of data itself. Our work suggests that future work should collect detailed ground truth on secondary activities being conducted during eating, as these activities could hold insights into when an eating activity starts or stops in the absence of video-based ground truth.
- Rawan Alharbi, Tammy Stump, Nilofar Vafaie, Angela Pfammatter, Bonnie Spring, and Nabil Alshurafa. 2018. I can’t be myself: Effects of wearable cameras on the capture of authentic behavior in the wild. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 90.Google Scholar
Digital Library
- Nabil Alshurafa, Haik Kalantarian, Mohammad Pourhomayoun, Jason J. Liu, Shruti Sarin, Behnam Shahbazi, and Majid Sarrafzadeh. 2015. Recognition of nutrition intake using time-frequency decomposition in a wearable necklace using a piezoelectric sensor. IEEE Sensors Journal 15, 7 (2015), 3909--3916.Google Scholar
Cross Ref
- Oliver Amft, Martin Kusserow, and Gerhard Troster. 2007. Automatic identification of temporal sequences in chewing sounds. In Proceedings of the 2007 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’07). IEEE, Los Alamitos, CA, 194--201.Google Scholar
Digital Library
- Oliver Amft and Gerhard Tröster. 2008. Recognition of dietary activity events using on-body sensors. Artificial Intelligence in Medicine 42, 2 (2008), 121--136.Google Scholar
Digital Library
- Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, and Gregory Abowd. 2017. EarBit: Using wearable sensors to detect eating episodes in unconstrained environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 37.Google Scholar
Digital Library
- Shengie Bi, Tao Wang, Nicole Tobias, Josephine Nordrum, Shang Wang, George Halvorsen, Sougata Sen, et al. 2018. Auracle: Detecting eating episodes with an ear-mounted sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 92.Google Scholar
Digital Library
- Box Inc.2019. Box: Content Management, Workflow, and Collaboration. Retrieved December 2, 2019 from https://www.box.com/home [Online; accessed 2-December-2019].Google Scholar
- Keum San Chun, Sarnab Bhattacharya, and Edison Thomaz. 2018. Detecting eating episodes by tracking jawbone movements with a non-contact wearable sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 4.Google Scholar
Digital Library
- Guang Deng and L. W. Cahill. 1993. An adaptive Gaussian filter for noise reduction and edge detection. In Proceedings of the 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference. IEEE, Los Alamitos, CA, 1615--1619.Google Scholar
- Bo Dong, Subir Biswas, Robert Gernhardt, and Janik Schlemminger. 2013. A mobile food intake monitoring system based on breathing signal analysis. In Proceedings of the 8th International Conference on Body Area Networks. 165--168.Google Scholar
Digital Library
- Yujie Dong, Jenna Scisco, Mike Wilson, Eric Muth, and Adam Hoover. 2014. Detecting periods of eating during free-living by tracking wrist motion. IEEE Journal of Biomedical and Health Informatics 18, 4 (2014), 1253--1260.Google Scholar
- Abul Doulah, Xin Yang, Jason Parton, Janine A. Higgins, Megan A. McCrory, and Edward Sazonov. 2018. The importance of field experiments in testing of sensors for dietary assessment and eating behavior monitoring. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’18). IEEE, Los Alamitos, CA, 5759--5762.Google Scholar
- Muhammad Farooq, Abul Doulah, Jason Parton, Megan A. McCrory, Janine A. Higgins, and Edward Sazonov. 2019. Validation of sensor-based food intake detection by multicamera video observation in an unconstrained environment. Nutrients 11, 3 (2019), 609.Google Scholar
- Juan M. Fontana, Muhammad Farooq, and Edward Sazonov. 2014. Automatic ingestion monitor: A novel wearable device for monitoring of ingestive behavior. IEEE Transactions on Biomedical Engineering 61, 6 (2014), 1772--1779.Google Scholar
Cross Ref
- Sebastian O. H. Madgwick, Andrew J. L. Harrison, and Ravi Vaidyanathan. 2011. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Proceedings of the 2011 IEEE International Conference on Rehabilitation Robotics. IEEE, Los Alamitos, CA, 1--7.Google Scholar
- Ryan Mattfeld, Elliot Jesch, and Adam Hoover. 2017. A new dataset for evaluating pedometer performance. In Proceedings of the 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’17). IEEE, Los Alamitos, CA, 865--869.Google Scholar
- Mark Mirtchouk, Drew Lustig, Alexandra Smith, Ivan Ching, Min Zheng, and Samantha Kleinberg. 2017. Recognizing eating from body-worn sensors: Combining free-living and laboratory data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 85.Google Scholar
Digital Library
- Eric Muth, Ryan Mattfeld, and Adam Hoover. 2017. Bites as a unit of measurement. In Advances in the Assessment of Dietary Intake.CRC Press, Boca Raton, FL, 149--161.Google Scholar
- Eric R. Muth and Adam Hoover. 2017. Designing mobile health technologies for self-monitoring: The bite counter as a case study. In Mobile Health. Springer, 101--120.Google Scholar
- Raul I. Ramos-Garcia, Eric R. Muth, John N. Gowdy, and Adam W. Hoover. 2015. Improving the recognition of eating gestures using intergesture sequential dependencies. IEEE Journal of Biomedical and Health Informatics 19, 3 (2015), 825--831.Google Scholar
Cross Ref
- James N. Salley, Adam W. Hoover, Michael L. Wilson, and Eric R. Muth. 2016. Comparison between human and bite-based methods of estimating caloric intake. Journal of the Academy of Nutrition and Dietetics 116, 10 (2016), 1568--1577.Google Scholar
- Edward Sazonov, Stephanie Schuckers, Paulo Lopez-Meyer, Oleksandr Makeyev, Nadezhda Sazonova, Edward L. Melanson, and Michael Neuman. 2008. Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiological Measurement 29, 5 (2008), 525.Google Scholar
Cross Ref
- Giovanni Schiboni and Oliver Amft. 2018. Sparse natural gesture spotting in free living to monitor drinking with wrist-worn inertial sensors. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. ACM, New York, NY, 140--147.Google Scholar
Digital Library
- Surya Sharma and Adam Hoover. 2019. A study on linear acceleration of the wrist during free-living. In Proceedings of the 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’19). IEEE, Los Alamitos, CA.Google Scholar
- Yiru Shen, James Salley, Eric Muth, and Adam Hoover. 2017. Assessing the accuracy of a wrist motion tracking method for counting bites across demographic and food variables. IEEE Journal of Biomedical and Health Informatics 21, 3 (2017), 599--606.Google Scholar
- Julius Sim and Chris C. Wright. 2005. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy 85, 3 (2005), 257--268.Google Scholar
Cross Ref
- TDK Invensense. 2018. MPU-9150. Retrieved August 31, 2020 from https://www.invensense.com/products/motion-tracking/9-axis/mpu-9150.Google Scholar
- Edison Thomaz, Irfan Essa, and Gregory D. Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, New York, NY, 1029--1040.Google Scholar
- Edison Thomaz, Aman Parnami, Jonathan Bidwell, Irfan Essa, and Gregory D. Abowd. 2013. Technological approaches for addressing privacy concerns when recognizing eating behaviors with wearable cameras. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, New York, NY, 739--748.Google Scholar
- U.S. Department of Labor. 2016. Bureau of Labor Statistics Data for Activity: Eating and Drinking. Retrieved August 31, 2020 from https://data.bls.gov/timeseries/TUU10101AA01013237.Google Scholar
- Wikipedia Contributors. 2019. Confusion Matrix—Table of Confusion. Retrieved August 6, 2019 from https://en.wikipedia.org/wiki/Confusion_matrix#Table_of_confusion [Online; accessed 6-August-2019].Google Scholar
- Wikipedia Contributors. 2019. Precision and Recall—Imbalanced Data. Retrieved August 6, 2019 from https://en.wikipedia.org/wiki/Precision_and_recall#Imbalanced_Data [Online; accessed 6-August-2019].Google Scholar
- Rui Zhang and Oliver Amft. 2017. Monitoring chewing and eating in free-living using smart eyeglasses. IEEE Journal of Biomedical and Health Informatics 22, 1 (2017), 23--32.Google Scholar
- Shibo Zhang, Rawan Alharbi, Matthew Nicholson, and Nabil Alshurafa. 2017. When generalized eating detection machine learning models fail in the field. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2017 ACM International Symposium on Wearable Computers. ACM, New York, NY, 613--622.Google Scholar
Digital Library
Index Terms
The Impact of Walking and Resting on Wrist Motion for Automated Detection of Meals
Recommendations
NeckSense: A Multi-Sensor Necklace for Detecting Eating Activities in Free-Living Conditions
We present the design, implementation, and evaluation of a multi-sensor, low-power necklace, NeckSense, for automatically and unobtrusively capturing fine-grained information about an individual's eating activity and eating episodes, across an entire ...
Detecting Eating Episodes by Tracking Jawbone Movements with a Non-Contact Wearable Sensor
Eating is one of the most fundamental human activities, and because of the important role it plays in our lives, it has been extensively studied. However, an objective and usable method for dietary intake tracking remains unrealized despite numerous ...
Gait-based authentication using a wrist-worn device
MOBIQUITOUS 2016: Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and ServicesEvery individual has a distinctive way of walking. For this reason gait can be a key element of biometric techniques aimed at authenticating and/or identifying the user of a wearable device. This paper presents a lightweight method that uses the ...






Comments